The goal for this push request is to normalize the construction and use
 of internal, external, and admin URLs. While extending Kolla-ansible
 to enable a more flexible method to manage external URLs, we noticed
 that the same URL was constructed multiple times in different parts
 of the code. This can make it difficult for people that want to work
 with these URLs and create inconsistencies in a large code base with
 time. Therefore, we are proposing here the use of
 "single Kolla-ansible variable" per endpoint URL, which facilitates
 for people that are interested in overriding/extending these URLs.

As an example, we extended Kolla-ansible to facilitate the "override"
of public (external) URLs with the following standard
"<component/serviceName>.<companyBaseUrl>".
Therefore, the "NAT/redirect" in the SSL termination system (HAproxy,
HTTPD or some other) is done via the service name, and not by the port.
This allows operators to easily and automatically create more friendly
 URL names. To develop this feature, we first applied this patch that
 we are sending now to the community. We did that to reduce the surface
  of changes in Kolla-ansible.

Another example is the integration of Kolla-ansible and Consul, which
we also implemented internally, and also requires URLs changes.
Therefore, this PR is essential to reduce code duplicity, and to
facility users/developers to work/customize the services URLs.

Change-Id: I73d483e01476e779a5155b2e18dd5ea25f514e93
Signed-off-by: Rafael Weingärtner <rafael@apache.org>

Change-Id: If90c2dfd32c8bc50671f6dd38e5a82b434c07151
Depends-On: https://review.opendev.org/#/c/720338

This patch introduces an optional backend encryption for the Nova API
service. When used in conjunction with enabling TLS for service API
endpoints, network communcation will be encrypted end to end, from
client through HAProxy to the Nova service.

Change-Id: I48e1540b973016079d5686b328e82239dcffacfd
Partially-Implements: blueprint add-ssl-internal-network

Change-Id: I173669bdf92b1f2ea98907ba16808ca3c914944c

This patch introduces a global keep alive timeout value for services
that leverage httpd + wsgi to handle http/https requests. The default
value is one minute.

Change-Id: Icf7cb0baf86b428a60a7e9bbed642999711865cd
Partially-Implements: blueprint add-ssl-internal-network

There is a time once every 2 years when ubuntu team releases new LTS
release. And then UCA joins with binary packages for current OpenStack
development cycle.

It is this time for Ubuntu 20.04 'focal'.

Includes CI fix to pass:

[CI] Temporarily block new Ansible

The proper fix [1] needs fixing older branches before newer.
This one allows to fix CI first, in the usual order.

To revert after [1] gets merged in all relevant branches.

[1] https://review.opendev.org/745648

Old-Change-Id: Ifbd37d8addd4322773118e2e9d46494741a8ae66
Related-Bug: #1891145

Depends-on: https://review.opendev.org/#/c/738994/


Change-Id: Ib8b70ee40ec2d19509cc84c0f530612f81907721
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Previously we mounted /etc/timezone if the kolla_base_distro is debian
or ubuntu. This would fail prechecks if debian or ubuntu images were
deployed on CentOS. While this is not a supported combination, for
correctness we should fix the condition to reference the host OS rather
than the container OS, since that is where the /etc/timezone file is
located.

Change-Id: Ifc252ae793e6974356fcdca810b373f362d24ba5
Closes-Bug: #1882553

Add trove-guestagent.conf templates for trove-guestagent service.
Default the Guest Agent config file to be injected during instance creation.

Change-Id: Id0750b84fef8e19658b27f8ae16a857e1394216e

This patch is a continuation of
I6a174468bd91d214c08477b93c88032a45c137be for the nova-cell role, which
was missed.

The Castellan (Barbican client) has different parameters to control
the used CA file.
This patch uses them.
Moreover, this aligns Barbican with other services by defaulting
its client config to the internal endpoint.

See also [1].

[1] https://bugs.launchpad.net/castellan/+bug/1876102

Closes-Bug: #1886615

Change-Id: I056f3eebcf87bcbaaf89fdd0dc1f46d143db7785

Glance role copies glance-image-import.conf
when enabled to allow configuration of
glance interoperable image import. Property
protection can be enabled and file is copied.

Change-Id: I5106675da5228a5d7e630871f0882269603e6571
Closesl-Bug: #1889272
Signed-off-by: nikparasyr <nik.parasyr@protonmail.com>

Change-Id: I59a15186bbe931efd8d99a990a3ceafbd264e1df

Change-Id: Ib08544a265fe1e0d599a6243cb9d38ed9a7769e1

These two roles were missing 'stop' and 'deploy-containers',
respectively.

Change-Id: Iaf434be9baf1973323bb177fad799aea39210fba

Masakari was introduced parallelly to deploy-containers action and
so we missed to add this functionality to it.

Change-Id: Ibef198d20d481bc92b38af786cdf0292b246bb12
Closes-Bug: #1889611

With an incorrectly named section, whatever's defined in here is
actually ignored which can result in unexpected behaviour.

Closes-Bug: 1889455

Change-Id: Ib2e2b53e9a3c0e62a2e997881c0cd1f92acfb39c
Signed-off-by: Nick Jones <nick@dischord.org>

Option "network_label_regex" from group "DEFAULT" is
deprecated for removal.

Change-Id: I8aab2ca322159e61e4cbe9a5b30825a71a991e7e

If not running containerised chrony, we need to check that host
has its own means of system clock synchronization.

Change-Id: I31b3e9ed625d63a4bf82c674593522268c20ec4c
Partial-Bug: #1885689

Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the check-containers.yml include, the
included file only has a single task, so the overhead of skipping this
task will not be greater than the overhead of the task import. It
therefore makes sense to switch to use import_tasks there.

Partially-Implements: blueprint performance-improvements

Change-Id: I65d911670649960708b9f6a4c110d1a7df1ad8f7

This change disables services in the Prometheus openstack-exporter
if they are not enabled in the deployment. Such behaviour allows
to avoid warnings and errors in the log files and keep the
log file contents clean and informative.

Change-Id: I4dcac976620a5f451e3d273183199aefe400994a

Change-Id: I2e22ec47f644de2f1509a0111c9e1fffe8da0a1a

Docker is manipulating iptables rules by default to provide network
isolation, and this might cause problems if the host already has an
iptables-based firewall.

This change introduces docker_disable_default_iptables_rules to
disable the iptables manipulation by putting "iptables: false" [1] to
daemon.json

For better defaults, this feature will be enabled by default in
Victoria.

[1] https://docs.docker.com/network/iptables/

Closes-Bug: #1849275

Change-Id: I165199fc98fb98f227f2a20284e1bab03ef65b5b

This fixes an issue where multiple Grafana instances would race
to bootstrap the Grafana DB. The following changes are made:

- Only start additional Grafana instances after the DB has been
  configured.

- During upgrade, don't allow old instances to run with an
  upgraded DB schema.

Change-Id: I3e0e077ba6a6f43667df042eb593107418a06c39
Closes-Bug: #1888681

This ensures that when using automatic Kafka topic creation, with more than one
node in the Kafka cluster, all partitions in the topic are automatically
replicated. When a single node goes down in a >=3 node cluster, these topics will
continue to accept writes providing there are at least two insync replicas.

In a two node cluster, no failures are tolerated. In a three node cluster, only a
single node failure is tolerated. In a larger cluster the configuration may need
manual tuning.

This configuration follows advice given here:

[1] https://docs.cloudera.com/documentation/kafka/1-2-x/topics/kafka_ha.html#xd_583c10bfdbd326ba-590cb1d1-149e9ca9886--6fec__section_d2t_ff2_lq

Closes-Bug: #1888522

Change-Id: I7d38c6ccb22061aa88d9ac6e2e25c3e095fdb8c3

fluentd logs currently to stdout, which is known to produce big docker logs
in /var/lib/docker. This change makes fluentd to log to /var/log/kolla/fluentd.

Closes-Bug: #1888852
Change-Id: I8fe0e54cb764a26d26c6196cef68aadc6fd57b90

This reverts commit 8fc86893.

It appears that it is still necessary to wait for ironic to be up, otherwise inspector may fail to start:

The baremetal service for 192.0.2.10:None exists but does not have any supported versions.

Change-Id: Ibc8314c91113618ce9e92b8933a63eba3cf3bbe1

octavia deploy failed due to mount a empyt directroy into container

Change-Id: Ifd95126da59f649b02ab39c0b209df4750bdcfce

From Ussuri, if CA certificates are copied into
/etc/kolla/certificates/ca/, these should be copied into all containers.
This is not being done for masakari currently.

Additionally, we are not setting the [DEFAULT] nova_ca_certificates_file
option in masakari.conf. This depends on masakari bug 1873736 being
fixed to work.

This change fixes these issues.

Change-Id: I9a3633f58e5eb734fa32edc03a3022a500761bbb
Closes-Bug: #1888655

Some CloudKitty API responses include a Location header using http
instead of https. Seen with `openstack rating module enable hashmap`.

Change-Id: I11158bbfd2006e3574e165b6afc9c223b018d4bc
Closes-Bug: #1888544

global file glance_backend_file parameters not take effect

Closes-Bug: #1888501

Change-Id: I3afd117633a84d342effb6baadf16fa42c16776c

A "@type copy" statement is already present at the beginning of each
match element, so extra "type copy" are not needed. They are causing the
following warnings in fluentd logs:

[warn]: parameter 'type' in <match syslog.local0.**>
[warn]: parameter 'type' in <match syslog.local1.**>

This commit also harmonizes indentation of the Monasca config block.

Change-Id: I779c2b942d007acbdd43d999f2fc0cdc131d431f
Related-Bug: #1885873

Change-Id: Ia134a518b63bb59cfad631cc488181f5245160e6

we should clone freezer code before run bootstray,
otherwise, the directory /opt/stack/freezer which is empty will
mount into freezer_api container.

Closes-Bug: #1888242

Change-Id: I7c22dd380fd5b1dff7b421109c4ae37bab11834a

Option "trove_auth_url/os_region_name" from group "DEFAULT" is deprecated.
Use option "auth_url/region_name" from group service_credentials

Change-Id: I15d6891582c92c7fc813f280a2b47ebaaca77eba

This makes use of udev rules to make it smarter and override
host-level packages settings.
Additionally, this masks Ubuntu-only service that is another
pain point in terms of /dev/kvm permissions.
Fingers crossed for no further surprises.

Change-Id: I61235b51e2e1325b8a9b4f85bf634f663c7ec3cc
Closes-bug: #1681461

Switch to the Confluent Kafka client in all remaining Python based
Monasca services. This should allow us to later un-pin the Kafka
messaging version for Monasca.

Change-Id: I42bc78ffe304ba21c448c2e08b025e93a70ddb44

Co-Authored-By: Doug Szumski <doug@stackhpc.com>
Closes-Bug: #1884090
Depends-On: https://review.opendev.org/#/c/736768

Change-Id: If2d0dd1739e484b14e3c15a185a236918737b0ab

The Castellan (Barbican client) has different parameters to control
the used CA file.
This patch uses them.
Moreover, this aligns Barbican with other services by defaulting
its client config to the internal endpoint.

See also [1].

[1] https://bugs.launchpad.net/castellan/+bug/1876102

Closes-Bug: #1886615

Change-Id: I6a174468bd91d214c08477b93c88032a45c137be

planned removal

Change-Id: Ib37ea4d42f82096a682cebc724c45c9dd39c8b47

The bug is fixed[1], releated task is unncessary.

[1]: https://storyboard.openstack.org/#!/story/2006393

Depends-On: Ib62ca3ee4626084e5e9b90e93e4fa97938023457
Change-Id: I2553c3c4a6d3c82405c68c52db2e7585477b1dff

The nova-cell role sets the following sysctls on compute hosts, which
require the br_netfilter kernel module to be loaded:

    net.bridge.bridge-nf-call-iptables
    net.bridge.bridge-nf-call-ip6tables

If it is not loaded, then we see the following errors:

    Failed to reload sysctl:
    sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
    sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-ip6tables: No such file or directory

Loading the br_netfilter module resolves this issue.

Typically we do not see this since installing Docker and configuring it
to manage iptables rules causes the br_netfilter module to be loaded.
There are good reasons [1] to disable Docker's iptables management
however, in which case we are likely to hit this issue.

This change loads the br_netfilter module in the nova-cell role for
compute hosts.

[1] https://bugs.launchpad.net/kolla-ansible/+bug/1849275



Co-Authored-By: Dincer Celik <hello@dincercelik.com>

Change-Id: Id52668ba8dab460ad4c33fad430fc8611e70825e