This patch is moving minsize and maxsize logrotate
options to defaults and use them in template.

Change-Id: Ic2dfc747faedef8a7a4bc5e61db7c0cb5bf93e88

Haproxy was renamed in [1].

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/770618

Change-Id: Ib2d7f0774fede570a8c4c315d83afd420c31da0b

Currently only operations done with default kolla_toolbox user are logged
to /var/log/kolla/ansible.log.

In order to fix logging, permissions to ansible.log must allow writing
for other users in kolla group - and then a separate patch will follow
to make custom ansible.cfg file usable by other toolbox users.

Partial-Bug: #1942846
Change-Id: I1be60ac7647b1a838e97f05f15ba5f0e39e8ae3c

This is required for libvirtd with cgroupsv2 (Debian Bullseye and
soon others).
Otherwise, device attachments simply fail.
The warning message suggests filtering will be disabled but it
actually just fails the action entirely.

Change-Id: Id1fbd49a31a6e6e51b667f646278b93897c05b21
Closes-Bug: #1941940

corrected nits from:
https://review.opendev.org/c/openstack/kolla-ansible/+/800068
https://review.opendev.org/c/openstack/kolla-ansible/+/803644

Change-Id: Ia30afd795067a36b132a8c75c72dd7c65d624a83

It was removed in [1] as part of cgroupsv2 cleanup.
However, the testing did not catch the fact that the legacy
cgroups behaviour was actually still breaking despite latest
Docker and setting to use host's cgroups namespace.

[1] 286a03ba

Closes-Bug: #1941706
Change-Id: I629bb9e70a3fd6bd1e26b2ca22ffcff5e9e8c731

In some situations it may be helpful to populate the fact cache on
demand. The 'kolla-ansible gather-facts' command may be used to do this.

One specific case where this may be helpful is when running kolla-ansible
with a --limit argument, since in that case hosts that match the limit
will gather facts for hosts that fall outside the limit. In the extreme
case of a limit that matches only one host, it will serially gather
facts for all other hosts. To avoid this issue, run 'kolla-ansible
gather-facts' without a limit to populate the fact cache in parallel
before running the required command with a limit.

Change-Id: I79db9bca23aa1bd45bafa7e7500a90de5a684593

to behave like it is most commonly expected - query Nova in the
same region.

Closes-Bug: #1939291
Change-Id: I584a83d352c747a799b5dab1d3b8159ba3805454

To follow best security practices and help fellow operators.

More details inline and in the linked bug report.

Closes-Bug: #1940547
Change-Id: Ide9e9009a6e272f20a43319f27d257efdf315f68

Sometimes, the registries may intermittently fail to deliver the
images. This is often seen in the CI, though it also happens with
production deployments, even those with internal registries and/or
registry mirrors - due to sheer load when trying to pull the
images from many hosts.

This patchs adds two new vars to control retry behaviour.
The default has been set to make users happier by default. :-)

Change-Id: I81ad7d8642654f8474f11084c6934aab40243d35

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>
Closes-Bug: #1940286
Change-Id: I647f8615e9fd0cc0db7c737ce4afbd1bdd0d40d4

This change enables the use of Docker healthchecks for
keystone-fernet container. It checks if "key 0" has
right permissions, and if rsync is able to distribute
keys to other keystones.

Implements: blueprint container-health-check
Change-Id: I17bea723d4109e869cd05d211f6f8e4653f46e17

This change enables the use of Docker healthchecks for
nova-spicehtml5proxy service.

Implements: blueprint container-health-check
Change-Id: I584c588c20781e6c6567429811aecf97967baea3

Certain overrides for rabbitmq may need to be set for `rabbitmqctl` in
kolla-toolbox aswell.
This commit allows to override `rabbitmq-env.conf` and `erl_inetrc` in
kolla-toolbox.

Change-Id: Idef6adcf9700f75a2db503444a8de093ee21a9c5

Closes-Bug: #1939883
Change-Id: Ica311acba445cccac1d20757ced6f15a064ebcaa

Kolla-ansible upgrade task is calling different
handlers as deploy task and these handlers are
missing healthcheck key. This patch is fixing
this.

Closes-Bug: #1939679
Change-Id: Id83d20bfd89c27ccf70a3a79938f428cdb5d40fc

We get a nice optimisation by using a filtered loop instead
of task skipping per service with 'when'.

Partially-Implements: blueprint performance-improvements
Change-Id: I8f68100870ab90cb2d6b68a66a4c97df9ea4ff52

This patch adds support for integrating Prometheus with Fluentd.
This can be used to extract useful information about the status
of Fluentd, such as output buffer capacity and logging rate,
and also to extract metrics from logs via custom Fluentd
configuration. More information can be found here in [1].

[1] https://docs.fluentd.org/monitoring-fluentd/monitoring-prometheus

Change-Id: I233d6dd744848ef1f1589a462dbf272ed0f3aaae

Change-Id: I0103d7ef55e6aebe043a582d36f1a2efa137f447

Missing comma in config.json template rendered invalid JSON with VMware
plugin agent.

Closes-Bug: #1939080

Change-Id: I3d0c6421e3da7e1e753b99cf87c32fc238a21523

Basically, there are three main installation scenario:

Scenario 1:
Ironic installation together with other openstack services
including keystone. In this case variable enable_keystone
is set to true and keystone service will be installed
together with ironic installation. It is possible realise this
scenario, no fix needed

Scenario 2:
Ironic installation with connection to already installed
keystone. In this scenario we have to set enable_keystone
to “No” to prevent from new keystone service installation
during the ironic installation process. But in other hand,
we need to have correct sections in ironic.conf to provide
all information needed to connect to existing keystone.
But all sections for keystone are added to ironic.conf only
if enable_keystone var is set to “Yes”. It isn’t possible
to realise this scenario. Proposed fix provide support for
this scenario, where multiple regions share the same
keystone service.

Scenario 3:
No keystone integration. Ironic don't connect to Keystone.
It is possible realise this scenario, no fix needed

Proposed solution also keep the default behaviour: if no
enable_keystone_integration is manually defined by default
it takes value of enable_keystone variable and all behaviour
is the same. But if we don't want to install keystone and
want to connect to existing one at the same time, it will be
possible to set enable_keystone var to “No”
(preventing keystone from installation) and at the same
time set ironic_enable_keystone_integration to Yes to allow
needed section appear in ironic.conf through templating.

Change-Id: I0c7e9a28876a1d4278fb2ed8555c2b08472864b9

Change-Id: Ib9ea83dd0019a4c4703e673a783c45ab07afe4e7

Change-Id: I0d7c7f47e6653cf2903589a9c86798a8c6404af5

This patch is fixing docker healthcheck for horizon
by changing value of horizon_listen_port, so
both apache's virtualhost and healthcheck will have
same correct port always. Also removing useless
apache's redirect as all redirects are done on
haproxy side.

Closes-Bug: #1933846
Change-Id: Ibb5ad1a5d1bbc74bcb62610d77852d8124c4a323

Kolla-ansible install python docker library in role/baremetal
to group/baremetal, because of this get container facts
for timesync checks is failing on deployment host.

This patch adding when conditional, so deployment host
will be skipped as there is no need to run timesync
checks.

Closes-Bug: #1933347
Change-Id: Ifefb9c74ee6a80cdbc458992d0196850ddfe7ffa

This trivial patch is setting "timeout tunnel" in haproxy's
configuration for spicehtml5proxy. This option extends time
when spice's websocket connection is closed, so spice will
not be freezed. Default value is set to 1h as it is in novnc.

Closes-Bug: #1938549
Change-Id: I3a5cd98ecf4916ebd0748e7c08111ad0e4dca0b2

Multiple inventories can now be passed to `kolla-ansible`.  This can be
useful to construct a common inventory that is shared between multiple
environments.

Change-Id: I2ac5d7851b310bea2ba362b353f18c592a0a6a2e

As mentioned in the Iced014acee7e590c10848e73feca166f48b622dc
commit message, in Ussuri+ we can use ``+sbwtdcpu none
+sbwtdio none`` as well. This is due to relying on RMQ-provided
erlang in version 23.x.

This change adds the extra arguments by default.
It should be backported down to Ussuri before we do a release with
Iced014acee7e590c10848e73feca166f48b622dc.

Change-Id: I32e247a6cb34d7f6763b544f247fd408dce2b3a2

Delete the "haproxy_single_service_listen.cfg.j2" template,
which has been replaced by "haproxy_single_service_split.cfg.j2"
and deprecated in the Victoria version

Change-Id: I3599f85afe9d3045820ea1ea70481ea2500e49ac

In Ussuri, nova stopped using separate Ceph keys for the volumes and vms
pools by default. Instead, we set ceph_nova_keyring to the value of
ceph_cinder_keyring by default, which is ceph.client.cinder.keyring.
This is in line with the Ceph OpenStack integration guide [1]. However,
the user used by nova to access the vms pool (ceph_nova_user) defaults
to nova, meaning that nova will still try to use a
ceph.client.nova.keyring, which probably does not exist. We did not see
this issue in CI, because we set ceph_nova_user to cinder.

This change fixes the issue by setting ceph_nova_user to the value of
ceph_cinder_user by default, which is cinder.

Closes-Bug: #1934145
Related-Bug: #1928690

[1] https://docs.ceph.com/en/latest/rbd/rbd-openstack/

Change-Id: I6aa8db2214e07906f1f3e035411fc80ba911a274

multiple external networks are supported by linuxbridge and OVS.
Currently the config template only works for OVS

Closes-Bug: #1863935
Change-Id: I9da331e007c25c4a760839c566831769a68507a9

Co-Authored-By: Boris Lukashev

Change-Id: I52eaf823ae84e01a09a6dcfcbffd7221ff8abfac
Closes-Bug: #1937911

In the Xena release, Ironic removed the iSCSI driver [1]. The
recommended driver is direct, which uses HTTP to transfer the disk
image. This requires an HTTP server, and the simplest option is to use
the one currently deployed when enable_ironic_ipxe is set to true. For
this reason, this patch always enables the HTTP server running on the
conductor.

iPXE is still enabled separately, since it cannot currently be used at
the same time as PXE.

[1] https://review.opendev.org/c/openstack/ironic/+/789382

Change-Id: I30c2ad2bf2957ac544942aefae8898cdc8a61ec6

The healthcheck checks for a process called httpd, but these distros
call it apache2.  This results in the ironic_ipxe container being marked
as unhealthy.

This change fixes the issue by making the process name distro dependent.

Change-Id: I0b0126e3071146e7f8593ba970ecbed65b36fcfa
Closes-Bug: #1937037

Since the Victoria release, manila-share.conf requires a glance section
for some drivers. This change adds the missing section.

It also uses the correct cinder_keystone_user variable to reference the
cinder user.

Closes-Bug: #1921935

Change-Id: Ib7ce4ed79c28456281087eb4156577f910c072e7

Backports: Wallaby, Victoria.

Change-Id: Ib9a5058d6290d805670001ac2b5a42630aeec2b2

Adds support for passing extra runtime options to cAdvisor.
By default new options disable exporting rarely useful metrics
and labels by cAdvisor. This helps reducing the load on Prometheus
and cAdvisor itself.

Change-Id: I81f3845d6cd03a70a0c8569f8d0ea421027df083

Currently, if you override docker_yum_url, the repo must contain a GPG
key at {{ docker_yum_url }}/gpg, despite the fact that the GPG key URL
can be overridden separately via docker_yum_gpgkey. This change uses
docker_yum_gpgkey consistently, avoiding the need to keep the key in the
repo.

Closes-Bug: #1934913
Change-Id: If8e6a02ce0760123f7b076c711727ef575965192

Remove tempest role as planned

Change-Id: If3cf073e88c83f670c867a49afe48845f9e81008

Ansible facts can have a large impact on the performance of the Ansible
control host. This patch introduces some control over which facts are
gathered (kolla_ansible_setup_gather_subset) and which facts are stored
(kolla_ansible_setup_filter). By default we do not change the default
values of these arguments to the setup module. The flexibility of these
arguments is limited, but they do provide enough for a large performance
improvement in a typical moderate to large OpenStack cloud.

In particular, the large complex dict fact for each interface has a
large effect, and on an OpenStack controller or hypervisor there may be
many virtual interfaces. We can use the kolla_ansible_setup_filter
variable to help:

    kolla_ansible_setup_filter: 'ansible_[!qt]*'

This causes Ansible to collect but not store facts matching that
pattern, which includes the virtual interface facts. Currently we are
not referencing other facts matching the pattern within Kolla Ansible.
Note that including the 'ansible_' prefix causes meta facts module_setup
and gather_subset to be filtered, but this seems to be the only way to
get a good match on the interface facts. To work around this, we use
ansible_facts rather than module_setup to detect whether facts exist in
the cache.

The exact improvement will vary, but has been reported to be as large as
18x on systems with many virtual interfaces.

For reference, here are some other tunings tried:

* Increased the number of forks (great speedup depending of the size of
  the deployment)
* Use `strategy = mitogen_linear` (cut processing time in half)
* Ansible caching (little speed up)
* SSH tunning (little speed up)

Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Closes-Bug: #1921538
Change-Id: Iae8ca4aae945892f1dc65e1b10381d2e26e88805