The default value of logging_context_format_string and
logging_user_identity_format has changed [1][2][3].
This is to correct fluentd to use the same format.

[1]https://opendev.org/openstack/oslo.log/commit/ebdee7f39920ad5b4268ee296952432b0d41a375
[2]https://opendev.org/openstack/oslo.log/commit/7ed065f38df0595881d07eb91cec9e5c07cb38be

[3]https://review.opendev.org/c/openstack/oslo.log/+/838185

Closes-Bug: #1872220

Change-Id: Ide0079ef4d28e3a3fa60288a74b936531c52ce60

opensearch-dashboards now use a dedicated user

Depends-On: https://review.opendev.org/c/openstack/kolla/+/883941
Change-Id: I6908f52f824a97a5a4a2bead92b7b2e5cdebdb9f

Depends-On: https://review.opendev.org/c/openstack/neutron/+/878535
Change-Id: I05d8b29b59a7de76da488f68775547a8f0f11d0f

We limit to 2.14.2 due to a regression in ansible-core [1] that breaks
conditional include_task loops in handlers. This is used for controlled
restarts of MariaDB and RabbitMQ.

[1]: https://github.com/ansible/ansible/commit/65366f663de7d044f42ae6dd53368fd4c1f88b35



Change-Id: I57425680a4cdbf0daeb9b2cc35920f1b933aa4a8
Co-Authored-By: Michal Nasiadka <michal@stackhpc.com>

As of I3629b84d3255a8fe9d8a7cea8c6131d7c40899e8 nova
now requires the service_user section to be configured
to address CVE-2023-2088. This change adds
the service user section to the nova.conf template in
the nova and nova-cell roles.

Related-Bug: #2004555
Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: I2189dafca070accfd8efcd4b8cc4221c6decdc9f
(cherry picked from commit a77ea13ef1991543df29b7eea14b1f91ef26f858)
(cherry picked from commit 03c12abbcc107bfec451f4558bc97d14facae01c)
(cherry picked from commit cb105dc293ff1cdb11ab63fa3e3bf39fd17e0ee0)
(cherry picked from commit efe6650d09441b02cf93738a94a59723d84c5b19)

The flags ``--db-nb-pid`` and ``--db-sb-pid`` are corected to be
``--db-nb-pidfile`` and ``--db-sb-pidfile`` respectively. See here for
reference:
https://github.com/ovn-org/ovn/blob/6c6a7ad1c64a21923dc9b5bea7069fd88bcdd6a8/utilities/ovn-ctl#L1045

Closes-Bug: #2018436
Change-Id: Ic1e8768374566eb2198302807ecc644a19cd3062

This patch add a way to choose container engine inside tool and test
scripts. This is in preparation for Podman introduction but still
leaves Docker as default container engine.

Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I395d2bdb0dfb4b325b6ad197c8893c8a0f768324

New openstack collection modules have changed output
dicts/variable names - adapting to that.

Also changing octavia amphora image to focal, since bionic
hasn't been rebuilt since May 2021.

Closes-Bug: #2012255
Change-Id: Icf38a52472d02ef7d69bcd3716afb16e859d44a2

Sometimes passwords, URLs and other values of the variables can
contain special symbols, for example the dollar sign, using these
values can lead to unpredictable attempts of the variable expansions
in the Bash scripts, such as openrc file, so we need to use single
quotes for all variables values.

Change-Id: Ib2aabadd0ffd6a8dc2591245f29b4478e03d92fc
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

Closes-Bug: #1919464
Change-Id: I64a5185627f98bcaf471ed8ccf0fd8f314769404
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

When using externally managed certificates, according to [1],
one should set `kolla_externally_managed_cert: yes` and ensure
that the certificates are in the correct place.

However, RabbitMQ precheck still expects the certificates to be
available on the controller node. This is incorrect.

Fix by not running the tasks in question when `kolla_externally_managed_cert: yes`

[1] https://docs.openstack.org/kolla-ansible/latest/admin/tls.html



Closes-Bug: 1999081
Related-Bug: 1940286
Signed-off-by: Magnus Lööf <magnus.loof@basalt.se>
Change-Id: I9f845a7bdf5055165e199ab1887ed3ccbfb9d808

This reverts commit 9867060b.

Reason for revert: seems this broke some jobs

Change-Id: I1ca81214ece403351c0a522ea05bf07802e4c4c0

Change-Id: I1d8021a1bc780449e3ca96183c6f4abaed17b382

The step "Run Nova upgrade checks" needs to be excluded from check mode.

Related-Bug: #2002657
Change-Id: Ida582606b03d1896a6c58d4b324d088dc3b71852

Currently, the process of enabling RabbitMQ HA with the variable
``om_enable_rabbitmq_high_availbility`` requires some manual steps to
migrate from transient to mirrored queues. In preparation for setting
this variable to ``True`` by default, this adds a precheck that will
fail if a system is currently running non-mirrored queues and
``om_enable_rabbitmq_high_availbility`` is set to ``True``.

Includes a helpful message informing the operator of their choice.
Either follow the manual procedure to migrate the queues described in
the docs, or set ``om_enable_rabbitmq_high_availbility`` to ``False``.

The RabbitMQ HA section of the reference docs is updated to include
these instructions.

Change-Id: Ic5e64998bd01923162204f7bb289cc110187feec

This patch introduces distributed lock for masakari-api
service when handle the concurrent notifications for the same
host failure from multiple masakari-hostmonitor services.

Change-Id: I46985202dc8da22601357eefe2727599e7a413e5

Closes-Bug: #2016627

Change-Id: I5ae1d911c5df423e0b70dab306709320083b7b69

The dib_env_vars variable in the Bifrost's dib.yml file can contain
the DIB_BLOCK_DEVICE_CONFIG environment variable which is always the
Multiline-YAML data. By default, the format of the data is not
preserved while the configuration is merged and saved for the
bifrost-deploy container.

This is because Ansible uses the PyYAML library which has a default
80 symbol string length limit. The official Ansible documentation [1]
recommends using to_yaml or to_nice_yaml filters with width parameter.
This change adds the same ability to the merge_yaml Ansible plugin.

1. https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#formatting-data-yaml-and-json



The related change for the diskimage-builder to solve the issue with
incorrect data provided by Kolla-Ansible is also provided:
I3b74ede69eb064ad813a9108ec68a228e549e8bb

Closes-Bug: #2014980
Related-Bug: #2014981
Change-Id: Id79445c0311916ac6c1beb3986e14f652ee5a63c
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

Change-Id: Ibc9cc91f64b0450de3cae6e2830b4ff2c52c0395

With the addition of the variable
`om_enable_rabbitmq_high_availability`, this feature in the upgrade
task should be brought back. It is also now used in the deploy task. The
`ha-all` policy is cleared only when
`om_enable_rabbitmq_high_availability` is set to `false`.

Change-Id: Ia056aa40e996b1f0fed43c0f672466c7e4a2f547

Puts the RabbitMQ node into maintenance mode before restarting the
container. This will make the node shutdown less disruptive. For details
on what maintenance mode does, see:
https://www.rabbitmq.com/upgrade.html#maintenance-mode

Change-Id: Ia61573f3fb95fe8fcde6b789ca77ef5b45fe0a65

Since RMQ 3.8 we can use rolling upgrade [1].

Depends-On: https://review.opendev.org/c/openstack/kolla/+/872393

[1]: https://www.rabbitmq.com/upgrade.html#rolling-upgrades

Change-Id: If6a7c6c12d9226a2406728108b3c87b3485ac55f

Add checking for container readiness before create sasl user

Closes-Bug: #2015589
Change-Id: Ic650ba6be1f192e3cbeaa94de3d00507636c1c92

Closes-Bug: #2005119
Change-Id: I542f7ae19b4400355b04854f42a1d1802a6efeea

This change fixes the output 'module_args' information of the plugins
'merge_configs' and 'merge_yaml' when Ansible is executed in maximum
verbose mode. Now all the plugin options are displayed instead of
standard 'copy' plugin options only.

Also, this change contains fixes already applied in the Kayobe
project to improve and synchronize the code of the plugins between
projects.

Change-Id: Ie2d9a0501fe29bfd854eb31258f282b197855948
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

Since CVE-2022-29404 is fixed [1,2] the default value for the
LimitRequestBody directive in the Apache HTTP Server has been changed
from 0 (unlimited) to 1 GiB. This limits the size of images (for
example) uploaded in Horizon. This change add the ability to
configure the limit.

1. https://access.redhat.com/articles/6975397
2. https://ubuntu.com/security/CVE-2022-29404



Closes-Bug: #2012588
Change-Id: I4cd9dd088cbcf38ff6f8d188ebcc56be7d9ea1c9
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

When upgrading Nova, we sometimes hit an error where an old hypervisor
that hasn’t been upgraded recently (for example due to broken hardware)
is preventing Nova API from starting properly. This can be detected
using the tool ``nova-status upgrade check`` to make sure that there are
no ``nova-compute`` that are older than N-1 releases. This is already
used in the Kolla Ansible upgrade task for Nova. However, this task uses
the current ``nova-api`` container, so computes which will be too old
after the upgrade are not caught.

This patch changes Kolla Ansible so that the upgraded ``nova-api`` image
is used to run the upgrade checks, allowing computes that will be too
old to be detected before the upgrades are performed.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/878744



Closes-Bug: #1957080
Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>
Change-Id: I3a899411001834a0c88e37f45a756247ee11563d

Change-Id: I54e68a3002d69f7b1be2704259c6a072f81aa586

Closes-Bug: #2012654

Change-Id: I9735b4409a48d80851cbc26a9edbf370af1d45bf

Following ideas here:
https://wiki.openstack.org/wiki/Large_Scale_Configuration_Rabbit

Make sure old messages with no consumer are dropped after the message
TTL of 10 mins, longer than the 1 min RPC timeout.
Also ensure queues expire after an hour of inactivity, so queues from
removed nodes or renamed nodes don't grow over time.

Change-Id: Ifb28ac68b6328adb604a7474d01e5f7a47b2e788

Adds two new flags to alter behaviour in RabbitMQ:
    * `rabbitmq_message_ttl_ms`, which lets you set a TTL on messages.
    * `rabbitmq_queue_expiry_ms`, which lets you set an expiry time on queues.
See https://www.rabbitmq.com/ttl.html for more information on both.

Change-Id: I51ca37ffbb1bb5c07f2d39873f0f33ca20263f2a

Changes the default value of `rabbitmq-ha-promote-on-shutdown` to
`"always"`.

We are seeing issues with RabbitMQ automatically recovering when nodes
are restarted. https://www.rabbitmq.com/ha.html#cluster-shutdown

Rather than waiting for operator interventions, it is better we allow
recovery to happen, even if that means we may loose some messages.
A few failed and timed out operations is better than a totaly broken
cloud. This is achieved using ha-promote-on-shutdown=always.

Note, when a node failure is detected, this is already the default
behaviour from 3.7.5 onwards:
https://www.rabbitmq.com/ha.html#promoting-unsynchronised-mirrors

Related-Bug: #1954925
Change-Id: I484a81163f703fa27112df22473d657e2a9ab964

Change-Id: I1649a389bdc3977b936402c3ce3e55056d74ba08

With the parameter ``mariadb_datadir_volume`` it is possible
to use a directory as volume for the mariadb service. By default,
a volume named mariadb is used (the previous default).

Change-Id: Ic61fe981825c5fa6f50e53c9555b6a102f42f522

With the new ``neutron_ovn_availability_zones`` parameter it is possible
to define network availability zones for OVN. Further details can be found
in the Neutron OVN documentation:
https://docs.openstack.org/neutron/latest/admin/ovn/availability_zones.html#how-to-configure-it

Change-Id: I203e0d400a3218d0b4a41f2a948207032c4febec

deployments

This allows services to work with etcd when coordination is enabled
for TLS internal deployments. Without this fix, we fail to connect to
etcd with the coordination backend and the service itself crashes.

Change-Id: I0c1d6b87e663e48c15a846a2774b0a4531a3ca68

Partial-Bug: #1930109

Change-Id: I383b2b5a139d24a419145473b66a34c06e32060a

Fourth part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


which was suggested to be split into smaller patches.

This commit refactors select methods from DockerWorker class
into ContainerWorker class. New class contains Docker independent
methods also used in Podman introduction and is inteded as a
parent class for specific worker classes.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I2dd5920410dda053f2dfedc4e2666c56b1a7095a

For Masakari and HACluster to work properly, the hostnames used
in HACluster need to match with the hostnames used in Nova.

Change-Id: Iac917ef4471905caab591cd64eab379e150a8524

Previously, when running one of the following commands:

  kolla-ansible deploy --check
  kolla-ansible genconfig --check

deployment or configuration generation fails for various reasons.

MariaDB fails to lookup the existing cluster.

Keystone fails to generate cron config.

Nova-cell fails to get the cell settings.

Closes-Bug: #2002661
Change-Id: I5e765f498ae86d213d0a4379ca5d473db1499962