This commit adds the ironic-prometheus-exporter, following the
conventions used by the previously integrated exporters. '[The] Ironic
Prometheus Exporter is a Tool to expose hardware sensor data in the
Prometheus format through an HTTP endpoint.'[0]

Prometheus has been enabled in CI jobs to ensure test coverage.

[0] https://opendev.org/openstack/ironic-prometheus-exporter

Depends-On: https://review.opendev.org/c/openstack/kolla/+/874415

Change-Id: I6d421effd833d2e0524dd0b81736445c9a730ea9

Sets the variable ``om_enable_rabbitmq_high_availability`` to ``true``
by default. An upgrade will therefore require some manual steps to
migrate from transient to durable queues. Note that this will be
caught by this precheck:
https://review.opendev.org/c/openstack/kolla-ansible/+/880274

Also updates the CI upgrade jobs to perform this migration. This will
need to be removed in Caracal.

Related-Bug: #2031294

Change-Id: I26a70d4722aaa4663eced5f5337840474c7b961c

This command can be invoked with ``kolla-ansible rabbitmq-reset-state``.
This is primarily designed to be used when enabling HA queues[1].

As such, this also updates the RabbitMQ documentation to use this
command.

[1] https://docs.openstack.org/kolla-ansible/latest/reference/message-queues/rabbitmq.html#high-availability

Change-Id: I6ad95a3618fc1a34af56657ef99ef14dc979f17a

Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/892323

Change-Id: I11db700511233aa60229ee65d0cc96e46aafdf90

Change-Id: I42f9f182a2dab8563008e8b817ac58a69b72b062

Threads are the recommended way to scale CPU performance since HAProxy
1.8.

Official documentation says: « While "nbproc" historically used to be the only
way to use multiple processors, it also involved a number of shortcomings
related to the lack of synchronization between processes (health-checks, peers,
stick-tables, stats, ...) which do not affect threads. As such, any modern
configuration is strongly encouraged to migrate away from "nbproc" to
"nbthread". ».

Change-Id: I6f2e9d74e68703c8e0827e495945a75f020e1561

MariaDB bootstrap has a phase where the first MariaDB container
is running with Galera bootstrap - after a check that WSREP
is synced is successful - we restart the container.

The bootstrap container is named mariadb and running with
docker_restart_policy: "no" - the restarted container should be running
in systemd.

Before this patch the code created a systemd unit but it was initially
stopped - so stopping was always a success - and the container would be
killed with SIGKILL on removal (which obviously breaks MariaDB).

This patch also improves docker/systemd stops by waiting for real
unit/container stop and adds failing CI for containers that are
killed with signal 9.

Closes-Bug: #2029613

Change-Id: I0a03e509ce228a50e081fcab44d2b4831251190c

This change block access to the public /server-status url on all
http services exposed by HAProxy, also fixes an issue with Horizon
where 'Require all granted' open access to the /server-status in
the HAProxy-less configurations. Without this change the issue
affects only Ubuntu/Debian installations where mod_status in Apache2
enabled by default.

Closes-Bug: #1996913
Change-Id: I3ec1af6353c3ecc64589599abe375b0ae9b14d5c
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

The directive used has the same semantic as what is done above for nbproc > 1:
it binds each thread to a CPU. It is simpler and does not require a loop because
it uses the auto: syntax available in HAProxy 2.4.

Change-Id: I1ce124b678140f5f4737df557683bb67bc7cfc66

Threads are the recommended way to scale CPU performance since HAProxy
1.8.

Official documentation says: « While "nbproc" historically used to be the only
way to use multiple processors, it also involved a number of shortcomings
related to the lack of synchronization between processes (health-checks, peers,
stick-tables, stats, ...) which do not affect threads. As such, any modern
configuration is strongly encouraged to migrate away from "nbproc" to
"nbthread". ».

While more recent versions of HAProxy automatically detect the number of
available CPU and enable threads for them, it can be useful to explicitely set
the value.

In this patch, setting cpu-map for threads is not supported.

Change-Id: Id917c70f3dbe52f24f25d9403ba8151729e8966b

With the parameter ironic_agent_files_directory it is possible to provide
the directory for the ironic-agent.kernel and ironic-agent.initramfs
files. By default the parameter is set to the value of node_custom_config.
This corresponds to the existing behaviour.

Change-Id: I53bb0eddc5380713a967356c85897d8df8ce505f

The external_protocol variable does not exist, resulting in an error
during the venus deployment process. This commit will fix that.

Closes-Bug: #2029353

Change-Id: I2d983eecd8861689fdab7d60bdb9dd34ea0c159e

With the libvirt driver, during live migration,skip comparing guest CPU
with the destination host. When using QEMU >= 2.9 and libvirt >= 4.4.0,
libvirt will do the correct thing with respect to checking CPU
compatibility on the destination host during live migration.[1]

[1] https://opendev.org/openstack/nova/commit/267a40663cd8d0b94bbc5ebda4ece55a45753b64

Change-Id: I947c94b59368c7a2740583bf57e407296473d75e

Labels are supposed to be strings, but prior to
https://github.com/ansible/ansible/pull/80040

 Ansible
did not enforce this.

Change-Id: Iefad160be12f1b5e689a74a82714857fa867d69a
Signed-off-by: Paul Arthur <paul.arthur@flowerysong.com>

The OpenSearch Dashboards container does not have a health
check defined when created. This causes the container to always
restart when reconfigured, even if no change has been made.

Change-Id: I0b437a77aeb61bc5ae9238f900a1fa00cbc34e18
Partial-Bug: #2028362

Change-Id: I7ea236f59a7ede1f5a9ab4c60e7e5aba907ea5b8

Change-Id: I7b998b34881084a68669dc9351ea1937c61534fa

Use case: exposing single external https frontend and
load balancing services using FQDNs.

Support different ports for internal and external endpoints.

Introduced kolla_url filter to normalize urls like:
- https://magnum.external:443/v1
- http://magnum.external:80/v1



Change-Id: I9fb03fe1cebce5c7198d523e015280c69f139cd0
Co-Authored-By: Jakub Darmach <jakub@stackhpc.com>

We've seen issues in CI when keepalived haproxy check script returns
an error and keepalived is switching to backup and then again to primary
on a single node environment.

Closes-Bug: #2025219

Change-Id: Iba62e76b3cf83f3ade6df81288d2d77129ffc725

This patch fixing issue with octavia security group
rules creation when using IPv6 configuration for octavia
management network.

Closes-Bug: #2023502
Change-Id: I3f8fbb0632ec6ecdc9f3820ebbcf01480de59e1f

Change-Id: Idbbd02b966922d5857ed54bac57668f0cf22113c

Replaces the instance label on prometheus metrics with the inventory
hostname as opposed to the ip address. The ip address is still used as
the target address which means that there is no issue of the hostname
being unresolvable. Can be optionally enabled or set to FQDNs by
changing the prometheus_instance_label variable as mentioned in the
release notes.

Co-Authored-By: Will Szumski <will@stackhpc.com>
Change-Id: I387c9d8f5c01baf6054381834ecf4e554d0fff35

Hardcoded docker value in commands is not supported anymore
and kolla_container_engine is used instead.

Change-Id: I25d9563c82842ac51d41467ff7b4144b306fdb12
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>

Closes-Bug: #2024314
Change-Id: I608b84905fa69346a33a1cef10f159b3412cbfd7
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

Ansible 2.14.3 introduced a change that broke the method used for
restarting MariaDB and RabbitMQ serially [1][2]. In
I57425680a4cdbf0daeb9b2cc35920f1b933aa4a8 we limited to 2.14.2 to work
around this. Ansible upstream claim this behaviour was unintentional,
and will not fix it.

This change moves to a different approach where we use separate plays
with a 'serial' keyword to execute the restart.

This change also removes the restriction on the maximum supported
version of 2.14.2 on ansible-core - any 2.14 release is now supported.

[1] https://github.com/ansible/ansible/commit/65366f663de7d044f42ae6dd53368fd4c1f88b35
[2] https://github.com/ansible/ansible/issues/80848

Depends-On: https://review.opendev.org/c/openstack/kolla/+/884208

Change-Id: I5a12670d07077d24047aaff57ce8d33ccf7156ff

This patch is adding a feature for an option to copy different
ceph configuration files and corresponding keyrings for cinder,
glance, manila, gnocchi and nova services.

This is especially useful when the deployment uses availability
zones as below example.

  - Individual compute can read/write to individual ceph
    cluster in same AZ.
  - Cinder can write to several ceph clusters in several AZs.
  - Glance can use multistore and upload images to
    several ceph clusters in several AZs at once.

Change-Id: Ie4d8ab5a3df748137835cae1c943b9180cd10eb1

Fix permissions for opensearch-dashboard data directory.

Closes-bug: #2020152

Change-Id: Ie4cec7649d89df5b8bb306563da2c62ea0cdd2c0
Signed-off-by: Mathias Fechner <fechner@osism.tech>

The venus containers failed to start with an error
(venus_api container):
/usr/local/bin/kolla_start: line 24: exec: venus-api: not found
because of [1] and also changes the encoding of the files form
dos to unix introduced in [2].

1. https://opendev.org/openstack/venus/src/branch/master/setup.cfg#L29-L30


2. If3562bbed6181002b76831bab54f863041c5a885

Change-Id: I8bee27882c15e39a3d2946787d56bc90db994887
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

According to the documentation [1] type of the Cyborg service should
be 'accelerator' and description 'Acceleration Service'. Also, this
change fixes incorrect endpoint URLs, and not configures an admin
endpoint [2] because the documentation [1] not updated yet.

1. https://docs.openstack.org/cyborg/latest/install/common.html


2. Icf3bf08deab2c445361f0a0124d87ad8b0e4e9d9

Closes-Bug: #2020080
Change-Id: I002db50cbad5a90e479498e605bdeab343e129c7
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

The default value of logging_context_format_string and
logging_user_identity_format has changed [1][2][3].
This is to correct fluentd to use the same format.

[1]https://opendev.org/openstack/oslo.log/commit/ebdee7f39920ad5b4268ee296952432b0d41a375
[2]https://opendev.org/openstack/oslo.log/commit/7ed065f38df0595881d07eb91cec9e5c07cb38be

[3]https://review.opendev.org/c/openstack/oslo.log/+/838185

Closes-Bug: #1872220

Change-Id: Ide0079ef4d28e3a3fa60288a74b936531c52ce60

opensearch-dashboards now use a dedicated user

Depends-On: https://review.opendev.org/c/openstack/kolla/+/883941
Change-Id: I6908f52f824a97a5a4a2bead92b7b2e5cdebdb9f

Depends-On: https://review.opendev.org/c/openstack/neutron/+/878535
Change-Id: I05d8b29b59a7de76da488f68775547a8f0f11d0f

We limit to 2.14.2 due to a regression in ansible-core [1] that breaks
conditional include_task loops in handlers. This is used for controlled
restarts of MariaDB and RabbitMQ.

[1]: https://github.com/ansible/ansible/commit/65366f663de7d044f42ae6dd53368fd4c1f88b35



Change-Id: I57425680a4cdbf0daeb9b2cc35920f1b933aa4a8
Co-Authored-By: Michal Nasiadka <michal@stackhpc.com>

As of I3629b84d3255a8fe9d8a7cea8c6131d7c40899e8 nova
now requires the service_user section to be configured
to address CVE-2023-2088. This change adds
the service user section to the nova.conf template in
the nova and nova-cell roles.

Related-Bug: #2004555
Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: I2189dafca070accfd8efcd4b8cc4221c6decdc9f
(cherry picked from commit a77ea13ef1991543df29b7eea14b1f91ef26f858)
(cherry picked from commit 03c12abbcc107bfec451f4558bc97d14facae01c)
(cherry picked from commit cb105dc293ff1cdb11ab63fa3e3bf39fd17e0ee0)
(cherry picked from commit efe6650d09441b02cf93738a94a59723d84c5b19)

Signed-off-by: FelipeAFV <ffigueroa@whitestack.com>

Closes-Bug: #2009518
Change-Id: I8c4b0053f2f16b6d243462c4b8117748d26143a0

The flags ``--db-nb-pid`` and ``--db-sb-pid`` are corected to be
``--db-nb-pidfile`` and ``--db-sb-pidfile`` respectively. See here for
reference:
https://github.com/ovn-org/ovn/blob/6c6a7ad1c64a21923dc9b5bea7069fd88bcdd6a8/utilities/ovn-ctl#L1045

Closes-Bug: #2018436
Change-Id: Ic1e8768374566eb2198302807ecc644a19cd3062

This patch add a way to choose container engine inside tool and test
scripts. This is in preparation for Podman introduction but still
leaves Docker as default container engine.

Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I395d2bdb0dfb4b325b6ad197c8893c8a0f768324

Adds a flag ``kolla-ansible octavia-certificates --check-expiry <days>``
to the ``octavia-certificates`` command to check if the certificates
will expire within a given number of days.

Change-Id: I869b8afd85fe282d823ecf3593aa22f94a61b2a0

New openstack collection modules have changed output
dicts/variable names - adapting to that.

Also changing octavia amphora image to focal, since bionic
hasn't been rebuilt since May 2021.

Closes-Bug: #2012255
Change-Id: Icf38a52472d02ef7d69bcd3716afb16e859d44a2

Sometimes passwords, URLs and other values of the variables can
contain special symbols, for example the dollar sign, using these
values can lead to unpredictable attempts of the variable expansions
in the Bash scripts, such as openrc file, so we need to use single
quotes for all variables values.

Change-Id: Ib2aabadd0ffd6a8dc2591245f29b4478e03d92fc
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>