When OpenStack is deployed with Kolla-Ansible, by default there
are no durable queues or exchanges created by the OpenStack
services in RabbitMQ. In Rabbit terminology, not being durable
is referred to as `transient`, and this means that the queue
is generally held in memory.

Whether OpenStack services create durable or transient queues is
traditionally controlled by the Oslo Notification config option:
`amqp_durable_queues`. In Kolla-Ansible, this remains set to
the default of `False` in all services. The only `durable`
objects are the `amq*` exchanges which are internal to RabbitMQ.

More recently, Oslo Notification has introduced support for
Quorum queues [7]. These are a successor to durable classic
queues, however it isn't yet clear if they are a good fit for
OpenStack in general [8].

For clustered RabbitMQ deployments, Kolla-Ansible configures all
queues as `replicated` [1]. Replication occurs over all nodes
in the cluster. RabbitMQ refers to this as 'mirroring of classic
queues'.

In summary, this means that a multi-node Kolla-Ansible deployment
will end up with a large number of transient, mirrored queues
and exchanges. However, the RabbitMQ documentation warns against
this, stating that 'For replicated queues, the only reasonable
option is to use durable queues: [2]`. This is discussed
further in the following bug report: [3].

Whilst we could try enabling the `amqp_durable_queues` option
for each service (this is suggested in [4]), there are
a number of complexities with this approach, not limited to:

1) RabbitMQ is planning to remove classic queue mirroring in
   favor of 'Quorum queues' in a forthcoming release [5].
2) Durable queues will be written to disk, which may cause
   performance problems at scale. Note that this includes
   Quorum queues which are always durable.
3) Potential for race conditions and other complexity
   discussed recently on the mailing list under:
   `[ops] [kolla] RabbitMQ High Availability`

The remaining option, proposed here, is to use classic
non-mirrored queues everywhere, and rely on services to recover
if the node hosting a queue or exchange they are using fails.
There is some discussion of this approach in [6]. The downside
of potential message loss needs to be weighed against the real
upsides of increasing the performance of RabbitMQ, and moving
to a configuration which is officially supported and hopefully
more stable. In the future, we can then consider promoting
specific queues to quorum queues, in cases where message loss
can result in failure states which are hard to recover from.

[1] https://www.rabbitmq.com/ha.html
[2] https://www.rabbitmq.com/queues.html
[3] https://github.com/rabbitmq/rabbitmq-server/issues/2045
[4] https://wiki.openstack.org/wiki/Large_Scale_Configuration_Rabbit
[5] https://blog.rabbitmq.com/posts/2021/08/4.0-deprecation-announcements/
[6] https://fuel-ccp.readthedocs.io/en/latest/design/ref_arch_1000_nodes.html#replication
[7] https://bugs.launchpad.net/oslo.messaging/+bug/1942933
[8] https://www.rabbitmq.com/quorum-queues.html#use-cases

Partial-Bug: #1954925
Change-Id: I91d0e23b22319cf3fdb7603f5401d24e3b76a56e

Without this configuration, all mount points are reporting the same
utilisation metrics [1]. With the rslave option, all root mounts from
the host are visible in the container, so we can remove the bind mounts
for /proc and /sys.

[1] https://github.com/prometheus/node_exporter#docker

Change-Id: I4087dc81f9d1fa5daa24b9df6daf1f9e1ccd702f
Closes-Bug: #1961438

An FCD, also known as an Improved Virtual Disk (IVD) or
Managed Virtual Disk, is a named virtual disk independent of
a virtual machine. Using FCDs for Cinder volumes eliminates
the need for shadow virtual machines.
This patch adds Kolla support.

Change-Id: Ic0b66269e6d32762e786c95cf6da78cb201d2765

The following variables are added:

* prometheus_blackbox_exporter_cmdline_extras
* prometheus_elasticsearch_exporter_cmdline_extras
* prometheus_haproxy_exporter_cmdline_extras
* prometheus_memcached_exporter_cmdline_extras
* prometheus_mysqld_exporter_cmdline_extras
* prometheus_node_exporter_cmdline_extras
* prometheus_openstack_exporter_cmdline_extras

Change-Id: I5da2031b9367115384045775c515628e2acb1aa4

NSXP is the OpenStack support for the NSX Policy platform.
This is supported from neutron in the Stein version. This patch
adds Kolla support

This adds a new neutron_plugin_agent type 'vmware_nsxp'. The plugin
does not run any neutron agents.

Change-Id: I9e9d8f07e586bdc143d293e572031368af7f3fca

Yoga testing runtime[1] has been updated to add py39
testing as voting. Unit tests update are handled by the
job template change in openstack-zuul-job

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

this commit updates the classifier in setup.cfg file.

[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html

Change-Id: I7057bee4d53a1be770b2ab5dd49cfb2c02091646

Change-Id: I27b0e42fba93a35c6d878d108bf1e7fdebc9e3db

Change-Id: I5b3ab3ab8153cda283dec772bf1393af0caf4137
Closes-Bug: 1919179

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: I75ca59d981bcd2dd51faa296ab0b4223a891f5cb

Change-Id: I9c736a586a757b49170977c7f9cf2c4890557a33

Change-Id: I3362bd283eb7fb80f5da70f2a388f89f220617ea
Closes-Bug: #1960503

Make it less likely to fail on false positives.

Change-Id: I493430e255c76bf2a30818b63913ecabb40c5afc

Allow operators to set haproxy socket to admin level.
This is done via the flag haproxy_socket_level_admin which
is set to "no" by default.

Closes-Bug: 1960215

Signed-off-by: Imran Hussain <ih@imranh.co.uk>
Change-Id: Ia0da89288d68f5803ace1934c013053f12343195

The change happened in Train, time to move on.

Change-Id: Ie58265284b2e6b4b30b24fc2f22dd4f5eec05d5b

The apparmor_parser actually doesn't remove the file or doesn't create
the symlink in '/etc/apparmor.d/disable' itself so the next run of the
baremetal role will fail with the error "Unable to remove "libvirtd".
Even more after reboot, the profile is still active. We need to
disable the profile completly ourselves. This change fixes the
idempotents of the baremetal role.

Closes-Bug: #1960302
Change-Id: I162e417387393e806886b1c9ea8053b89778b4d1
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

The default configuration was changed to use the advanced cache pool in
keystonemiddleware 9.3.0 (Xena release) [1].

This reverts commit 5a52d8e4 (except the
release note).

[1] https://review.opendev.org/c/openstack/keystonemiddleware/+/773939

Change-Id: I290d0a81c57c189b6eb62fc3eee3ed19f441671b

parted hangs waiting for user input (see examples below)
on Debian and Ubuntu nodes which have created a cinder
volume on lvm, causing POST_FAILURE of the entire CI job.
Zun (Cinder iSCSI LVM) jobs are affected.
parted seemingly tries to interpret contents of the created
volume and fails miserably.
Since there is no reason why we would need to see the output
of parted specifically, this patch is switching to use
lsblk to simply list visible block devices.
Along with the rest of the commands, this should be just
the right level of detail.
And we avoid having parted interpret internals of otherwise
opaque block devices.

Example issues:

Warning: Not all of the space available to
/dev/mapper/cinder--volumes-cinder--volumes--pool appears to be used, you can
fix the GPT to use all of the space (an extra 9732096 blocks) or continue with
the current setting?
Fix/Ignore?

Warning: Not all of the space available to
/dev/mapper/cinder--volumes-cinder--volumes--pool-tpool appears to be used, you
can fix the GPT to use all of the space (an extra 9732096 blocks) or continue
with the current setting?
Fix/Ignore?

Warning: Not all of the space available to
/dev/mapper/cinder--volumes-cinder--volumes--pool_tdata appears to be used, you
can fix the GPT to use all of the space (an extra 9732096 blocks) or continue
with the current setting?
Fix/Ignore?

Change-Id: I7beecf2dd6c49c8934722cf22efa74e920ecb060

Enable libvirt TLS in CI jobs with TLS enabled.

Uses the new functionality of the certificates command to generate
certificates for both libvirt client and server (added in
I1bde9fa018f66037aec82dc74c61ad1f477a7c12).

Change-Id: Ica304685b043f699799ccee6c9c2fbcf968888db

Adds support to the 'kolla-ansible certificates' command for generating
certificates for libvirt TLS, when libvirt_tls is true. The same
certificate and key are used for the libvirt client and server.

The certificates use the same root CA as the other generated
certificates, and are written to
{{ node_custom_config }}/nova/nova-libvirt/, ready to be picked up by
nova-libvirt and nova-compute.

Change-Id: I1bde9fa018f66037aec82dc74c61ad1f477a7c12

Enables zun to access cinder volumes when cinder is configured to use
external ceph.
Copies ceph config file and ceph cinder keyring to /etc/ceph in
zun_compute container.

Closes-Bug: 1848934
Change-Id: Ie56868d5e9ed37a9274b8cbe65895f3634b895c8

This gets rid of one of the steps in the authentication flow.

Closes-Bug: 1930055
Change-Id: I4ed4651b55a912f1d9aec7277bae6bb4776f1e0a

This is required to use, e.g., Cinder iSCSI backend with Glance.

Closes-Bug: #1959663
Change-Id: I6d5e0e4ab70922a772f3c82e914f9e7d37cf0318

Custom themes support for horizon deployment.
 - horizon role task added for copying theme files
 - added theme copying into the container in templates
 - docs and reno updated

Change-Id: If9982c8e18be31772cb031ef72b7eebd4d768be5
Co-Authored-By: Jakub Darmach <jakub@stackhpc.com>
Depends-On: https://review.opendev.org/c/openstack/kayobe/+/824565
Depends-On: https://review.opendev.org/c/openstack/kolla/+/826672