This change serialises the neutron l3 agent restart process and adds a
user configurable delay between restarts. This can prevent connectivity
loss due to all agents being restarted at the same time.

Routers increase the recovery time, making this issue more prevalent.

Change-Id: I3be0ebfa12965e6ae32d1b5f13f8fd23c3f52b8c

This commit adds SystemdWorker class to kolla_docker ansible module.
It is used to manage container state via systemd calls.

Change-Id: I20e65a6771ebeee462a3aaaabaa5f0596bdd0581
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>

As rabbitmq's configuration file is not ini or yaml file,
there is no option to extend configuration by new config
options via merge_configs or merge_yaml.

This patch moves config options to dictionary
so it can be overriden in /etc/kolla/globals.yml.

Change-Id: I5cd772f4fb80a0e200fb24d67be735ca81e3fdeb

Makes sure the facts required to generate octavia.conf are available
when using genconfig.

This change also ensures that the necessary tasks run when using Ansible
check mode.

Closes-Bug: #1987299
Change-Id: Ib8fbee2d3abdcfd2eae0f9b3e9b69eeb0e3086e0

A combination of durable queues and classic queue mirroring can be used
to provide high availability of RabbitMQ. However, these options should
only be used together, otherwise the system will become unstable. Using
the flag ``om_enable_rabbitmq_high_availability`` will either enable
both options at once, or neither of them.

There are some queues that should not be mirrored:
* ``reply`` queues (these have a single consumer and TTL policy)
* ``fanout`` queues (these have a TTL policy)
* ``amq`` queues (these are auto-delete queues, with a single consumer)
An exclusionary pattern is used in the classic mirroring policy. This
pattern is ``^(?!(amq\\.)|(.*_fanout_)|(reply_)).*``

Change-Id: I51c8023b260eb40b2eaa91bd276b46890c215c25

When running in check mode, some prechecks previously failed because
they use the command module which is silently not run in check mode.
Other prechecks were not running correctly in check mode due to e.g.
looking for a string in empty command output or not querying which
containers are running.

This change fixes these issues.

Closes-Bug: #2002657
Change-Id: I5219cb42c48d5444943a2d48106dc338aa08fa7c

Prevent the haproxy-config role from attempting to modify firewalld when
running kolla-ansible genconfig.

Closes-Bug: #2002522
Change-Id: Ie8a524cc944aa8cb9cf0999b1b8da79f30b40092

Change-Id: I8855bd60c2fd77f33fb55d4123131a94327bd166

The ``[oslo_messaging_rabbit] heartbeat_in_pthread`` config option
is set to ``true`` for wsgi applications to allow the RabbitMQ
heartbeats to function. For non-wsgi applications it is set to ``false``
as it may otherwise break the service [1].

[1] https://docs.openstack.org/releasenotes/oslo.messaging/zed.html#upgrade-notes

Change-Id: Id89bd6158aff42d59040674308a8672c358ccb3c

Setting ovn-monitor-all to 'true' will configure
ovn-controller to monitor all OVS database records
unconditionally. That will release some CPU resource
from OVS Southbound DB but will increase number of events
coming to ovn-controller.

Default value is 'false'.

Change-Id: I291e166013d8c88f00e84ceaf308251c352c9a79

ovn-controller should be deployed first according to OVN upgrade guide.
Since we are getting newer OVN/OVS versions from RDO/Ubuntu in a cycle,
let's apply that to deployment.

Closes-Bug: #1979329

Change-Id: I017aec611a057db1634cfc2634164b21cb210193

Regularly, we experience issues in Kolla Ansible deployments because we
use wrong options in OpenStack configuration files. This is because
OpenStack services ignore unknown options. We also need to keep on top
of deprecated options that may be removed in the future. Integrating
oslo-config-validator into Kolla Ansible will greatly help.

Adds a shared role to run oslo-config-validator on each service. Takes
into account that services have multiple containers, and these may also
use multiple config files. Service roles are extended to use this shared
role. Executed with the new command ``kolla-ansible validate-config``.

Change-Id: Ic10b410fc115646d96d2ce39d9618e7c46cb3fbc

Add file to the reno documentation build to show release notes for
stable/zed.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.

Sem-Ver: feature
Change-Id: I8f24a2318b5bd5ff60a235c093db022344dec644

Change-Id: I2db0ddcdacb1502878609f51c4252210c8d5fd6c

Change-Id: Ieb2f4ee5d4c64c92e589d4da63c5b66472052163

Their cleanup has been added to monasca cleanup command.

Change-Id: I19a846e2683ae70b33ca64d2aba7ac71eb724588

Change-Id: If98434d7941cf319b0689441827290791f596a04

This change replaces ElasticSearch with OpenSearch, and Kibana
with OpenSearch Dashboards. It migrates the data from ElasticSearch
to OpenSearch upon upgrade.

No TLS support is in this patch (will be a followup).

A replacement for ElasticSearch Curator will be added as a followup.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/830373



Co-authored-by: Doug Szumski <doug@stackhpc.com>
Co-authored-by: Kyle Dean <kyle@stackhpc.com>
Change-Id: Iab10ce7ea5d5f21a40b1f99b28e3290b7e9ce895

Instead of handling everything in one role - let's have small
fit-for-purpose roles, because in reality these are two hosts
roles and performance should be better with this approach.

[1]: https://docs.ovn.org/en/latest/intro/install/ovn-upgrades.html

Change-Id: I8f9dbe9d950323f16375ad5e1dbaedfb1be6585f

This reverts commit 8bf8656d.

Reason for revert: Setting ovn-chassis-mac-mappings on network nodes 
is causing mac flooding [1] [2] for traffic between external ports, 
and very slow troughput in consequence.
OVN HA Chassis priorities between gateways should probably be managed
by Neutron [3]

[1] https://mail.openvswitch.org/pipermail/ovs-discuss/2020-September/050691.html
[2] https://mail.openvswitch.org/pipermail/ovs-discuss/2022-May/051837.html
[3] https://mail.openvswitch.org/pipermail/ovs-discuss/2022-October/052068.html

Change-Id: Ia3b279d7e2c08464fda1a5dc41518296f559e93f

Previously ovn-chassis-mac-mappings [1] has been added only to
ovn-controller-compute group. However external ports are being
scheduled on network nodes, therefore we need also do that there.

Closes-Bug: 1995078

[1] https://github.com/ovn-org/ovn/blob/v22.09.0/controller/ovn-controller.8.xml#L239

Change-Id: Ie62e9220bad56262cad602ca1480e6ca65827819

Libvirt needs to be able to plug ports into openvswitch bridges.
It does this using the ovs-vsctl command, which it searches for
in $PATH[1, 2]. This change will optionally install a wrapper
script that executes the ovs-vsctl commands in the context of the
openvswitchd container. This is useful when running libvirt on the
host whilst still running openvswitch in a container. The advantage
of this method over install the packages on the host is that it
ensures client compatability with the daemon. The default is set
to false as the wrapper could overwrite ovs-vsctl installed on the
host.

[1] https://github.com/libvirt/libvirt/blob/ee51ab86c2e52b6ff1b17a4c7ad11439fd610c9e/src/util/virnetdevopenvswitch.c#L59
[2] https://github.com/libvirt/libvirt/blob/a89b17c2a75cfbaeb9e430f88e0f8a7475eb4f54/docs/kbase/internals/command.rst#id3

Closes-Bug: #1995409
Change-Id: Iaa6bfb012ae847f5f6aa0a1fc1c27970ac265f93

Kolla Ansible is switching to OpenSearch and is dropping support for
deploying ElasticSearch. This is because the final OSS release of
ElasticSearch has exceeded its end of life.

Monasca is affected because it uses both Logstash and ElasticSearch.
Whilst it may continue to work with OpenSearch, Logstash remains an
issue.

In the absence of any renewed interest in the project, we remove
support for deploying it. This helps to reduce the complexity
of log processing configuration in Kolla Ansible, freeing up
development time.

Change-Id: I6fc7842bcda18e417a3fd21c11e28979a470f1cf

This trivial patch is just adding missed logrotate
configuration for proxysql.

Closes-Bug: #1995248
Change-Id: I3ad88d03836930160b6db43a7cad63b34ffc62b0

The correct option to use is valid_interfaces [1], not os_endpoint_type.
The os_endpoint_type option was removed in Train.

[1] https://docs.openstack.org/ironic-inspector/wallaby/configuration/sample-config.html

Change-Id: I3906d7b9a2bebfe5c323cba5f80add3e932468c8
Closes-Bug: #1995246
Related-Bug: #1990675

By default ProxySQL's default value of max_replication_lag
is 0 which is in fact disabling this feature [1].
If it is greater than 0, ProxySQL will regularly monitor
replication lag and if it goes beyond the configured threshold
it will temporary shun the host until replication catches up.

This should be configurable via kolla-ansible as every
openstack deployment can be different in terms of network
delays, database load etc.. , so user should have option
to configure when database backend will be shunned.

[1] https://proxysql.com/documentation/main-runtime/

Change-Id: I66171638abc712cb84b380042f1d29f54c499e73

During zun_cni_daemon binds the port to container netns,
zun_cni_damon creates a new net namepsaces(cni-xxx),
Currently, the namespace is only present inside the
zun_cni_daemon container, if this container restart or
rerun, all zun capsules will lost network capability.

Closes-Bug: #1993551

Change-Id: I3642bbf1ad8e8f4744b215fb8deff25fd4ceae75

We agreed that CentOS Stream 9 images are not published as we keep it
for CI use only (to check potential failures before it hits RHEL).

We recommend Rocky Linux 9 instead.

Change-Id: I06e6746e5c2abbdcd97912ea2f99d82fc662531d

Adds a deprecation notice for Monasca service together with
its dependecies: Kafka, Storm and Zookeeper.

Change-Id: Ia9daf170ce9157edb2132c69ee6a923bc4d6f980

Kolla Ansible now supports failing execution early if fact collection
fails on any of the hosts. This is to avoid late failures due to missing
facts (especially cross-host).

Change-Id: I7a74b937ded0b9da0621cf413f3a5d0d13a2cd68
Partial-Bug: #1833737

By resetting image_upload_use_cinder_backend to upstream default.

When uploading volume to glance image, cinder looks at the backend's
image_upload_use_cinder_backend config knob to decide whether to try link
the glance image to a cloned volume made by cinder, i.e. by doing all work
locally and only updating glance's locations for the image (when the knob
is set to True). However, after all [1], [2] and [3], which happens since
Victoria, this option requires further config from user (using volume type
with image_service:store_id property (aka extra spec) set to the desired
glance store (even if there is only one cinder store configured).

Please read the bug report as to why the option removal is the
best option (TL;DR it is the most compatible approach).

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/708114
[2] https://review.opendev.org/c/openstack/glance_store/+/746556
[3] https://review.opendev.org/c/openstack/cinder/+/661676

Closes-Bug: #1991516
Change-Id: Ife87ee0241d907a0c407eb21811a354ed1734408

This is generally considered insecure because it may reveal
sensitive data [1].
Furthermore, it happens that the default Ceph perms cause fatal
ERRORs with this setting:
1) when Glance wants to remove an image, it cannot list children
because Cinder or Nova might have created a linked volume clone
behind the scenes and it is put in another pool (volumes/vms)
which Glance cannot normally access;
2) when Nova wants to create an image, it lacks permissions
to write to the images pool.

Thus, I propose that Kolla Ansible stops setting this by default
and relies on the working defaults.
The downside is that this disables optimisations in Cinder and Nova.
On the other hand, these optimisations have nasty behaviour of
being linked directly to the original image, preventing its removal.

[1] https://docs.openstack.org/glance/yoga/configuration/glance_api.html#DEFAULT.show_multiple_locations

Change-Id: I63ee9a6eefd8593f2169bba34dbb699f413d7cf8
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/860093
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/860291
Closes-Bug: #1992153

In the Victoria cycle, Nova merged improved support for
managing resource providers:
https://review.opendev.org/q/topic:bp%252Fprovider-config-file

See the blueprint for more details:
https://docs.openstack.org/nova/latest/admin/managing-resource-providers.html

This change allows us to copy the necessary configuration.

Change-Id: I0a3caaad73bc6fe27380e7f6bf6b792aca51c84c

Currently kolla-ansible sets haproxy balance algorithm to source for
horizon. We can set it to round-robin if the cache backend is memcached
or using the database as the session storage backend. So we can
distribute http requests evenly to all available horizon instances.

Closes-Bug: #1990523
Change-Id: I0721cadcf53d59947bc0db6a193bfafe49c41ad3

JWT failed to validate on auth-oidc endpoint used by openstack cli
with "could not find key with kid: XX" error. To fix this we need
to use jwks provided in "jwks_uri" by OIDC metadata endpoint.

Missing "ServerName" directive from vhost config causes redirection
to fail in some cases when external tls is enabled.

  - added "keystone_federation_oidc_jwks_uri" variable
  - added "OIDCOAuthVerifyJwksUri" to keystone vhost config
  - added "ServerName" to keystone vhost config
  - jinja templating additional whitespace trimmed to
    correct end result indentation and empty newlines

Closes-bug: 1990375
Change-Id: I4f5c1bd8be8e23cf6299ca4bdfd79e9d98c9a9eb

With this option enabled, dnsmasq can offer the same IP address to
multiple hosts when their requests are close to each other. Remove this
option in order to use the built-in hashing mechanism which will
allocate random IP addresses, which should be less likely to conflict.

Closes-Bug: #1991390
Change-Id: I09a9fa2d0c54635b899ad7906cc2e2e4580ef5ad

Change-Id: Ia8acdf69cb3676ec939777c32f0568cb720c471f

Change-Id: Ic89097fdc72d4fa11754201ed6e388bf79ca40b6

Bind9 is running without limit for UDP listeners.
This patch is changing this behaviour and sets max 32
of UDP listeners. This is needed because of bug below [1].

[1] https://bugs.launchpad.net/ubuntu/+source/bind9/+bug/1827923

Change-Id: Ie4c2ac4d5e990ebdc30c3a94d855703d814f1fee

The admin endpoint is kept on upgrade to allow the upgrade to
happen (as it allows to rewrite the previous admin endpoint entry
to the new one).

Change-Id: I1c16892bab67f281d539843f1f0fa658df1c4874
Depends-On: https://review.opendev.org/c/openstack/kolla/+/854837