Regularly, we experience issues in Kolla Ansible deployments because we
use wrong options in OpenStack configuration files. This is because
OpenStack services ignore unknown options. We also need to keep on top
of deprecated options that may be removed in the future. Integrating
oslo-config-validator into Kolla Ansible will greatly help.

Adds a shared role to run oslo-config-validator on each service. Takes
into account that services have multiple containers, and these may also
use multiple config files. Service roles are extended to use this shared
role. Executed with the new command ``kolla-ansible validate-config``.

Change-Id: Ic10b410fc115646d96d2ce39d9618e7c46cb3fbc

This change replaces ElasticSearch with OpenSearch, and Kibana
with OpenSearch Dashboards. It migrates the data from ElasticSearch
to OpenSearch upon upgrade.

No TLS support is in this patch (will be a followup).

A replacement for ElasticSearch Curator will be added as a followup.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/830373



Co-authored-by: Doug Szumski <doug@stackhpc.com>
Co-authored-by: Kyle Dean <kyle@stackhpc.com>
Change-Id: Iab10ce7ea5d5f21a40b1f99b28e3290b7e9ce895

Instead of handling everything in one role - let's have small
fit-for-purpose roles, because in reality these are two hosts
roles and performance should be better with this approach.

[1]: https://docs.ovn.org/en/latest/intro/install/ovn-upgrades.html

Change-Id: I8f9dbe9d950323f16375ad5e1dbaedfb1be6585f

Typo fix and adding condition on not checking docker SDK version
when container engine is not docker

This is a followup to Ic30b67daa2e215524096ad1f4385c569e3d41b95
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Change-Id: Iafa24db06ad46bcfe250451ed98bc3c48d8a5138

This option was removed from Neutron in the Zed release [1]. This can be
backported to Yoga where the default value was changed to True [2].

[1] https://review.opendev.org/c/openstack/neutron/+/837286
[2] https://review.opendev.org/c/openstack/neutron/+/807848

Change-Id: Ibcd81a3a5f4b8de60459b3a4cfc30a50a06a436f

This reverts commit 8bf8656d.

Reason for revert: Setting ovn-chassis-mac-mappings on network nodes 
is causing mac flooding [1] [2] for traffic between external ports, 
and very slow troughput in consequence.
OVN HA Chassis priorities between gateways should probably be managed
by Neutron [3]

[1] https://mail.openvswitch.org/pipermail/ovs-discuss/2020-September/050691.html
[2] https://mail.openvswitch.org/pipermail/ovs-discuss/2022-May/051837.html
[3] https://mail.openvswitch.org/pipermail/ovs-discuss/2022-October/052068.html

Change-Id: Ia3b279d7e2c08464fda1a5dc41518296f559e93f

A few minor fixes were noted in this review [1], and they
are addressed here.

TrivialFix

[1]: https://review.opendev.org/c/openstack/kolla-ansible/+/861392/

Change-Id: If30d9c2b48615dfb54edcb8d782c4c24b968ac4b

Seems we missed this in Ic1eed7d19e9b583e22419625c92ac3507ea4614d

Change-Id: Ib8505b8cde4a018737d10da1576248e349215fb3

Previously ovn-chassis-mac-mappings [1] has been added only to
ovn-controller-compute group. However external ports are being
scheduled on network nodes, therefore we need also do that there.

Closes-Bug: 1995078

[1] https://github.com/ovn-org/ovn/blob/v22.09.0/controller/ovn-controller.8.xml#L239

Change-Id: Ie62e9220bad56262cad602ca1480e6ca65827819

Libvirt needs to be able to plug ports into openvswitch bridges.
It does this using the ovs-vsctl command, which it searches for
in $PATH[1, 2]. This change will optionally install a wrapper
script that executes the ovs-vsctl commands in the context of the
openvswitchd container. This is useful when running libvirt on the
host whilst still running openvswitch in a container. The advantage
of this method over install the packages on the host is that it
ensures client compatability with the daemon. The default is set
to false as the wrapper could overwrite ovs-vsctl installed on the
host.

[1] https://github.com/libvirt/libvirt/blob/ee51ab86c2e52b6ff1b17a4c7ad11439fd610c9e/src/util/virnetdevopenvswitch.c#L59
[2] https://github.com/libvirt/libvirt/blob/a89b17c2a75cfbaeb9e430f88e0f8a7475eb4f54/docs/kbase/internals/command.rst#id3

Closes-Bug: #1995409
Change-Id: Iaa6bfb012ae847f5f6aa0a1fc1c27970ac265f93

Kolla Ansible is switching to OpenSearch and is dropping support for
deploying ElasticSearch. This is because the final OSS release of
ElasticSearch has exceeded its end of life.

Monasca is affected because it uses both Logstash and ElasticSearch.
Whilst it may continue to work with OpenSearch, Logstash remains an
issue.

In the absence of any renewed interest in the project, we remove
support for deploying it. This helps to reduce the complexity
of log processing configuration in Kolla Ansible, freeing up
development time.

Change-Id: I6fc7842bcda18e417a3fd21c11e28979a470f1cf

From OpenStack Zed the Pure Storage Cinder driver supports
NVMe-RoCE as a dataplane protocol. This patch adds support
for this new driver type.

Also amend a couple of documentation formatting typos.

Change-Id: Ic1eed7d19e9b583e22419625c92ac3507ea4614d

Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

THis change adds container_engine to module parameters
so when we introduce podman, kolla_toolbox can be used
for both engines.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: Ic2093aa9341a0cb36df8f340cf290d62437504ad

Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

This change adds container_engine variable to kolla_container_facts
module, this prepares module to be used with docker and podman as well
without further changes in roles.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I9e8fa30646844ab4a288555f3aafdda345b3a118

This trivial patch is just adding missed logrotate
configuration for proxysql.

Closes-Bug: #1995248
Change-Id: I3ad88d03836930160b6db43a7cad63b34ffc62b0

The correct option to use is valid_interfaces [1], not os_endpoint_type.
The os_endpoint_type option was removed in Train.

[1] https://docs.openstack.org/ironic-inspector/wallaby/configuration/sample-config.html

Change-Id: I3906d7b9a2bebfe5c323cba5f80add3e932468c8
Closes-Bug: #1995246
Related-Bug: #1990675

Change-Id: I87845ec386fda3c6582abad37ae7d8600f222000

First part of patchset:
 https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

This implements kolla_container_engine variable
in command calls of docker,so later on it can be
also used for podman without further change.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Change-Id: Ic30b67daa2e215524096ad1f4385c569e3d41b95

A recent patch [1] enabled sink related changes to nova/neutron even
when designate is not enabled. This patch fixes that.

[1] - https://review.opendev.org/c/openstack/kolla-ansible/+/802301

Change-Id: I6d76f342a7cdbcc61d1522689ea489b60353adcd

By default ProxySQL's default value of max_replication_lag
is 0 which is in fact disabling this feature [1].
If it is greater than 0, ProxySQL will regularly monitor
replication lag and if it goes beyond the configured threshold
it will temporary shun the host until replication catches up.

This should be configurable via kolla-ansible as every
openstack deployment can be different in terms of network
delays, database load etc.. , so user should have option
to configure when database backend will be shunned.

[1] https://proxysql.com/documentation/main-runtime/

Change-Id: I66171638abc712cb84b380042f1d29f54c499e73

During zun_cni_daemon binds the port to container netns,
zun_cni_damon creates a new net namepsaces(cni-xxx),
Currently, the namespace is only present inside the
zun_cni_daemon container, if this container restart or
rerun, all zun capsules will lost network capability.

Closes-Bug: #1993551

Change-Id: I3642bbf1ad8e8f4744b215fb8deff25fd4ceae75

Following up on [1] and fix freezer deployment accidentally broken
after removing 'domain_name' from the 'openstack_auth'.

1. Ib631e2211682862296cce9ea179f2661c90fa585

Change-Id: Ie928f8a4506f41407d76edcb6b52ca7cddb52214
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

We agreed that CentOS Stream 9 images are not published as we keep it
for CI use only (to check potential failures before it hits RHEL).

We recommend Rocky Linux 9 instead.

Change-Id: I06e6746e5c2abbdcd97912ea2f99d82fc662531d

Some time ago we dropped RHEL as one of possible options. During 'Zed'
cycle we added Rocky Linux 9 as alternative to CentOS Stream 9.

This change updates some mentions of both.

Change-Id: I9ed93efcb7d1ff97b1c7d8342db8252aba2a9887

Add noqa for:
Object of type PosixPath is not JSON serializable

Change-Id: Id6ef88bb8cd16120bf31da679d1129d99f4b9fd8

Kolla Ansible now supports failing execution early if fact collection
fails on any of the hosts. This is to avoid late failures due to missing
facts (especially cross-host).

Change-Id: I7a74b937ded0b9da0621cf413f3a5d0d13a2cd68
Partial-Bug: #1833737

By resetting image_upload_use_cinder_backend to upstream default.

When uploading volume to glance image, cinder looks at the backend's
image_upload_use_cinder_backend config knob to decide whether to try link
the glance image to a cloned volume made by cinder, i.e. by doing all work
locally and only updating glance's locations for the image (when the knob
is set to True). However, after all [1], [2] and [3], which happens since
Victoria, this option requires further config from user (using volume type
with image_service:store_id property (aka extra spec) set to the desired
glance store (even if there is only one cinder store configured).

Please read the bug report as to why the option removal is the
best option (TL;DR it is the most compatible approach).

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/708114
[2] https://review.opendev.org/c/openstack/glance_store/+/746556
[3] https://review.opendev.org/c/openstack/cinder/+/661676

Closes-Bug: #1991516
Change-Id: Ife87ee0241d907a0c407eb21811a354ed1734408

These are not used by the relevant daemons and so can be dropped to,
e.g., avoid creating the cinder volume on hosts where there is no
cinder.

Change-Id: Ia8d906a9e0227f361883a7ec1ec8dcd73e4104dc

This is generally considered insecure because it may reveal
sensitive data [1].
Furthermore, it happens that the default Ceph perms cause fatal
ERRORs with this setting:
1) when Glance wants to remove an image, it cannot list children
because Cinder or Nova might have created a linked volume clone
behind the scenes and it is put in another pool (volumes/vms)
which Glance cannot normally access;
2) when Nova wants to create an image, it lacks permissions
to write to the images pool.

Thus, I propose that Kolla Ansible stops setting this by default
and relies on the working defaults.
The downside is that this disables optimisations in Cinder and Nova.
On the other hand, these optimisations have nasty behaviour of
being linked directly to the original image, preventing its removal.

[1] https://docs.openstack.org/glance/yoga/configuration/glance_api.html#DEFAULT.show_multiple_locations

Change-Id: I63ee9a6eefd8593f2169bba34dbb699f413d7cf8
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/860093
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/860291
Closes-Bug: #1992153

Change-Id: Ic267b0bc1153940f7595a1cf93ff2c62dd084d4e

In the Victoria cycle, Nova merged improved support for
managing resource providers:
https://review.opendev.org/q/topic:bp%252Fprovider-config-file

See the blueprint for more details:
https://docs.openstack.org/nova/latest/admin/managing-resource-providers.html

This change allows us to copy the necessary configuration.

Change-Id: I0a3caaad73bc6fe27380e7f6bf6b792aca51c84c

Change-Id: Ie09bf108250a71d539002dd5ccfa63dd71bcfe90

Currently kolla-ansible sets haproxy balance algorithm to source for
horizon. We can set it to round-robin if the cache backend is memcached
or using the database as the session storage backend. So we can
distribute http requests evenly to all available horizon instances.

Closes-Bug: #1990523
Change-Id: I0721cadcf53d59947bc0db6a193bfafe49c41ad3

These are upstream defaults, no need to carry them around.

TrivialFix

Change-Id: I2907d5f38c6a74776961bd473553edf2d83f7257

This patch also changes python version and default tag for centos.
prometheus-efk and venus jobs commented out, elasticsearch images
are unbuildable
cells is commented out because proxysql is unbuildable

Change-Id: Ic358f8b600317d3c2fc45130a59785225aea1153

JWT failed to validate on auth-oidc endpoint used by openstack cli
with "could not find key with kid: XX" error. To fix this we need
to use jwks provided in "jwks_uri" by OIDC metadata endpoint.

Missing "ServerName" directive from vhost config causes redirection
to fail in some cases when external tls is enabled.

  - added "keystone_federation_oidc_jwks_uri" variable
  - added "OIDCOAuthVerifyJwksUri" to keystone vhost config
  - added "ServerName" to keystone vhost config
  - jinja templating additional whitespace trimmed to
    correct end result indentation and empty newlines

Closes-bug: 1990375
Change-Id: I4f5c1bd8be8e23cf6299ca4bdfd79e9d98c9a9eb

With this option enabled, dnsmasq can offer the same IP address to
multiple hosts when their requests are close to each other. Remove this
option in order to use the built-in hashing mechanism which will
allocate random IP addresses, which should be less likely to conflict.

Closes-Bug: #1991390
Change-Id: I09a9fa2d0c54635b899ad7906cc2e2e4580ef5ad

By the comment message, it should no longer be necessary to wait
at this stage and we can speed up the process a little bit.

Change-Id: Ia96bfa79aaad5fbd54a9f527702cca7a63616bf7

They served us well in Yoga but they are no longer needed in Zed.
This also avoids the early deletion of the ironic-conductor, making
it really roll.

Change-Id: I9bc85d894b5bf947ac8fca505df446b99b0bb99b

Fix bifrost stop.yml after I9faecfe6ece6d3c35396e3378c1e3930a487e130

Change-Id: I850cbbb83d10b1518cc73612a591b160c2d49f1c