The ProxySQL startup script was incorrectly using
the `--reload` flag, which only reloads/merges the configuration
without initializing the database from the config file [1]. This
change corrects it to use the `--initial` flag, ensuring that
the database is always reloaded from the configuration at startup.

[1] https://proxysql.com/documentation/configuring-proxysql/#initialstartup

Change-Id: I9cc721555a6d19409b6ac24432b6b34a83efc42c

This patch resolves an issue where ProxySQL could not
bind due to incorrectly formatted IPv6 addresses in the
`mysql_ifaces` configuration. The kolla's
`put_address_in_context` filter is now used, ensuring
the addresses are properly enclosed in square brackets
for correct binding.

Closes-Bug: #2081106
Change-Id: Ic166b8d9a500023c8d23ec9fee03b28b268b26e7

From version 2.1, ProxySQL has a built-in ProxySQL
Prometheus exporter. This patch adds an option to
easily enable this exporter [1].

[1] https://proxysql.com/documentation/prometheus-exporter

Change-Id: I8776cdc0a6ec9e4e35a2424dd0984488514a711f

The --reload parameter ensures that any changes
in the proxysql configuration file are applied
to the already existing internal proxysql database.

Change-Id: I9215d6cef3795030676c44a8184d99ba46dcb60c

Commit [1] introduced a bug into kolla-ansible
where there is incorrect indentation in the haproxy
configuration file. This patch fixes it.

[1] https://github.com/openstack/kolla-ansible/commit/b13fa5a92cb6d768c5839bd11667e2ca72a7cd2f

Closes-Bug: #2080034
Change-Id: I3375e303bc358fc79d1fa2e219e6ec1dba7a38ba

harden the TLS default config according to the mozilla
"modern" recommendation:

https://ssl-config.mozilla.org/#server=haproxy&version=2.1&config=modern&openssl=1.1.1k&guideline=5.7



if you want to revert to the old settings, set:

kolla_haproxy_ssl_settings: "legacy" in globals.yaml
alternatively you can also set it to "intermediate"
for a middle ground between security and accessibility.

this also adjusts the glance and neutron tls proxy ssl settings
in their dedicated haproxy config templates to use the same mechanism.

also add some haproxy related docs to the TLS guide and cross reference
it from the haproxy-guide.

Closes-Bug: #2060787

Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: I311c374b34f22c78cc5bcf91e5ce3924c62568b6

Refactor that prepares kolla_container_facts
module for introducing more actions that will be moved
from kolla_container module and kolla_container_volume_facts.

This change is based on a discussion about adding a new action
to kolla_container module that retrieves all names of the running
containers. It was agreed that kolla-ansible should follow Ansible's
direction of splitting modules between action modules and facts
modules. Because of this, kolla_container_facts needs to be able
to handle different requests for data about containers or volumes.

Change-Id: Ieaec8f64922e4e5a2199db2d6983518b124cb4aa
Signed-off-by: Ivan Halomi <ivan.halomi@tietoevry.com>

The Kolla project supports building images with
user-defined prefixes. However, Kolla-ansible is unable
to use those images for installation.

This patch fixes that issue.

Closes-Bug: #2073541
Change-Id: Ia8140b289aa76fcd584e0e72686e3786215c5a99

Closes-Bug: #2070051

Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/923544

Change-Id: I221eb136e77c61aef39e8646b48b927352d1419d

Most roles are not leveraging the jinja filters available.
According to [1] filtering the list of services makes the execution
faster than skipping the tasks.

This patchset also includes some cosmetic changes to genconfig.
Individual services are now also using a jinja filter. This has
no impact on performance, just makes the tasks look cleaner.

Naming of some vars in genconfig was changed to "service" to make
the tasks more uniform as some were previously using
the service name and some were using "service".

Three metrics from the deployment were taken and those were
- overall deployment time [s]
- time spent on the specific role [s]
- CPU usage (measured with perf) [-]
Overall genconfig time went down on avg. from 209s to 195s
Time spent on the loadbalancer role went down on avg. from 27s to 23s
Time spent on the neutron role went down on avg from 102s to 95s
Time spent on the nova-cell role went down on avg. from 54s to 52s
Also the average CPUs utilized reported by perf went down
from 3.31 to 3.15.
For details of how this was measured see the comments in gerrit.

[1] - https://github.com/stackhpc/ansible-scaling/blob/master/doc/skip.md



Change-Id: Ib0f00aadb6c7022de6e8b455ac4b9b8cd6be5b1b
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>

Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: Iffcce707e0248f166cd71a0b0c1c4a032c8435c6

-f configdir has been supported in HaProxy since 1.7

Partial-Bug: #2048130
Change-Id: Icb95cb072cb3eb5c27ffd79a127069bfcf21a61a

Closes-Bug: 2063896
Change-Id: Ie0d6a8f458562eb32171b5fe23b8cd0cd375dcfb

It was deprecated in Antelope cycle.

Change-Id: I499e69ec6db63e4067e49376e2a1f3e01e48fe62

It's inactive and hasn't produced a 2024.1 release [1].
In addition to that, there's a CVE that hasn't really been patched [2].

Also drop outward_rabbitmq that was used only with Murano.

[1]: https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html#current-inactive-projects
[2]: https://lists.openstack.org/archives/list/openstack-announce@lists.openstack.org/thread/4FYM6GSIM5WZSJQIG4TT5Q3UBKQIHLWX/

Change-Id: I691205730b0e10a42ce61f3340cc39ee51bd1010

It's inactive and hasn't produced a 2024.1 release [1].

[1]: https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html#current-inactive-projects

Change-Id: I217b3633f07e5b2c657e20b19aaa4fbb46535a97

It's inactive and hasn't produced a 2024.1 release [1].

[1]: https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html#current-inactive-projects

Change-Id: I888963751b6e1ed080588297c2889e700431516c

It's inactive and hasn't produced a 2024.1 release [1].

[1]: https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html#current-inactive-projects

Change-Id: Ic988295bc5b8acb19df008fe0d52a3bcc6de2135

It's inactive and hasn't produced a 2024.1 release [1].
There are some efforts to restore Freezer, but let's remove it for now.

[1]: https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html#current-inactive-projects

Change-Id: Ie42012af9e5c64bca23a6e6826bfc4651fd194bd

This new role will handle setting sysctl values.

It also handles cases when IPv6 setting is changed, but IPv6 is
not enabled on the system by skipping those settings.

This is an augmentation of previous patch:
Icccfc1c509179c3cfd59650b7917a637f9af9646

Related-bug: #1906306
Change-Id: I5d6cda3307b3d2f27c1b2995f28772523b203fe7
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>

This way the playbooks won't try to set ipv6 systemctl options
unless ipv6 is available on the system.

Closes-bug: #1906306
Change-Id: Icccfc1c509179c3cfd59650b7917a637f9af9646

HAProxy exposes a Prometheus metrics endpoint, it just needs to be
enabled. Enable this and remove configuration for
prometheus-haproxy-exporter. Remaining prometheus-haproxy-exporter
containers will automatically be removed.

Change-Id: If6e75691d2a996b06a9b95cb0aae772db54389fb
Co-Authored-By: Matt Anson <matta@stackhpc.com>

I35317ea0343f0db74ddc0e587862e95408e9e106 changed certificate path but omitted
single frontend template.

Change-Id: I638ba32e97234900745df62056710dcc37e7db77

Changes name of ansible module kolla_docker to
kolla_container.

Change-Id: I13c676ed0378aa721a21a1300f6054658ad12bc7
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>

Add support for automatic provisioning and renewal of HTTPS
certificates via LetsEncrypt.

Spec is available at:
https://etherpad.opendev.org/p/kolla-ansible-letsencrypt-https

Depends-On: https://review.opendev.org/c/openstack/kolla/+/887347


Co-Authored-By: Michal Arbet <michal.arbet@ultimum.io>
Implements: blueprint letsencrypt-https
Change-Id: I35317ea0343f0db74ddc0e587862e95408e9e106

Threads are the recommended way to scale CPU performance since HAProxy
1.8.

Official documentation says: « While "nbproc" historically used to be the only
way to use multiple processors, it also involved a number of shortcomings
related to the lack of synchronization between processes (health-checks, peers,
stick-tables, stats, ...) which do not affect threads. As such, any modern
configuration is strongly encouraged to migrate away from "nbproc" to
"nbthread". ».

Change-Id: I6f2e9d74e68703c8e0827e495945a75f020e1561

The directive used has the same semantic as what is done above for nbproc > 1:
it binds each thread to a CPU. It is simpler and does not require a loop because
it uses the auto: syntax available in HAProxy 2.4.

Change-Id: I1ce124b678140f5f4737df557683bb67bc7cfc66

Threads are the recommended way to scale CPU performance since HAProxy
1.8.

Official documentation says: « While "nbproc" historically used to be the only
way to use multiple processors, it also involved a number of shortcomings
related to the lack of synchronization between processes (health-checks, peers,
stick-tables, stats, ...) which do not affect threads. As such, any modern
configuration is strongly encouraged to migrate away from "nbproc" to
"nbthread". ».

While more recent versions of HAProxy automatically detect the number of
available CPU and enable threads for them, it can be useful to explicitely set
the value.

In this patch, setting cpu-map for threads is not supported.

Change-Id: Id917c70f3dbe52f24f25d9403ba8151729e8966b

Use case: exposing single external https frontend and
load balancing services using FQDNs.

Support different ports for internal and external endpoints.

Introduced kolla_url filter to normalize urls like:
- https://magnum.external:443/v1
- http://magnum.external:80/v1



Change-Id: I9fb03fe1cebce5c7198d523e015280c69f139cd0
Co-Authored-By: Jakub Darmach <jakub@stackhpc.com>

We've seen issues in CI when keepalived haproxy check script returns
an error and keepalived is switching to backup and then again to primary
on a single node environment.

Closes-Bug: #2025219

Change-Id: Iba62e76b3cf83f3ade6df81288d2d77129ffc725

Previously, firewalld rules were applied to configuration, then
firewalld reloaded to pick up all the new rules. Reloading firewalld can
be disruptive because it sets all chains to a DROP policy while building
up its firewall rules, breaking open connections.

This change switches to applying rules both permanently (to config) and
immediately, such that no reload is required.

Change-Id: I8e48b7827b33bdd2061d0e89c905bea8e29f60e8

When running in check mode, some prechecks previously failed because
they use the command module which is silently not run in check mode.
Other prechecks were not running correctly in check mode due to e.g.
looking for a string in empty command output or not querying which
containers are running.

This change fixes these issues.

Closes-Bug: #2002657
Change-Id: I5219cb42c48d5444943a2d48106dc338aa08fa7c

assert will also fail when we're not meeting the conditions, makes
clear what we're actually testing, and isn't listed as a skipped task
when the condition is ok.

Change-Id: Ia72c7052d7f9b8c7d86d74a15dcd9e003178972b

Regularly, we experience issues in Kolla Ansible deployments because we
use wrong options in OpenStack configuration files. This is because
OpenStack services ignore unknown options. We also need to keep on top
of deprecated options that may be removed in the future. Integrating
oslo-config-validator into Kolla Ansible will greatly help.

Adds a shared role to run oslo-config-validator on each service. Takes
into account that services have multiple containers, and these may also
use multiple config files. Service roles are extended to use this shared
role. Executed with the new command ``kolla-ansible validate-config``.

Change-Id: Ic10b410fc115646d96d2ce39d9618e7c46cb3fbc

This change replaces ElasticSearch with OpenSearch, and Kibana
with OpenSearch Dashboards. It migrates the data from ElasticSearch
to OpenSearch upon upgrade.

No TLS support is in this patch (will be a followup).

A replacement for ElasticSearch Curator will be added as a followup.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/830373



Co-authored-by: Doug Szumski <doug@stackhpc.com>
Co-authored-by: Kyle Dean <kyle@stackhpc.com>
Change-Id: Iab10ce7ea5d5f21a40b1f99b28e3290b7e9ce895

Kolla Ansible is switching to OpenSearch and is dropping support for
deploying ElasticSearch. This is because the final OSS release of
ElasticSearch has exceeded its end of life.

Monasca is affected because it uses both Logstash and ElasticSearch.
Whilst it may continue to work with OpenSearch, Logstash remains an
issue.

In the absence of any renewed interest in the project, we remove
support for deploying it. This helps to reduce the complexity
of log processing configuration in Kolla Ansible, freeing up
development time.

Change-Id: I6fc7842bcda18e417a3fd21c11e28979a470f1cf

Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

This change adds container_engine variable to kolla_container_facts
module, this prepares module to be used with docker and podman as well
without further changes in roles.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I9e8fa30646844ab4a288555f3aafdda345b3a118

First part of patchset:
 https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

This implements kolla_container_engine variable
in command calls of docker,so later on it can be
also used for podman without further change.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Change-Id: Ic30b67daa2e215524096ad1f4385c569e3d41b95

By default ProxySQL's default value of max_replication_lag
is 0 which is in fact disabling this feature [1].
If it is greater than 0, ProxySQL will regularly monitor
replication lag and if it goes beyond the configured threshold
it will temporary shun the host until replication catches up.

This should be configurable via kolla-ansible as every
openstack deployment can be different in terms of network
delays, database load etc.. , so user should have option
to configure when database backend will be shunned.

[1] https://proxysql.com/documentation/main-runtime/

Change-Id: I66171638abc712cb84b380042f1d29f54c499e73

mainly jinja spacing and jinja[invalid] related

Change-Id: I6f52f2b0c1ef76de626657d79486d31e0f47f384