This is a follow up to Ie7ae47e5a624da31cf6cc5cd3c9239450487b8ed, but
for cephadm jobs.

Change-Id: I194529c178efcff0cb2a0ee1e14273389cd8f87f
(cherry picked from commit 4e2af187)

Ansible 2.16.14 includes a fix for CVE-2024-11079 [1], which results
in the following error:

    jinja2.exceptions.UndefinedError:
        'ansible.vars.hostvars.HostVarsVars
         object' has no attribute 'ansible_host'

This issue occurs because Ansible now includes `localhost` in
`hostvars` without the `ansible_host` variable set, which
unexpectedly breaks our test inventory rendering.

This patch adds a filter to exclude `localhost`
during the rendering process, as it is irrelevant for our purposes.

[1] https://github.com/ansible/ansible/pull/84353

Change-Id: Ie7ae47e5a624da31cf6cc5cd3c9239450487b8ed
(cherry picked from commit 0298699a)

Change-Id: I14d3ee3f94b8138188f70b2b5106145d33943eaf
(cherry picked from commit 61045807)

Change-Id: Idd143389534ef2195a07059efca732fb81f92171

Change-Id: I044affa85290caf9d71e8ba5bc7cf0f5be8941e7

Change from file to template

Closes-Bug: #2089229
Change-Id: I7cbc6a94608baf4f04ef231cc88397fc5dcf0a9b
(cherry picked from commit f30dd3e5)

Change-Id: I0e07271312bbed49be7a986a2beda04f316dfc8f
(cherry picked from commit b5d594d3)

Change-Id: I3434eb7f751553a3a29a718ac20975529736bc3b
(cherry picked from commit 1cec85d6)

From [1]:
If check is true, and the process exits with a non-zero
exit code, a CalledProcessError exception will be raised.
Attributes of that exception hold the arguments, the exit
code, and stdout and stderr if they were captured.

[1]: https://docs.python.org/3/library/subprocess.html

Closes-Bug: #2089173

Change-Id: I8cf38c2f7db1493e7303e94c212251fbeafaced3
(cherry picked from commit 4f62dd46)

Also rewrite/reformat while here.

Change-Id: Ife2acdb0d4360c58c7de68cf259a0ed4a86234c2
(cherry picked from commit 2339561e)

I have no clue how it worked previously in CI, but now
it's using default path to the inventory - which does
not exist.

In addition to that, type=int in cliff will default to
None, so the check had to be rewritten - because we
always did cert expiry check instead of generating them.

Change-Id: I84d71558c2666ba2cfa47054f782d25ff0c1f691
(cherry picked from commit cd8ecfc8)

Rework the base jobs structure to include a mid-level
kolla-ansible-scenario-base that includes common
files: stanza.

Change-Id: I548b22b27dff111d625361835029354557d8c9ca
(cherry picked from commit 04873199)

One task in destroy role was missing arguments that were
supplied by the old bash CLI but not supplied by
the new python CLI.

Closes-Bug: #2086187
Change-Id: I6ced070ca10b63c4d27ac76c1e82dc4312c1b165
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>
(cherry picked from commit a1eec249)

As version-check.yml is added to deploy.yml, we must make sure the
tasks are only run when the rabbitmq container exists.

Change-Id: Iaa31bae739110094affb5e402ed9ac40b153ac3d
(cherry picked from commit f5ad7829)

Change-Id: Ia414f82384cc6984ff688ce3c9872f47a66357ed

When installing kolla-ansible with `pip install ./kolla-ansible`, pip
always creates a direct_url.json file, even when not using an editable
installation.

We see this behaviour with Python 3.12, while direct_url.json is only
created for editable installations on Python 3.9, which was used when
this code was initially developed for Kayobe.

When using a regular (non-editable) installation, this would make
kolla-ansible invoke site.yml from the source directory instead of the
virtualenv installation, causing a failure to load Ansible collections:

    Invalid plugin FQCN (ansible.utils.ipaddr): unable to locate collection ansible.utils

Fix by returning the source URL only if dir_info.editable is True.

Change-Id: Icdc2cedaa6a6e3a6b4351b1f4369e2e8b3a2dc97
(cherry picked from commit 1c7d17d1)

Update the URL to the upper-constraints file to point to the redirect
rule on releases.openstack.org so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.

Until the requirements repository has as stable/2024.2 branch, tests will
continue to use the upper-constraints list on master.

Change-Id: I03e63ea4a6a3080c28baa162e91e97319ae2d872

Change-Id: I3ed8176f3f42e0f543c5615d60c7ace9e71f21dd

Moving the CLI to python allows for easier
maintenance and larger feature-set.

This patch introduces a few breaking changes!
The changes stem the nature of the cliff package.
- the order of parameters must be
  kolla-ansible <action> <arguments>
- mariadb_backup and mariadb_recovery now are
  mariadb-backup and mariadb-recovery

Closes-bug: #1589020
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>
Change-Id: I9749b320d4f5eeec601a055b597dfa7d8fb97ce2

This patch enables internal TLS database
connection for Keystone.

Change-Id: I816d051e933a560629d9b9c95362f668abe4ade7

This patch ads an ability to receive TLS connections
to ProxySQL. Certificates and variable lookups are
added in order for TLS to be enabled by
<project_name>_database_internal_tls_enable.
Note that in order for this to work, mysql
connection strings need to have TLS enabled,
which can be added in separate per-service patches

Change-Id: I2c06ce5e138f52259c1725dae37f25c1b00d1e6b

Change-Id: I33a3ec11b0cdef94b08cd7551008284755824cb7

It has been removed in I23867aa98f68298beb5db4558c66c1ffd4e7d6f1

Change-Id: I12d287b9f7f1e5ddf754b7f2ca1dee39778e710e

This commit adds TLS connection between ProxySQL and MariaDB.
Frontend TLS ( between services and ProxySQL) will be
added in another commit.

Parialy Implements: mariadb-ssl-support

Change-Id: I154cbb096469c5515c9d8156c2c1c5dd07b95849
Signed-off-by: Matus Jenca <matus.jenca@dnation.cloud>

Ubuntu package does not have that in dependencies and cephadm
fails without that. See [1].

[1]: https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2085603

Change-Id: I5124f7078e15645979b68986dceb51948c89a520

The MySQL monitor user privileges were updated to include
the REPLICATION CLIENT privilege in addition to USAGE in
order to align with ProxySQL documentation [1].
This change ensures that the monitor user can check replication
lag, as previously only the USAGE privilege was granted,
which was sufficient for basic connection and read-only
checks but not for replication monitoring.

[1] https://proxysql.com/documentation/backend-monitoring/

Change-Id: I4172cf1d49e9f988cbf2bbe3c3f6835f0de4944d

We are not testing mariadb backup, let's test
it in mariadb scenario.

Change-Id: I81d6ee944b3ed0e75129772bb993ce7a6147c3e8

After switching to ProxySQL as default we see following logs:
CRITICAL neutron [None req-c214fdae-5da7-402d-92b0-0572c278a5b5 - - - - - -] Unhandled error: sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (9001, 'Max connect timeout reached while reaching hostgroup 0 after 10150ms')

Mainly in upgrade jobs, which otherwise pass successfuly - just fail on this
check.

Change-Id: I4336ec62a0a2dfbe815842f1bacb02135bcf4c0e