Change from file to template

Closes-Bug: #2089229
Change-Id: I7cbc6a94608baf4f04ef231cc88397fc5dcf0a9b
(cherry picked from commit f30dd3e5)

Change-Id: I0e07271312bbed49be7a986a2beda04f316dfc8f
(cherry picked from commit b5d594d3)

Change-Id: I3434eb7f751553a3a29a718ac20975529736bc3b
(cherry picked from commit 1cec85d6)

From [1]:
If check is true, and the process exits with a non-zero
exit code, a CalledProcessError exception will be raised.
Attributes of that exception hold the arguments, the exit
code, and stdout and stderr if they were captured.

[1]: https://docs.python.org/3/library/subprocess.html

Closes-Bug: #2089173

Change-Id: I8cf38c2f7db1493e7303e94c212251fbeafaced3
(cherry picked from commit 4f62dd46)

Also rewrite/reformat while here.

Change-Id: Ife2acdb0d4360c58c7de68cf259a0ed4a86234c2
(cherry picked from commit 2339561e)

I have no clue how it worked previously in CI, but now
it's using default path to the inventory - which does
not exist.

In addition to that, type=int in cliff will default to
None, so the check had to be rewritten - because we
always did cert expiry check instead of generating them.

Change-Id: I84d71558c2666ba2cfa47054f782d25ff0c1f691
(cherry picked from commit cd8ecfc8)

Rework the base jobs structure to include a mid-level
kolla-ansible-scenario-base that includes common
files: stanza.

Change-Id: I548b22b27dff111d625361835029354557d8c9ca
(cherry picked from commit 04873199)

One task in destroy role was missing arguments that were
supplied by the old bash CLI but not supplied by
the new python CLI.

Closes-Bug: #2086187
Change-Id: I6ced070ca10b63c4d27ac76c1e82dc4312c1b165
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>
(cherry picked from commit a1eec249)

Change-Id: Ia414f82384cc6984ff688ce3c9872f47a66357ed

When installing kolla-ansible with `pip install ./kolla-ansible`, pip
always creates a direct_url.json file, even when not using an editable
installation.

We see this behaviour with Python 3.12, while direct_url.json is only
created for editable installations on Python 3.9, which was used when
this code was initially developed for Kayobe.

When using a regular (non-editable) installation, this would make
kolla-ansible invoke site.yml from the source directory instead of the
virtualenv installation, causing a failure to load Ansible collections:

    Invalid plugin FQCN (ansible.utils.ipaddr): unable to locate collection ansible.utils

Fix by returning the source URL only if dir_info.editable is True.

Change-Id: Icdc2cedaa6a6e3a6b4351b1f4369e2e8b3a2dc97
(cherry picked from commit 1c7d17d1)

Update the URL to the upper-constraints file to point to the redirect
rule on releases.openstack.org so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.

Until the requirements repository has as stable/2024.2 branch, tests will
continue to use the upper-constraints list on master.

Change-Id: I03e63ea4a6a3080c28baa162e91e97319ae2d872

Change-Id: I3ed8176f3f42e0f543c5615d60c7ace9e71f21dd

Moving the CLI to python allows for easier
maintenance and larger feature-set.

This patch introduces a few breaking changes!
The changes stem the nature of the cliff package.
- the order of parameters must be
  kolla-ansible <action> <arguments>
- mariadb_backup and mariadb_recovery now are
  mariadb-backup and mariadb-recovery

Closes-bug: #1589020
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>
Change-Id: I9749b320d4f5eeec601a055b597dfa7d8fb97ce2

This patch enables internal TLS database
connection for Keystone.

Change-Id: I816d051e933a560629d9b9c95362f668abe4ade7

This patch ads an ability to receive TLS connections
to ProxySQL. Certificates and variable lookups are
added in order for TLS to be enabled by
<project_name>_database_internal_tls_enable.
Note that in order for this to work, mysql
connection strings need to have TLS enabled,
which can be added in separate per-service patches

Change-Id: I2c06ce5e138f52259c1725dae37f25c1b00d1e6b

Change-Id: I33a3ec11b0cdef94b08cd7551008284755824cb7

It has been removed in I23867aa98f68298beb5db4558c66c1ffd4e7d6f1

Change-Id: I12d287b9f7f1e5ddf754b7f2ca1dee39778e710e

This commit adds TLS connection between ProxySQL and MariaDB.
Frontend TLS ( between services and ProxySQL) will be
added in another commit.

Parialy Implements: mariadb-ssl-support

Change-Id: I154cbb096469c5515c9d8156c2c1c5dd07b95849
Signed-off-by: Matus Jenca <matus.jenca@dnation.cloud>

Ubuntu package does not have that in dependencies and cephadm
fails without that. See [1].

[1]: https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2085603

Change-Id: I5124f7078e15645979b68986dceb51948c89a520

The MySQL monitor user privileges were updated to include
the REPLICATION CLIENT privilege in addition to USAGE in
order to align with ProxySQL documentation [1].
This change ensures that the monitor user can check replication
lag, as previously only the USAGE privilege was granted,
which was sufficient for basic connection and read-only
checks but not for replication monitoring.

[1] https://proxysql.com/documentation/backend-monitoring/

Change-Id: I4172cf1d49e9f988cbf2bbe3c3f6835f0de4944d

We are not testing mariadb backup, let's test
it in mariadb scenario.

Change-Id: I81d6ee944b3ed0e75129772bb993ce7a6147c3e8

After switching to ProxySQL as default we see following logs:
CRITICAL neutron [None req-c214fdae-5da7-402d-92b0-0572c278a5b5 - - - - - -] Unhandled error: sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (9001, 'Max connect timeout reached while reaching hostgroup 0 after 10150ms')

Mainly in upgrade jobs, which otherwise pass successfuly - just fail on this
check.

Change-Id: I4336ec62a0a2dfbe815842f1bacb02135bcf4c0e

The backup user was missing the necessary CREATE
privilege for the mariadb_backup_history table
within the mysql schema, causing backups to fail
when attempting to create this table.

This patch addresses the issue by granting the backup
user the required CREATE permission specifically for
the mariadb_backup_history table. With this change,
the backup process can now complete successfully
without manual intervention for user permissions.

Closes-Bug: #2061889
Change-Id: Ic92c8959972329adbd4b89c521aa87678f25b4e4

In 2023.1, mariadb was at version 10.6, which meant that
the PERCONA_SCHEMA.xtrabackup_history table was used to
log historicals for backups.

Starting in 2023.2 onwards, mariadb is at version 10.11, and the
default table used is now mysql.mariadb_backup_history.

Because the mysql database already exists, there is no reason
to try and create it. Instead, we just need to update the defaults
and ensure the permissions get added to the correct database.

Related-Bug: #2061889
Change-Id: If146d8f896c70374884807d42ca0e12df3276d48

It's been some time since ProxySQL has been
with us in Kolla. Let's switch the load balancer
for MariaDB connections from HAProxy to ProxySQL.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/928956
Change-Id: I42ba4fb83b5bb31058e888f0d39d47c27b844de5

In single-node clusters, ProxySQL shuns the server on MySQL
errors, causing failures during upgrades or container restarts.
This change increases the timeout to 10 seconds, allowing
the backend time to recover and preventing immediate errors
in CI environments.

Change-Id: I70becdc3fcb4ca8f7ae31d26097d95bdc6dd67eb

Change-Id: Ie2b0b2d5fca7b9d7c613a67a134c4650de2a5af6