Delegate executing uri REST methods to the current module containers
using kolla_toolbox. This will allow self signed certificate that are
already copied into the container to be automatically validated. This
circumvents requiring Kolla Ansible to explicitly disable certificate
validation in the ansible uri module.

Partially-Implements: blueprint custom-cacerts

Change-Id: I2625db7b8000af980e4745734c834c5d9292290b

When kolla_copy_ca_into_containers is set to "yes", the Certificate
Authority in /etc/kolla/certificates will be copied into service
containers to enable trust for that CA. This is especially useful when
the CA is self signed, and would not be trusted by default.

Partially-Implements: blueprint custom-cacerts

Change-Id: I4368f8994147580460ebe7533850cf63a419d0b4

This change introduces prune-images command.

Uses docker_prune module of Ansible that comes with version 2.8.

Depends-On: https://review.opendev.org/#/c/699333/

Implements: blueprint docker-image-pruning

Change-Id: Icbf374dd50e1cc1f1604bb4fa779b34279efd50c

Introduce user modifiable variables instead of fixed-names
of Ceph keyring files for external Ceph functionality.

Change-Id: I1a33b3f9d6eca5babf53b91187461e43aef865ce

These affected both deploy (and reconfigure) and upgrade
resulting in WSREP issues, failed deploys or need to
recover the cluster.

This patch makes sure k-a does not abruptly terminate
nodes to break cluster.
This is achieved by cleaner separation between stages
(bootstrap, restart current, deploy new) and 3 phases
for restarts (to keep the quorum).

Upgrade actions, which operate on a healthy cluster,
went to its section.

Service restart was refactored.

We no longer rely on the master/slave distinction as
all nodes are masters in Galera.

Closes-bug: #1857908
Closes-bug: #1859145
Change-Id: I83600c69141714fc412df0976f49019a857655f5

To use an iSCSI Cinder backend as its store, glance_api must run
privileged and have /dev and /etc/iscsi properly mounted

Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I988d3c9d0564483440ae17203ad88a8049abbea4
Closes-Bug: #1855695

Since [1] nova-compute uses rbd python library instead of libvirt to cleanup
volumes and get pool info - so it requires cinder keyring on filesystem.

In external ceph case it is often that nova key does not exist (is simply a copied
cinder key) and the rbd user is set to cinder - therefore the earlier mentioned
operations will fail due to a missing keyring on the filesystem.

[1]: https://review.opendev.org/#/c/668564/

Change-Id: Idef21dc5f7e9ff512bc8920630a3de61a1e69eee
Backport: train
Closes-Bug: #1859408

Include a reference to the globally configured Certificate Authority to
all services. Services use the CA to verify HTTPs connections.

Change-Id: I38da931cdd7ff46cce1994763b5c713652b096cc
Partially-Implements: blueprint support-trusted-ca-certificate-file

This patch mounts the kolla_logs volume into the Elasticsearch
container so that logs are no longer written to the container
filesystem. It is up to the user to migrate any existing logs
into the kolla_logs volume, if they so desire.

Closes-Bug: #1859162
Change-Id: Ia1743e202e310fc88a61476c80eadf3855256c20

For the CentOS 7 to 8 transition, we will have a period where both
CentOS 7 and 8 images are available. We differentiate these images via a
tag - the CentOS 8 images will have a tag of train-centos8 (or
master-centos8 temporarily).

To achieve this, and maintain backwards compatibility for the
openstack_release variable, we introduce a new 'openstack_tag' variable.
This variable is based on openstack_release, but has a suffix of
'openstack_tag_suffix', which is empty except on CentOS 8 where it has a
value of '-centos8'.

Change-Id: I12ce4661afb3c255136cdc1aabe7cbd25560d625
Partially-Implements: blueprint centos-rhel-8

Maximum supported version is set to 2.9

Updated the minimum supported version to 2.8

Implements: blueprint ansible-max-version

Change-Id: I97cc95e37f49886e6d74f2d5a789b923b14b5a2d

In CentOS/RHEL 8 there is no scsi-target-utils package, nor is it
available in EPEL. It is removed from kolla in [1]. In RHEL 7 and beyond
the LIO kernel subsystem can be used instead of the tgtd daemon.

This change removes support for the SCSI target daemon on CentOS/RHEL 8.
The 'tgtd' image is no longer available for CentOS/RHEL 8.

[1] https://review.openstack.org/#/c/613815/5

Change-Id: I718fc16cde2dd177b2a1c2f79b932426034897fe
Related: blueprint centos-rhel-8

Currently used cephfs driver have been deprecated in Pike [1], change to use
the proper one.

[1]: https://opendev.org/openstack/manila/src/branch/master/releasenotes/notes/rename-cephfs-native-driver-3d9b4e3c6c78ee98.yaml

Closes-Bug: #1858773
Change-Id: I33bea1d0049accd48c61f85c1165bee1e1cf0c87

It advertises C7 as an IPv6-compatible platform.
This is possible thanks to fixes in [1] and [2].

[1] https://review.opendev.org/699458
aka 7054b27d
[2] https://review.opendev.org/699172
aka 908bffcf

Change-Id: Ia353a1663a16f48ac83e5ee9a2cf1d6e183ac3a3
Closes-bug: #1848444
Closes-bug: #1848452
Related-bug: #1856532
Related-bug: #1856725

This change applys the HAProxy tag to the entire play, ensuring HAProxy
configuration is generated for all services when the HAProxy tag is
specified.

Change-Id: I67f57c831a713142d38c6e7b70f814a9ee8e5aae
Closes-Bug: #1855094

Currently External Ceph Cinder config requires the user to create cinder
service custom configuration.

This change alters the if/else statements to template out cinder backends
configuration when cinder_backend_ceph is True.

Change-Id: I143c3b44d2839e56d1dbf28484c0eaae0a753dc9

Ironic provides a feature to allow instance images to be served from a
local HTTP server [1]. This is the same server used for PXE images with
iPXE. This does not work currently because the ironic_ipxe container
does not have access to /var/lib/ironic/images (ironic docker volume),
where the images are cached. Note that to make use of this feature, the
following is required in ironic.conf:

[agent]
image_download_source = http

This change fixes the issue by giving ironic_ipxe container access to
the ironic volume.

[1] https://docs.openstack.org/ironic/latest/admin/interfaces/deploy.html#deploy-with-custom-http-servers

Change-Id: I501d02cfd40fbacea32d551c3912640c5661d821
Closes-Bug: #1856194

2020 is coming, everyone should be using Python 3 now.

As per the official python support timeline set forth by the OpenStack
TC [1], OpenStack Train (in our case, kolla-ansible 9.x) is the last
release that will support python2.7.

[1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html

Implements: blueprint drop-py2-support

Change-Id: Ibb3b12a779ecfd424053d0b3e98dac2f21d909bc

This allows users to supply an Elasticsearch Curator actions file
to manage log retention [1]. Curator then runs on a cron job, which
defaults to every day. A default curator actions file is provided,
which can be customised by the end user if required.

[1] https://www.elastic.co/guide/en/elasticsearch/client/curator/current/actionfile.html

Change-Id: Ide9baea9190ae849e61b9d8b6cff3305bdcdd534

WSGI log files use a different input configuration than OpenStack log
files. Currently this depends on log files matching either *-access.log
or *-error.log. Some services use *_access.log or *_error.log, so are
not parsed correctly.

This change modifies the fluentd configuration to accept an underscore
or hyphen for WSGI log file names.

Change-Id: I566d6cac0b6749054fd5422ec8f36f99dacb1db7
Closes-Bug: #1720371

To fix instability and availability issues:

etcd3 is not available in repos for binary kolla images.

etcd3 does not support eventlet-based services [1].

[1] https://review.opendev.org/466098

Change-Id: I430bab735da204fc81696130b17931a89214c876
Closes-bug: #1852086
Closes-bug: #1854932

Change-Id: Ia02f83dfaaba53f95e373b2b2be3f74cfb7ae578
Closes-Bug: #1855085

Depends-On: https://review.opendev.org/692948/
Depends-On: https://review.opendev.org/692691/
Change-Id: I07827b896d36c3723697540fcff164224f6729af

In a deployment where Prometheus is enabled and
Alertmanager is disabled the task "Copying over
prometheus config file" in
'ansible/roles/prometheus/tasks/config.yml' will
fail to template the Prometheus configuration file
'ansible/roles/prometheus/templates/prometheus.yml.j2'
as the variable 'prometheus_alert_rules' does not
contain the key 'files'. This commit fixes this bug.

Change-Id: Idbe1e52dd3693a6f168d475f9230a253dae64480
Closes-Bug: #1854540

We mount Swift volumes with xfs.
The 'nobarrier' option we used was made noop [1]
and deprecated [2] (with warning) in kernel 4.10.
In 4.19 it was removed [3] resulting in an error
when using e.g. Debian Buster as host.
The noop patch was backported to CentOS 7 so
it is safe to remove this option with no behavior
change and backport to where needed.
Ubuntu Bionic uses 4.15 which only warns.
CentOS 8 uses 4.18 which only warns as well.
Debian Buster uses 4.19 exactly which breaks.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2291dab2c9d1880efd19469df2042e2277c8b7a4
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cf4573d899cd80d8578c050061dc342f99f3a32
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1c02d502c20809a2a5f71ec16a930a61ed779b81

Change-Id: I006dea21321146c7fc738d0b41c401b72d271a99
Closes-bug: #1800132

Adds support for configuration of the Docker client timeout via
'docker_client_timeout'.

This change also increases the default timeout to 120 seconds, as we
sometimes see timeouts in CI and heavily loaded or underpowered
environments. Increasing 'docker_client_timeout' further may be helpful
in cases where Docker reports 'Read timed out'.

Change-Id: I73745771078cb2c0ebae2b1d87ba2c4c12958d82
Closes-Bug: #1809844

Change-Id: I799993728112a525e34cfbc4e786a10f0ed03be9

It turned out the previous fix ([1]) was incomplete.
Additionally, it seems we have to limit Tacker server
to one instance co-located with conductor.

[1] https://review.opendev.org/684275
commit b96ade3c

Change-Id: I9ce27d5f68f32ef59e245960e23336ae5c5db905
Closes-bug: #1853715
Related-bug: #1845142

Change-Id: Ie35ea07b8b6f95cbb56eb722ae2366c00243e562

Opendaylight support has been deprecated in Train - time to remove it.

Change-Id: I3a61bfbcbf366c327ea3e25d2424bc3fedca29f0

* Deploy services using kolla-ansible deploy
* Reconfigure the image for one or more services to use an invalid
* config
* Deploy/reconfigure services using kolla-ansible reconfigure

The invalid config could be a wrong docker registry, wrong image name,
wrong tag, etc.

The restart handler for the service fails, and the old container is
left running.

The restart handler for the service fails, and the old container is
stopped and removed. This leaves the service in a broken state.

This change fixes the issue by pulling the image if necessary prior to
stopping and removing the container.

Change-Id: I85b2a1b224d4c4d85c32c4922a2cd2c41171a1dc
Closes-Bug: #1852572

This was deprecated in the Train release in favour of enable_mariadb.

Change-Id: Iea0c6eb51ff26817eeb913c9aa241a9fe7553588

Allow users to create/override HAProxy service configuration by
copying over '*.cfg' files from {{ node_custom_config
}}/haproxy/services.d/

Ex: /etc/kolla/config/haproxy/services.d/radosgw.cfg

Change-Id: Id84e3b6e62e544582d6917047534e846e026798d
Signed-off-by: Keith Plant <kplantjr@gmail.com>

Add file to the reno documentation build to show release notes for
stable/train.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/train.

Change-Id: I7cc8f7853d49a22be6cc04312366a811c0d2799c
Sem-Ver: feature

Currently, Xtrabackup is used for database backups. However, Xtrabackup
is not compatible with MariaDB 10.3. This change switches to use
mariabackup [1], which is available in the mariadb image.

The documented full and incremental restore procedures have been
modified to use mariabackup, following [2] and [3].

[1] https://mariadb.com/kb/en/library/mariabackup-overview/
[2] https://mariadb.com/kb/en/library/full-backup-and-restore-with-mariabackup/
[3] https://mariadb.com/kb/en/library/incremental-backup-and-restore-with-mariabackup/

Change-Id: Id52b9b1f7b013277e401b1f6b8aed34473d2b2c4
Closes-Bug: #1843043
Depends-On: https://review.opendev.org/691290

Adds rabbitmq_server_additional_erl_args variable which
is appended to RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS
environment variable to RabbitMQ server startup script.

This can be used to configure the schedulers.

Docs attached.

Change-Id: Id683c8cc6dac61354ffd94f3b460335b42136ba2
Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Related-bug: #1846467

Change-Id: If72fd2c060c8ecd014a61338a3506d93578804b2

Both swift.conf and proxy-server.conf are affected be
/etc/kolla/config/swift/proxy-server.conf. However, some options in
proxy-server.conf are not valid in swift.conf.

This change keeps this path for proxy-server.conf, but modifies the path
for swift.conf to /etc/kolla/config/swift/proxy-server/swift.conf. The
same applies for other services, object-*, account-*, container-*.

Change-Id: I600891a15244ce705861f6ec93eec1d5ba83c1b8
Closes-Bug: #1849265

Since at least the Rocky release we have been enabling
this needlessly.
The enable_haproxy_memcached parameter is not documented but it
can be kept as it is very light on maintenance.

Change-Id: I8b3a6a9f676d2d79657d859190198b17cc8e8a82

Change-Id: Ifd96a8127b9f2e398b8e853fb1de08eaf22eb696