In Ibecac60d1417269bbe25a280996ca9de6e6d018f, the services in the common
role were marked as being mapped to the 'all' group, since the
'service_mapped_to_host' filter expects every service definition to have
either a 'group' or 'host_in_groups' field. While this allows the filter
to pass the common services without error, it will not actually show
them as being mapped to any hosts. This is because the filter uses the
'group_names' variable, which contains all of the groups that a host
belongs to, except the default 'all' group.

This change fixes the issue by returning True from
service_mapped_to_host when the service's group is 'all'.

Change-Id: I39c8416f5d30a535c1743f9c43434b7d2a382196
Related-Bug: #1868596

This patch introduces an optional backend encryption for Keystone
service. When used in conjunction with enabling TLS for service API
endpoints, network communcation will be encrypted end to end, from
client through HAProxy to the Keystone service.

Change-Id: I6351147ddaff8b2ae629179a9bc3bae2ebac9519
Partially-Implements: blueprint add-ssl-internal-network

manila share container name variable is fixed in some places,
but in the defaults directory, manila share container_name variable
is variable. If the manila share container_name variable is changed
during deployment, it will not be assigned to container name,
but a fixed 'manila_share' name.

Change-Id: Iea23c62518add8d6820b76b16edd3221906b0ffb

The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found by updated hacking version.

Remove hacking and friends from lower-constraints, they are not needed
during installation.

Change-Id: I7ef5ac8a89e94f5da97780198619b6facc86ecfe

We don't need tox nor tell our users to use it when doing kolla
builds.
Tox is going away from base infra images.
It's already gone from aarch64 (arm64) ones.

Change-Id: I2eb5203ad93c011b8806f4b6fb56db081c14a2cb

The use of default(omit) is for module parameters, not templates. We
define a default value for openstack_cacert, so it should never be
undefined anyway.

Change-Id: Idfa73097ca168c76559dc4f3aa8bb30b7113ab28

Now that py2 is gone, oslotest dropped dependency on mock and will
soon affect Ussuri CI [1], let's use unittest.mock built in py3.

This also fixes py38 jobs and proactively prevents py36 and py37
failing due to [1]. This is because we never included mock in
test-requirements (but in lower-constraints where it does not
really belong at all) and instead relied on oslotest to bring
it in.

[1] https://review.opendev.org/716322

Change-Id: I30e82e2d87418272a71c7ee089a8acdaf8872158

Currently there are a few services that perform host configuration
tasks. This is done in config.yml. This means that these changes are
performed during 'kolla-ansible genconfig', when we might expect not to
be making any changes to the remote system.

This change separates out these host configuration tasks into a
config-host.yml file, which is included directly from deploy.yml.

One change in behaviour is that this prevents these tasks from running
during an upgrade or genconfig. This is probably what we want, but we
should be careful when any of these host configuration tasks are
changed, to ensure they are applied during an upgrade if necessary.

Change-Id: I001defc75d1f1e6caa9b1e11246abc6ce17c775b
Closes-Bug: #1860161

One way to improve the performance of Ansible is through fact caching.
Rather than gather facts in every play, we can configure Ansible to
cache them in a persistent store. An example Ansible configuration for
doing this is as follows:

[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./facts
fact_caching_timeout = 86400

This does not affect Kolla Ansible however, since we use the setup
module which unconditionally gathers facts regardless of the state of
the cache. This gets worse with large inventories limited to a small
batch of hosts via --limit or serial, since the limited hosts must
gather facts for all others.

One way to detect whether facts exist for a host is via the
'module_setup' variable, which exists only when facts exist. This change
uses the 'module_setup' fact to determine whether facts need to be
gathered for hosts outside of the batch. For hosts in the batch, we
switch from using the setup module to gather_facts on the play, which
can use the 'smart' gathering logic.

Change-Id: I04841fb62b2e1d9e97ce4b75ce3a7349b9c74036
Partially-Implements: blueprint performance-improvements

In [1] only neutron-openvswitch-agent was fixed and not xenapi.
That merged in Ussuri and went cleanly into Train.
In Stein and Rocky, the backport was not clean and
accidentally fixed xenapi instead of the regular one.

Neither the original bug nor its incomplete fix were released,
except for Rocky. :-(
Hence this patch also removes the confusing reno instead of
adding a new one.

[1] https://review.opendev.org/713129

Change-Id: I331417c8d61ba6f180bcafa943be697418326645
Closes-bug: #1869832
Related-bug: #1867506

In stable branches we are getting hit by more py2-incompats.
Let's pin u-c in all CI pip invocations.

Change-Id: Ie2bcc7c115cd2aaf4639d90803216011b346daf3

Not everyone wants Kafka data stored on a Docker volume. This
change allows a user to flexibly control where the data is stored.

Change-Id: I2ba8c7a85c7bf2564f954a43c6e6dbb3257fe902

Docker registry password is, by default, sourced from
passwords.yml file.

Cleans up globals.yml to make it clearer.
Also follows the "present defaults" behaviour of the other vars.

Change-Id: Icc993e82a6a435f948e3d17e410eb14717cb0e2d

keystone and keystone_fernet container name variable is fixed
in some places, but in the defaults directory, keystone
and keystone_fernet container_name variable is variable.
If the keystone and keystone_fernet container_name variable is
changed during deployment, it will not be assigned to keystone
and keystone_fernet, but a fixed 'keystone' and 'keystone_fernet' name.

Change-Id: Ifc8ac69e6abc4586f0e4fd820b9022aea9f76396

kolla-toolbox container name variable is fixed in some places,
but in the defaults directory, kolla-toolbox container_name variable
is variable. If the kolla-toolbox container_name variable is changed
during deployment, it will not be assigned to kolla-toolbox,
but a fixed 'kolla-toolbox' name.

Change-Id: I9579017761ff47477dba597282be9ae6fab4242a

This patch fix creating statck resource failure in heat.

Change-Id: I00c23f8b89765e266d045cc463ce4d863d0d6089
Closes-Bug: #1869137

Change-Id: I9395ae32378f4ff1fd57be78d7daec7745579e04
Closes-Bug: #1869133

Deploy HAProxy on one or more servers. Add another server to the
inventory in the haproxy group, and run the following:

kolla-ansible prechecks --limit <new host>

The following task will fail:

    TASK [haproxy : Checking if kolla_internal_vip_address and
    kolla_external_vip_address are not pingable from any node]

This happens because ansible does not execute on hosts where
haproxy/keepalived is running, and therefore does not know that the VIP
should be active.

This change skips VIP prechecks when not all HAProxy hosts are in the
play.

Closes-Bug: #1868986

Change-Id: Ifbc73806b768f76f803ab01c115a9e5c2e2492ac

mariadb container name variable is fixed in some places,
but in the defaults directory, mariadb container_name variable
is variable. If the mariadb container_name variable is changed
during deployment, it will not be assigned to container_name,
but a fixed 'mariadb' name.

Change-Id: Ie8efa509953d5efa5c3073c9b550be051a7f4f9b

The 'kolla-ansible stop' command can be used to stop the services
running on hosts. However, if you run this command in an environment
with heterogeneous nodes (most real world scenarios have at least
control/compute), then it fails. This is because it only checks
whether a container is enabled, and not whether the host is in the
correct group. For example, it fails with nova-libvirt:

    No such container: nova_libvirt to stop.

This change fixes the issue by only attempting to stop containers on
hosts to which they are mapped.

Change-Id: Ibecac60d1417269bbe25a280996ca9de6e6d018f
Closes-Bug: #1868596

We released CirrOS 0.5.1. Time to move then.

Change-Id: Ibca24836f19b3cbf6166fa39a3702883938feda8