Kayobe allows specifying a --limit argument, which is passed through to
Ansible. In some cases we wish to add an intersection with a group. This
allows us to reuse playbooks for the seed, overcloud etc.

For example, the lvm.yml playbook specifies a host list of
seed-hypervisor:seed:overcloud. When executed as part of a kayobe
overcloud host configure command, Kayobe passes a limit of overcloud. If
the user specifies a --limit argument, this gets intersected with the
overcloud limit: host1:&overcloud.

The problem happens if the user specifies multiple parts to the host
pattern in their limit using a comma, e.g. host1,host2. This results in
host1,host2:&overcloud. Ansible ignores the colon, and treats this as
host1 or host2:&overcloud.

The solution is to use a comma to join the patterns if the user has used
a comma: host1,host2,&overcloud

Change-Id: Ibe42fa372c6fa0c539d2c2b0e238601286dc213d
Story: 2008255
Task: 41111

In some environments the can_talk_to_docker key is missing from the
result of docker_host_info when the call succeeds. Assume we can talk to
Docker if this key is not present, instead of failing due to a missing
attribute error.

Change-Id: I7670410394b0bb5fc96e4166b6a20714f847250b

In particular, lvm_group_data_enabled variables were missing.

Change-Id: Ia73655812feaf5ce0598cdc71b673126423843a0

Without this setting, bifrost creates a bifrost firewalld zone only
allowing network traffic for Ironic services and assigns the
provisioning network interface to it, potentially causing loss of
connectivity.

Using the public zone is suggested as a workaround [1] but is not
sufficient: it allows SSH traffic, but blocks other services deployed on
the seed, such as Docker registry traffic.

[1] https://review.opendev.org/#/c/754406/

Change-Id: I80f9d95f02e11fda5916f9a9dd257b688a9db7e2
Story: 2008153
Task: 40899

To avoid switching existing deployments from devicemapper to overlay2,
we check the existing storage driver configuration directly with the
Docker daemon, or if unreachable by reading the /etc/docker/daemon.json
configuration file.

Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>
Story: 2005667
Task: 30972

Change-Id: Iaf2ee8c9f302f4684ae039bb00b2e2e5969cf1fc

While we always test baremetal compute in CI, development environments
may not. Given that Ironic is now disabled by default, we should make
this work out of the box.

Story: 2008207
Task: 41003

Change-Id: Id3128380f5ff74d24265f6b2132c6d7992bf00ba

We do not need the workload provisioning, cleaning or inspection
networks when Ironic is disabled in the overcloud.

Change-Id: I300d0ef136224126f25d2c70a80a42afeea5f586
Story: 2008207
Task: 40992

Ironic is now disabled by default in the overcloud. This brings Kayobe's
default set of services into line with Kolla Ansible. For environments
using Ironic in the overcloud, set kolla_enable_ironic to true in
kolla.yml.

Story: 2008207
Task: 40991

Change-Id: I33eb4fa534847e199a599350b525d4762a2beaac

The flag dnf_install_epel was being used to predicate installation
of the EPEL RPM repo package, but not preventing the installation
of configuration for access to EPEL.  This small patch completes
the job.

Updates the overcloud host configure CI job to install EPEL mirrors,
since it was previously relying on this buggy behaviour.

Change-Id: Ib417837d0772338b16ea9f7f2540549f277d5de8

When generating host variable files for Kolla Ansible, we have some
heavy usage of set_fact to set variables for network interfaces, in a
play targeted at all hosts. There are also tasks using the fail action
plugin to perform verification.

At scale this has a significant impact, due to the number of tasks
executed against all hosts. These tasks are executed at the beginning of
many commands, so the scope is broad.  There is also some tricky logic
involved, which is difficult to express in Ansible/Jinja.

This change replaces the use of set_fact with a custom Ansible action
plugin. The plugin executes locally on the Ansible controller, and
returns a dict of Ansible facts to set for each host. The plugin is
executed once for each overcloud host, and returns all relevant facts.
The plugin also performs verification. Extraction into a Python module
allows for unit testing.

This has been shown to have a significant improvement on execution time,
particularly as the number of hosts reaches 100 or more.

Story: 2007993
Task: 40641

Change-Id: I443da1ae05fcca2d7d8dff7db563eeda37e9f502

Change-Id: I5fc49fb734d0fe94f5f75c66eb4c1a935774ef30

This reverts commit bb9a595e.

Change-Id: Icd7b0f3b02bf3f22468daa8893c043ac9f3be4d6

Ironic has an issue where nodes will be switched to use the direct
deploy interface during upgrade, if it is enabled. This should be fixed
via https://review.opendev.org/755295, after which this patch should be
reverted.

Change-Id: I55af5d979bdbee9dd963845204c2f082fa7b27d7

Change-Id: Ie68ba316327437b4503f4a0aa48a58e89d403790

Story: 2008170
Task: 40925

Change-Id: I3014983f481a5dca7c93e140b3e10caa5d537669

When molecule runs, it installs ansible>2.8, which is version 2.10.0
since today. This now also installs the ansible-base package, currently
at version 2.10.1. Then it installs the version of ansible required by
kayobe, i.e. ansible>=2.8.0,<2.10.0 (currently 2.9.13).

We end up with a tox molecule venv containing:

ansible==2.9.13
ansible-base==2.10.1

This causes the following issue:

ERROR! Unexpected Exception, this is probably a bug: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader' (/home/zuul/src/opendev.org/openstack/kayobe/.tox/molecule/lib/python3.8/site-packages/ansible/utils/collection_loader/__init__.py)

This commit adds requirements.txt to the molecule tox deps, which
ensures the correct version of ansible gets installed first.

Change-Id: I7ef890a2ae6fd67f53e1b8bca244a1c0d44fea53

Kayobe generates a host_vars file for each host in the Kolla Ansible
inventory. These contain network interfaces and other host-specific
things. Currently this is done by iterating over all hosts, which does
not scale well with a large number of hosts.

This change extracts the host vars generation into a separate role, and
executes it in a play targeted at all hosts, with delegate_to:
localhost. This ensures that host variable files are generated in
parallel.

Story: 2007993
Task: 40629

Change-Id: Iae75e17024adee9c2874c14d3ed36f4c87ba48d7

hacking 3.0.1 fix the pinning of flake8 to avoid bringing in a new
version with new checks.

bumping the min version for hacking so that any older hacking versions
which auto adopt the new checks are not used.

Change-Id: I3f4fca9a0df2f45241288fd876064433f4e2f737

Adds support for HTTP basic authentication with the Docker registry.

The kolla docker registry password is now written to passwords.yml.

Change-Id: Ie6e854a66a6660d4e02771fe2b5dd97af814194d
Story: 2007952
Task: 40429

This makes the docs more consistent with other projects.

Change-Id: I75732859a67851df564dc472fa839f08be3367cc

requirements.txt is unnecessary when running `tox -e releasenotes` since
release notes requirements are in doc/requirements.txt. This PS removes
the requirements.txt from tox.ini and reuses [docs] deps.

Sync from kolla-ansible and kolla.

Change-Id: I9df2f98772c143a9662fd5c123cc9f6d5410c5e5

The variables in yum.yml and yum-cron.yml were deprecated in Ussuri.
This patch removes them, and updates the defaults in dnf.yml.

Story: 2008160
Task: 40906

Change-Id: I97cc98dd2ff726e5885fefcab17f17796d9fd453