Refactored the code to switch the distribution source of the roles
to new collections for the roles that have already been migrated into
collections.

Updated:

 - requirements
 - role invocations
 - documentation
 - release notes
 - minor whitespace fixes

Change-Id: I8dbb88aa2061b3afcab07caeb30f45b3a5c939af

This fix was added when upgrading from CentOS Linux 8.2 to 8.3. This is
likely not required on currently supported operating systems.

Change-Id: Ic2c47ae4514d98ff18c5d17ba62b50c47f3929c1

overcloud-dib: added overcloud_dib_dib_upper_constraints_file
ipa: added ipa_build_dib_upper_constraints_file
added os_images_dib_upper_constraints_file to image build
playbooks

New variables were added to facilitate control over the DIB
upper constraints without changing them for other components.
They are empty by default in order to facilitate Rocky9
image builds.

Change-Id: Ib50dd61685f13c60ace67213ddd1e714a80dece3

Adds support for installing additional build host dependencies when
building IPA and overcloud host images via
'ipa_build_dib_host_packages_extra' and
'overcloud_dib_host_packages_extra'.

Support for installing additional host packages was added in
stackhpc.os-images v1.15.0 [1].

[1] https://github.com/stackhpc/ansible-role-os-images/pull/63

Change-Id: I1a05d448031cb415cec4cd601a030abdb8a5eba5

By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.

This change updates all references to Ansible facts within Kayobe
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.

This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.

[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars

Story: 2007993
Task: 42464
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/791276

Change-Id: I14db53ed6e57d37bbd28dd5819e432e3fe6628b2

Change-Id: I2ed9da2580a83f167d4b10b747e0f8225c235afe

There are a few places where we have a play with one task that uses the
group_by module to generate groups that are used in a subsequent play.
These plays do not need to gather facts for their target hosts.

In particular, the kolla-openstack.yml playbook should really be able to
operate without access to remote hosts.

This change also adds changed_when: false to all group_by tasks, since
they show as changed but haven't really made any changes outside of
Ansible.

TrivialFix

Change-Id: I18f67eda4e48058f3f402b9d0e692d6eeb02855f

This variable allows to customise the upper constraints file used to
install packages inside the ipa-build-dib virtual environment. This can
be used when we need a newer version of diskimage-builder than the one
available in upper constraints for the current release.

Change-Id: Idbe57e7edc3fae25153f5e24ad6b7847b1c4660c

Diskimage builder supports specifying a list of packages to install via
-p. We currently support this for the root image, but not IPA images.

This change adds a new configuration option, 'ipa_build_dib_packages',
that should be a list of additional packages to install in locally built
IPA images. This affects the following commands:

kayobe seed deployment image build
kayobe overcloud deployment image build

This depends on
https://github.com/stackhpc/ansible-role-os-images/pull/25.

Also adds an example to the documentation for the equivalent option for
root image.

Change-Id: I04191d9541894b6a264e966c9ecb1056e0edade4
Story: 2006855
Task: 37446

Kayobe uses a number of virtual environments on the remote hosts for
python dependencies such as shade, python-openstackclient, docker, etc.
By default these are stored in /opt/kayobe/venvs/. Typically we do not
provide version restrictions when installing these packages, so over the
course of time they may become stale and incompatible.

This change installs the latest version of packages allowed by OpenStack
upper constraints.

It also adds a new variable, 'pip_upper_constraints_file', to set the
upper constraints file. The existing variable
'kolla_upper_constraints_file' now defaults to the value of
'pip_upper_constraints_file'.

Change-Id: I8d2956f95bbc44b5a9e88e7569372048a62f12f5
Story: 2005923
Task: 34193

Change-Id: I21a682c72cee97583dcb4f1a01fa023c6e885608

It is hard to know when disk images need to be rebuilt, and the
stackhpc.os-images role simply checks for the presence of the DIB output
directory to indicate that an image exists. In some cases, we may wish to force
the rebuilding of a deployment (IPA) image, when we know there are changes to
apply. This change adds a --force-rebuild argument to the following commands:

kayobe seed deployment image build
kayobe overcloud deployment image build

Change-Id: I580811a4f621df9445ef32773ca66bcf303dad9b

When specifying tags with 'kayobe overcloud post configure --tags <tags>', the
expected plays typically do not run. This seems to be due to an ansible bug,
where if a dynamically created group is referenced before creation, it will
remain empty. See https://github.com/ansible/ansible/issues/20360.

This change works around this issue by modifying the name of the dynamic groups
used in the post configure playbooks to be unique to each playbook.

Fixes: #138
(cherry picked from commit 77a41a9d0a37acd2d4c25bd7d969d965a9552428)

Change-Id: I7a884f538cfeba9d6b4b990ec445b5ad6f914193

Having tagged plays allows us to easily run a subset of the plays for a
command, and perform targeted operations with less risk of unintended
consequences.

The tags are typically named after the playbook, although some of the
overcloud playbooks have been tagged without an overcloud- prefix.

We use the group_by module to create groups based on host variables.
When the variable is a boolean, we should convert it to a bool using the
bool filter, to ensure that the group name has a consistent format.

This includes:
* Building and registering IPA images
* Registering a provisioning network
* Registering ironic inspector introspection rules
* Creating an inspection store container

Passing variables inline with the role name appears not to work.

Overcloud deployment images can now be built via:

kayobe overcloud deployment image build

This should be done prior to running kayobe overcloud service deploy.
In order to build IPA images, the ipa_build_images variable should be
set to True. In this case, these images will be used by the overcloud's
ironic inspector service during hardware inspection, and by ironic
during provisioning.