Libvirt debug logs can grow very large, particularly on Ubuntu (we have
seen up to 1.1GB). Disable libvirt debug logging using variable
nova_libvirt_logging_debug which was added to Kolla Ansible during the
Wallaby cycle (see change I90bfd1b300ad3202ea4d139fda6d6beb44c5820f) and
backported to Victoria and Ussuri.

Change-Id: Ic047e80aa382148b009b1c03581b417d17ec2598

Ansible facts can have a large impact on the performance of the Ansible
control host. This patch introduces some control over which facts are
gathered (kayobe_ansible_setup_gather_subset) and which facts are stored
(kayobe_ansible_setup_filter). By default we do not change the default
values of these arguments to the setup module. The flexibility of these
arguments is limited, but they do provide enough for a large performance
improvement in a typical moderate to large OpenStack cloud.

In particular, the large complex dict fact for each interface has a
large effect, and on an OpenStack controller or hypervisor there may be
many virtual interfaces. We can use the kayobe_ansible_setup_filter
variable to help:

    kayobe_ansible_setup_filter: 'ansible_[!qt]*'

This causes Ansible to collect but not store facts matching that
pattern, which includes the virtual interface facts. Currently we are
not referencing other facts matching the pattern within Kayobe.
Note that including the 'ansible_' prefix causes meta facts module_setup
and gather_subset to be filtered, but this seems to be the only way to
get a good match on the interface facts. To work around this, we use
ansible_facts rather than module_setup to detect whether facts exist in
the cache.

The exact improvement will vary, but has been reported to be as large as
18x on systems with many virtual interfaces.

This change also introduces a new command to gather facts for Kayobe &
Kolla Ansible on demand, 'kayobe overcloud facts gather'. This can be
used to populate a fact cache.

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/794610
Story: 2007993
Task: 42586

Change-Id: I5ce3c734433e1682ee942867505468c57440e689

This change adds support for installing Ansible collections via
requirements.yml in Kayobe or Kayobe config.

Story: 2008391
Task: 41315

Change-Id: I764ff019a18266b593add7ab80ee095d7d07a869

This follows the same change in Kolla-Ansible.

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/807344
Change-Id: Ib2d7f0774fede570a8c4c315d83afd420c31da0b

Bashate only accepts one --ignore option, so first was ignored.

Change-Id: I2b7062c4f482faf91c0df40fb79fad75eb326d83

Change-Id: I84a51fcd880971487cda6ccc5edfa4161dbb2156

Molecule 3.5.1 was released by accident [1] and is breaking our
kayobe-tox-molecule job. Avoid 3.5.x releases for now.

[1] https://github.com/ansible-community/molecule/discussions/3240

Change-Id: Ifd75f704177751ec740475bcce8a9f7b8a1578f3

This saves 156MB of storage in each CentOS job on the master branch.

Backport note: ironic-ipxe is Xena only. Storage space saved on Wallaby
and older will be about half (varies depending on Linux distribution and
OpenStack release).

Change-Id: Icb3569bc764435127a00ee15583c6f3e4e598eca

Change-Id: I5bdad8a2f379c11e6b55df8fd2481f5ae01ed6b9

Five levels of depth gives us details about each Docker volume. Remove
files or directories smaller than 1MB from the output to keep it short.

Change-Id: I300e37106ab4dc2ebfa70371391c99cd43c7b14c

Change-Id: I4ed284173841def54953d832c3fcd288e1d2605a

Change-Id: I670593ba8cbcd7c523b9bd6c42b4d0ca29e856ea

Currently the logs get printed in Zuul job stdout and only a small portion
of them is redirected to a file - this patch changes that to send all output
to desired files.

Change-Id: I45e937ec7c102e89edee20679b8959bf7f6c7152

Arguments are passed through to kayobe-env in kayobe-config, which
allows to set the Kayobe environment.

Change-Id: I4c72e32e5379237340284a09874b0c500e41ad0f

This may be used to deploy different images on different hosts.

Change-Id: I941cc28a914f3a56f50abdde70f13d6616ff52e7
Story: 2002098
Task: 41694

Change-Id: I70d5f78871867f3a71150eacc3df1857949e6df9

Tox jobs are timing out in CI due to pip dependency resolution
backtracking taking too long. When run locally, pip fails to resolve
dependencies due to upper-constraints now allowing pluggy 1.0.0 [1],
while molecule still depends on pluggy >= 0.7.1, < 1.0. See similar
tripleo bug for more details [2].

Split molecule dependencies in a separate requirements file and remove
upper constraints from its tox environment for now. As a bonus, it
should speed up execution of the jobs due to installing fewer
dependencies.

Stop using upper-constraints when installing docker python library [3].
This part of the commit can be reverted when docker 5.0.2 is in upper
constraints.

Update Kolla inventory template following renaming of haproxy role to
loadbalancer [4]. With the existing template, haproxy/keepalived
containers are not created and service deployment fails while trying to
connect to MySQL using the VIP.

[1] https://review.opendev.org/c/openstack/requirements/+/806680
[2] https://bugs.launchpad.net/tripleo/+bug/1942508
[3] https://review.opendev.org/c/openstack/kayobe/+/807128
[4] https://review.opendev.org/c/openstack/kolla-ansible/+/770618



Change-Id: I4cad83c92c32d1db2a5d3c03a78a1c533a8b9633
Co-Authored-By: Mark Goddard <mark@stackhpc.com>

The CentOS Stream 8 section used the wrong heading style, which broke
the rest of the document, making the other headings the wrong sizes.

Change-Id: I8015c905074016c53c803ce40a299ad5c911ada6

The command `kayobe overcloud introspection data save` has been failing
due to lack of OpenStack credentials when running the bifrost dynamic
inventory. Add OS_CLOUD=bifrost.

Story: 2009129
Task: 43059
Change-Id: I4a15578135a48cf282672288f4db24ded65f3b82

By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.

This change updates all references to Ansible facts within Kayobe
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.

This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.

[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars

Story: 2007993
Task: 42464
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/791276

Change-Id: I14db53ed6e57d37bbd28dd5819e432e3fe6628b2

If no public API network is used, as in the case of typical all-in-one
environments, the checks in kolla-ansible.yml, task "Validate Kolla
Ansible API address configuration" may fail. This happens only when
using the new kolla_internal_vip_address variable, and not if the legacy
vip_address network attribute is used.

This change fixes the issue by only checking for valid external VIP
address/FQDN configuration when a separate public API network is in use.
If the external VIP/FQDN are not set, then Kayobe no longer defaults
their values to the internal VIP/FQDN, instead relying on the Kolla
Ansible defaults to do this. This may have consequences for anyone
relying on these variables in configuration.

Change-Id: I0b1c963006c044ade5c67c4046da5e2addc9a8c7
Story: 2009134
Task: 43070

Change-Id: I0c41d59123aba3351ac3b0a5a76cff517cca93e4