Yoga upper constraints were used to keep compatibility with Python 3.6.
This is not needed with all supported OS using Python 3.9 or newer.

This reverts commits d2e0d64e and
d190e9e3.

Change-Id: I35a07bcc2b7c9cbb49fa60e6802cc6288a34fbd8

CentOS Stream 8 support has been dropped. Migration path will be present
in Yoga release - as a followup change.

MichaelRigart.interfaces does not support custom routes for
NetworkManager yet. It has been disabled in CI for Rocky Linux 9
temporarily.

Non-voting CentOS Stream 9 CI overcloud job is using RL9 container
images (as kolla CI is not building CS9 images anymore).

Change-Id: Idf5ee822b03ba40179803c981500a6bad37594bf

Fixed sed replacement line.

Change-Id: I2e26a08ea37e5f558af533b38d096f011ed2a5b8

This is changed in order to facilitate running Ironic checks before
OpenStack is deployed.

Change-Id: Ibc4c9c93f52414c23cd7d8b4f1b32c85cb233681

This is required to be able to install tenks. Otherwise, we try to
install Jinja2 3.1.2 which requires Python 3.7 or newer.

Change-Id: Ie497b191b6de8bc818dc4a2a12f7129a02d0fd00

Enable the Ironic ipxe boot interface by default, following a similar
change in Ironic [1].

Drop the kolla_enable_ironic_ipxe flag, following a similar change in
Kolla Ansible [2]. Both PXE and iPXE are now enabled by default. Users
may revert to using PXE for ironic inspector's dnsmasq, by setting
ironic_dnsmasq_serve_ipxe to false in etc/kayobe/kolla/globals.yml.

[1] https://review.opendev.org/c/openstack/ironic/+/816824
[2] https://review.opendev.org/c/openstack/kolla-ansible/+/834512/

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/832159
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/834511
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/837069

Change-Id: Ifb80bd15a20c9cfb8fbc6e0f6ac23baae631a18e

In some cases it may be desirable to run libvirt daemon on the host. For
example, when mixing host and container OS distributions.

This change makes it possible to disable the nova_libvirt container, by
setting kolla_enable_nova_libvirt_container to false.

The stackhpc.libvirt-host role is used in order to install and configure
a libvirt daemon on compute hosts when
kolla_enable_nova_libvirt_container is false.

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/825357
Depends-On: https://review.opendev.org/c/openstack/kayobe-config-dev/+/829225
Depends-On: https://github.com/stackhpc/ansible-role-libvirt-host/pull/51

Story: 2009858
Task: 44495

Change-Id: I73fef63fb886a9d543d2f4231fb009523495edb3

Change-Id: If7d6e58b19f98ccb7cc4c209e458cb6f4f4765ad

Previously we were using the zuul user in the TLS jobs. This was due to
a permissions issue when accessing the CA certificate in kayobe-config
in the zuul user's home directory.

This change reverts to the default of using the stack user for the TLS
jobs. In order to make this work, the generated CA cert chain is added
to the trust store.

Change-Id: I875f8976df75dee68ba00842fe624c29cc1b123c

As a first step towards supporting multiple overcloud disk images, this
change introduces a new command to build a disk image directly with DIB:
`kayobe overcloud host image build`.

It also disables building a root disk image during Bifrost bootstrap if
overcloud_dib_build_host_images is set to true.

Change-Id: I93d242889e225b4e60254f6b9cc5eeb457294ac8
Story: 2002098
Task: 41693

Depends-On: https://review.opendev.org/c/openstack/kayobe/+/812670
Depends-On: https://review.opendev.org/c/openstack/kayobe-config-dev/+/805239
Depends-On: https://review.opendev.org/c/openstack/kayobe/+/813212

Change-Id: I7dbd643fec72223bfda20cbbc37809ae78073804

Arguments are passed through to kayobe-env in kayobe-config, which
allows to set the Kayobe environment.

Change-Id: I4c72e32e5379237340284a09874b0c500e41ad0f

The iscsi deploy interface was removed from Ironic [1]. Switch to the
direct deploy interface instead, which is the default.

[1] https://review.opendev.org/c/openstack/ironic/+/789382

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/801267
Change-Id: Ia0dcd3d302e0f04adbda4f3abf75a0ca7dd20dee

/usr/bin/python may be python 2 on Focal, which causes problems with
Ansible on the control host. By installing the python-is-python3 package
we ensure that the correct interpreter is used.

This change updates the installation documentation and development
environment scripts.

Story: 2004960
Task: 42579

Change-Id: Ie94099075bae3c491f9cf830c38e6cfc8af605a6

All instances of 'kayobe control host bootstrap' in the development
scripts use a helper function, except for during seed_hypervisor_deploy.
The helper adds a retry mechanism to combat flakiness often seen during
Ansible Galaxy installs.

This change fixes the issue.

TrivialFix

Change-Id: I954cb604a18874744b3673ebf2e2c29caa18ce8f

A bug has been introduced to the which package in CentOS Stream 8 which
causes it to fail when used with the following bash options:

set -u
set -o pipefail

Then, when running which we see the following output:

environment: line 1: _declare: unbound variable

As found by Pierre, this seems to be caused by the implementation of
which as a bash function which references an unbound variable
(_declare). It's tracked in Fedora by
https://bugzilla.redhat.com/show_bug.cgi?id=1944877#

.

This change works around the issue by using the /usr/bin/which binary.

Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>

Change-Id: I468d4e0460c13791b9f01d5854ef45472528c6fe
Story: 2008795
Task: 42215

We still see flakiness when downloading content from Ansible Galaxy,
often HTTP 520. This change increases the retries from 3 to 10, and adds
a 5 second delay between attempts.

Change-Id: I0c46e5fcc6979027dc6f1bc5cc49e923a205f654
Related: https://github.com/ansible/galaxy/issues/2429

The 'openstack server show <server> -f value -c addresses' command
previously had output like this:

    <network name>=<IP>

Now it shows a Python output like this:

    {'<network name>': ['IP']}

This broke the parsing of the command output when determining which IP
address to use to access a bare metal instance via SSH.

This change fixes the issue by querying the server's port in Neutron,
and using the fixed IP address.

Change-Id: I55b5f185fb7136d3c6fa565aa46598f21c94eb43

* Use source images
* Need to specify bash for &> syntax

Issues worked around:

* Manually configuring bridge via ip commands makes ifup fail to bring
  up the link. Adds a kayobe-network-bootstrap Zuul CI role that adds
  persistent configuration for the all-in-one network.

* bridge not active after interfaces role bounce. Added a pause, similar
  to https://github.com/michaelrigart/ansible-role-interfaces/pull/31

* fails installing docker python module for kolla user. WARNING: The
  repository located at mirror-int.ord.rax.opendev.org is not a trusted
  or secure host and is being ignored ERROR: No matching distribution
  found for docker===4.4.0 Adding trusted host for PyPI mirror.

* Tenks fails to create block devices - missing qemu-img (in qemu-utils)

* Tenks qemu emulator is different on Ubuntu

Remaining issues:

* Bare metal testing is unreliable on Ubuntu - some jobs see IPMI
  failures such as the following:

    ipmitool chassis bootdev pxe

    Error setting Chassis Boot Parameter 5\nError setting Chassis Boot
    Parameter 0\n

  Bare metal testing is disabled on Ubuntu for now.

Depends-On: https://review.opendev.org/766984
Depends-On: https://review.opendev.org/766958

Story: 2004960
Task: 29393

Change-Id: I1985efae7c18f55c3ff7c27c17d6242523904f3e

Also increase attempts to 12, in line with Kolla Ansible CI.

Change-Id: I81cabf27f44af3c8135efe8e427db1ffee5f0091

This requires stackhpc.os-images v1.10.0 or newer, for compatibility
with CentOS 8 when SELinux is enabled: we disable SELinux, but without
rebooting it stays enabled.

This Ansible role was updated to v1.10.2 in master and stable/victoria
by I5efdbd52556721914fe69d7c6ba454b2c721b643, for another reason.
Remember to bump the requirement when backporting to earlier releases.

It also needs changes in the way we interact with Bifrost to avoid using
the env-vars file which has been removed. This is implemented by change
I25078e69acdf41a4ef9957f99fe5047de54b778d.

Finally, it requires building seed deployment images only after
deploying Bifrost, because the task copying images onto the seed expects
/etc/kolla/bifrost to exist.

We also copy log files to identify issues when the job fails.

Change-Id: I4719b4d397c01b35c78cb84c6d686dd27742d1c0

While we always test baremetal compute in CI, development environments
may not. Given that Ironic is now disabled by default, we should make
this work out of the box.

Story: 2008207
Task: 41003

Change-Id: Id3128380f5ff74d24265f6b2132c6d7992bf00ba

Change-Id: I5fc49fb734d0fe94f5f75c66eb4c1a935774ef30

Adds the kayobe-seed-vm-centos8 CI job to configure the Zuul VM as a
seed hypervisor, and use nested virt to provision a seed VM.  This
ensures that the seed hypervisor code paths are tested.

The job uses a Cirros image for the seed VM rather than the usual CentOS
cloud image. This is to reduce bandwidth required to download the image.
It does mean that the resulting seed VM cannot be used as a seed, but
nested virt would make this slow and unreliable anyway. Cirros does not
load cdrom drivers by default, so we add the configdrive as a disk
rather than a cdrom device.

Depends-On: https://review.opendev.org/617161

Change-Id: I2268a1ddf9a2870c713f32a40689e1686365aabd
Story: 2001655
Task: 6683

This ensures we are using the appropriate IPA images in the upgraded
environment.

Change-Id: I4a72d9ae49ad41716522c3074c16d8ca23c3ff94

* Always use Python 3
* Drop code paths for CentOS 7
* Drop support for Yum
* Remove support for host NTP daemon, always use chrony
* Switch references from 'yum_install_epel' to 'dnf_install_epel'
* Remove overcloud host image workaround for tagged VLAN admin network
* Remove the kayobe.utils.yum_install function, which is unused

Change-Id: I368f6edafed9779658798fc342116b4c1b3ffd48
Story: 2006574
Task: 39481

Backport: train, stein, rocky

This fixes issues seen with a-universe-from-nothing using stable/train.

Change-Id: Ib477de5f3af2e4c182d0c2999c274dbb5553531c
Story: 2007572
Task: 39469

Tests various non-default configuration:

* Custom users
* Network interfaces, VLANs, bridges, bonds
* Software RAID
* LVM & docker devicemapper
* timezone
* Package mirrors
* yum-cron / DNF automatic

This improved test coverage allows us to be more confident about these
features working on CentOS 8.

Change-Id: I36148e4356deb7d5ec00d8d3ebeb2d3932ff4f94
Story: 2006574
Task: 38938

Detect current branch from .gitreview and use upper constraints to
install python-openstackclient, to guarantee compatibility with the
Python version in use.

Change-Id: Ie44508fe3d3b08190afa5a43748e43548a63ff82

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: I2a7a82d7f576739c5516a0072f953712ffa5c233
Story: 2004959
Task: 29392

Add testing of the dataplane to the overcloud jobs. To support both
baremetal provisioning and VMs with VXLAN tenant networks, we use the
provision-net as our external network to which floating IPs are
attached.

We depend on a backport of parts of this patch to allow testing of SSH
connectivity after upgrades.

Depends-On: https://review.opendev.org/708915/
Depends-On: https://review.opendev.org/709145/

Change-Id: I6316e8959cff987e4e97280889e1038a9537ed32

In case of failures in kayobe-overcloud-centos, the error message fails
with:
kayobe/dev/functions: line 569: LOGDIR: unbound variable

Example:
https://zuul.opendev.org/t/openstack/build/ce1fadc3ee6d4842a599da57a670cc18

This can be reproduced with:
set -eu

if [[ -n ${LOGDIR} ]]; then
    echo "LOGDIR set"
else
    echo "else"
fi

Fix the error reporting with assigning an empty string to LOGDIR by
default.

Change-Id: Ieef73950f89e4dfb727ddc59ef2750d9b81f3c58

We no longer need to cd to the kayobe repo path, since all tested
versions of kayobe support pip installation.

Change-Id: Id7b574cbc6bb5195eee16cfc8443075712ce3254

This is to resolve the following issue in CI:

    c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
     #include <ffi.h>
                     ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/zuul/kayobe-venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-PV3WhJ/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-PV3WhJ/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ZvlZVY/install-record.txt --single-version-externally-managed --compile --install-headers /home/zuul/kayobe-venv/include/site/python2.7/cffi Check the logs for full command output.

Change-Id: I1bd724be7dc77058870f37cb1c9404472fa466ca
see: https://zuul.opendev.org/t/openstack/build/c20a316a699b4073abf75960634ebfcd

Docker CE has added a default DROP policy to the FORWARD chain.  When
nova-compute runs on the controller, kolla ansible sets the
net.bridge.bridge-nf-call-iptables sysctl to 1, which causes iptables to
process frames forwarded across bridges.

Currently, the kayobe-overcloud-centos job is failing quite frequently
with timeouts when deploying bare metal compute. Experimentation with
iptables hasn't revealed why this only happens sometimes, or exactly
what traffic is being blocked, but opening up the firewall does seem to
fix the issue. We won't see this in production since control and compute
services are on separate hosts.

This change updates the iptables configuration used in CI to forward all
frames on the bridge, breth1.

Change-Id: If96437b73b9b5c58600ba1b004f53ee0c1f14398
Story: 2006534
Task: 36590

This network conflicts with the ironic provision-net since both are flat
networks on physnet1. We don't need the external network in our CI, so
don't create it.

Change-Id: Id17d4ea00ceb684b34bf12e911758d504c520de0
Depends-On: https://review.opendev.org/#/c/670129/

Galaxy install often fails in CI, scuppering an otherwise good run.

Change-Id: I3d02afe33cdf32b1d285bd4bdc4a8074883113ca

Allows users to explicitly specify which type of tenks
deployment they wish to create / destroy.
Preserves existing behaviour with defaults.
Modifies Zuul tests to use new tenks-deploy entrypoints.

Change-Id: I9aafed8481fd7564c0fc0abe5f6b21eceb824d75

This is supported in kolla-ansible via the ironic_inspector_pxe_filter
variable, which can be added to globals.yml. The default value for that
variable changed in the Stein release from 'iptables' to 'dnsmasq',
since the iptables filter does not work with Docker CE [1].

This change removes the inspector_manage_firewall variable.

This change also adds an iptables rule in CI tests to allow DHCP packets
to be forwarded, to ensure bare metal servers can be deployed.

[1] https://bugs.launchpad.net/kolla-ansible/+bug/1823044

Depends-On: https://review.openstack.org/649673
Change-Id: Idac6777b4d97fbd17698fc2086ceb068d7b2e326
Related-Bug: #1823044

This script is used by developers to activate the kayobe virtual
environment and source the configuration's kayobe-env file. A cd to /tmp
is an unexpected outcome of running the script.

To test the location-independent installation, remove the chdir from the
zuul job tasks that execute kayobe commands.

Change-Id: I59194952901fa648382489f48dc7aafb03d3a682
Story: 2004252
Task: 29347