This commit changes seed-images jobs to build seed and overcloud
container images, each with a customisable regex.

The rocky9 job continues to build only the base image, although for both
overcloud and seed instead of seed only.

The other seed-images jobs (centos9s and ubuntu-jammy) now build all
default overcloud images. This takes longer and can fail for various
reasons unrelated to Kayobe, but this is fine since the jobs are marked
as experimental.

This is to attempt to catch bugs in the image build code, such as the
ones fixed by [1].

Note that the ubuntu-jammy job does not build the bifrost-deploy seed
image because the build is currently timing out. The centos9s job builds
it successfully.

Also update release documentation to remind developers to run
experimental jobs.

[1] https://review.opendev.org/c/openstack/kayobe/+/921012

Change-Id: Idf705d9cf41766a897444898ac31c4635b70ec16

Enable that by default in CI

Change-Id: I654f4dfa73afee478fb2c39594dc6f36081fb7c0

Change ``ipa_build_dib_elements_default`` and
``ipa_build_dib_env_default`` to use ``os_distribution`` and
``os_release`` by default. This allows for Ubuntu images to be built
when running on Ubuntu.

Rocky will still build CentOS images, as Rocky IPA images have not been
tested yet.

Change-Id: Iefd2d0b7a3a3e07f5c112d58e2ec0b3da0a747d3

CentOS Stream 8 support has been dropped. Migration path will be present
in Yoga release - as a followup change.

MichaelRigart.interfaces does not support custom routes for
NetworkManager yet. It has been disabled in CI for Rocky Linux 9
temporarily.

Non-voting CentOS Stream 9 CI overcloud job is using RL9 container
images (as kolla CI is not building CS9 images anymore).

Change-Id: Idf5ee822b03ba40179803c981500a6bad37594bf

The disable-selinux role has been renamed to selinux and now supports
setting desired state.

Previously Kayobe was defaulting to disabling and rebooted the host - to
avoid audit logs filling up. This change allows operators to define
desired SELinux state and defaults to permissive - to adhere to those
site policies that require SELinux to be at least in permissive state.

Change-Id: I42933b0b7d55c69c9f6992e331fafb2e6c42d4d1

Change-Id: Ibf4b928222713bedb7e856307f5ad91e60953795

Disk and container image builds tend to be fairly unreliable.
With 3 voting seed jobs all building images, this can introduce
instability into the CI jobs.

This change adds a non-voting kayobe-seed-images-centos8s job, which
does the following:

* Builds IPA images
* Builds an overcloud host image
* Builds a base container image

Similar Rocky and Ubuntu jobs are added to the experimental pipeline,
and may be run by commenting 'check experimental' in gerrit.

The existing kayobe-seed-* jobs no longer build images.

Change-Id: Idecda342f3ab86733e8d59061458d44af834dbb0

The contextfilter decorator was deprecated in jinja2 3.0.0, and has been
dropped in 3.1.0. This results in the following warning, and failed
attempts to use filters:

    [WARNING]: Skipping plugin (networks.py) as it seems to be invalid:
    module 'jinja2' has no attribute 'contextfilter'

This change switches to use the pass_context decorator. The minimum
version of Jinja2 is raised to 3 to ensure pass_context is present.

This change also includes some changes to address issues with image
builds in CI, caused by CentOS Scream.

1. disable IPA image builds in seed deploy jobs

IPA image builds will be split out into a separate job. For now, disable
them.

2. disable overcloud host image builds in seed deploy jobs

Overcloud host image builds will be split out into a separate job. For
now, disable them.

Depends-On: https://review.opendev.org/c/openstack/kayobe/+/835279
Change-Id: If657bf5b0117812d3c53942464cc41cf86cc8ad5

We build IPA images and a deployment image in the seed jobs, so we don't
need to download Cirros or IPA images. Also, these downloads depend on
external resources which may make jobs less reliable.

For seed upgrade jobs, disable IPA and deployment image downloads.

Change-Id: Ib59c8bc2d8938eca18c943bb2e66ed185152a739

The kayobe-seed-ubuntu-focal job is currently fairly unreliable, often
failing to build the base container image.

We are not using the mirrors provided by OpenDev infra, which may be
making these builds less reliable.

This change disables container image builds in CI on Ubuntu. It should
be reverted if they are made more reliable.

Change-Id: I648fa6423ad9ff43120c7808f080b0359ad8621c

This change uses the new Galaxy requirements file in Kolla Ansible to
install the openstack.kolla collection.

Cross-project dependencies on ansible-collection-kolla are supported.

Story: 2009854
Task: 44504

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/819430
Change-Id: Iac185dd2bbbca128c6cf71b2734e94b3e1c6133b

As a first step towards supporting multiple overcloud disk images, this
change introduces a new command to build a disk image directly with DIB:
`kayobe overcloud host image build`.

It also disables building a root disk image during Bifrost bootstrap if
overcloud_dib_build_host_images is set to true.

Change-Id: I93d242889e225b4e60254f6b9cc5eeb457294ac8
Story: 2002098
Task: 41693

Currently we do not have any coverage of building container images. This
change adds a basic test of building the base image to the seed jobs.

A potential future enhancement could involve pushing to a local registry
running on the seed.

Change-Id: Id7378dd844aca7cb5634535308eba0f60342c81d

We can now rely on the os_distribution variable to set sensible defaults
for Ubuntu. It is set automatically in kayobe-config-dev.

The interfaces pause workaround is no longer required since switching to
systemd-networkd.

Depends-On: https://review.opendev.org/c/openstack/kayobe-config-dev/+/788475
Change-Id: I76d658b93b9195c760c3a91fe5aee96bad8d6b42

Based on the discussions from recent Kolla meetings [1][2][3],
we established that, to avoid the DockerHub pull limit issues
in our CI, we will try using quay.io.

The images have already been published via Kolla CI
for master/Wallaby [4] and Victoria [5].

This change applies only to CI.
The user-facing move is yet to be discussed.

[1] http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-03-17-15.00.log.html#l-102
[2] http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-03-24-15.00.log.html#l-137
[3] http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-03-31-15.01.log.html#l-50
[4] https://review.opendev.org/c/openstack/kolla/+/781130
[5] https://review.opendev.org/c/openstack/kolla/+/781899

See related Kolla Ansible change
I22e11a994801447526ac26fc2815a1679876225e

Change-Id: Id6690dc9a486af0ed7d42555d84cf77d6de0b6af

This commit adds Zuul job for deploying seed services against
Ubuntu 20.04 LTS (Focal Fossa).

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Story: 2004960
Task: 41807
Depends-On: https://review.opendev.org/c/openstack/kayobe/+/772929
Change-Id: I839cd1b9d63c9bbf3e0c75def76b8bff68637c75

* Use source images
* Need to specify bash for &> syntax

Issues worked around:

* Manually configuring bridge via ip commands makes ifup fail to bring
  up the link. Adds a kayobe-network-bootstrap Zuul CI role that adds
  persistent configuration for the all-in-one network.

* bridge not active after interfaces role bounce. Added a pause, similar
  to https://github.com/michaelrigart/ansible-role-interfaces/pull/31

* fails installing docker python module for kolla user. WARNING: The
  repository located at mirror-int.ord.rax.opendev.org is not a trusted
  or secure host and is being ignored ERROR: No matching distribution
  found for docker===4.4.0 Adding trusted host for PyPI mirror.

* Tenks fails to create block devices - missing qemu-img (in qemu-utils)

* Tenks qemu emulator is different on Ubuntu

Remaining issues:

* Bare metal testing is unreliable on Ubuntu - some jobs see IPMI
  failures such as the following:

    ipmitool chassis bootdev pxe

    Error setting Chassis Boot Parameter 5\nError setting Chassis Boot
    Parameter 0\n

  Bare metal testing is disabled on Ubuntu for now.

Depends-On: https://review.opendev.org/766984
Depends-On: https://review.opendev.org/766958

Story: 2004960
Task: 29393

Change-Id: I1985efae7c18f55c3ff7c27c17d6242523904f3e

Change-Id: Id5ed0b594d6270c4a03606caa776bbb291be2694

This workaround is no longer needed for current CentOS 8 images as
resolv.conf file comes now empty.

Moreover, the workaround has been preventing seed host configure from
running successfully on Ubuntu hosts.

Change-Id: I30e34e3c58db3f3a93a07f3d6d53671286e4260e

This requires stackhpc.os-images v1.10.0 or newer, for compatibility
with CentOS 8 when SELinux is enabled: we disable SELinux, but without
rebooting it stays enabled.

This Ansible role was updated to v1.10.2 in master and stable/victoria
by I5efdbd52556721914fe69d7c6ba454b2c721b643, for another reason.
Remember to bump the requirement when backporting to earlier releases.

It also needs changes in the way we interact with Bifrost to avoid using
the env-vars file which has been removed. This is implemented by change
I25078e69acdf41a4ef9957f99fe5047de54b778d.

Finally, it requires building seed deployment images only after
deploying Bifrost, because the task copying images onto the seed expects
/etc/kolla/bifrost to exist.

We also copy log files to identify issues when the job fails.

Change-Id: I4719b4d397c01b35c78cb84c6d686dd27742d1c0

CentOS 8 removes interfaces from their bridge during ifdown, and removes
the bridge if there are no interfaces left. When Kayobe bounces veth
links plugged into the bridge, it causes the bridge which has the IP we
are using for SSH to be removed. Use a dummy interface in CI to avoid
this problem.

Kolla-ansible has dropped all CentOS 7 jobs on master now, and prechecks
only allow CentOS 7 hosts. Drop all CentOS 7 jobs. We will have to run
without upgrade jobs in place until Train supports CentOS 8.

Depends-On: https://review.opendev.org/695881

Change-Id: I7c1a885b36445e33d4db1b1c8533db28a644b4a1
Story: 2006574
Task: 38870

Currently Kayobe stderr is logged to the Zuul playbook output. This
change redirects stderr to the ansible log files.

Change-Id: Ia2f25a2e41859159275d418b57ab59bb97f58d6a

Kayobe uses a number of virtual environments on the remote hosts for
python dependencies such as shade, python-openstackclient, docker, etc.
By default these are stored in /opt/kayobe/venvs/. Typically we do not
provide version restrictions when installing these packages, so over the
course of time they may become stale and incompatible.

This change installs the latest version of packages allowed by OpenStack
upper constraints.

It also adds a new variable, 'pip_upper_constraints_file', to set the
upper constraints file. The existing variable
'kolla_upper_constraints_file' now defaults to the value of
'pip_upper_constraints_file'.

Change-Id: I8d2956f95bbc44b5a9e88e7569372048a62f12f5
Story: 2005923
Task: 34193

Adds a new abstract parent job for the following kayobe deploy jobs:

kayobe-overcloud-centos
kayobe-overcloud-upgrade-centos
kayobe-seed-centos
kayobe-seed-upgrade-centos

This parent includes commonly used variables, and pre-run and post-run
playbooks for preparing and executing diagnostics, plus other
initialisation.

Change-Id: I9ab89fbc49bb32d86af56d50ec3914740e65d0f8

The project infrastructure is now hosted at opendev.org.

Kayobe projects have been moved from the openstack/ namespace to x/.

Change-Id: I16b523285eb34c1cd0eaf920aa4512d0ae1b4f05

This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.

Use master branch of:

* Kolla
* Kolla ansible
* Bifrost
* IPA
* OpenStack services
* Requirements

Also, use the infra docker registry mirror at port 8082 which does not
have any path component in its URL, and so works with Docker CE.

Depends-On: https://review.openstack.org/#/c/651254/
Change-Id: Icdb62aa0083d9272a08421138719cf5f6b52ec5a
Story: 2001864
Task: 14316

This script is used by developers to activate the kayobe virtual
environment and source the configuration's kayobe-env file. A cd to /tmp
is an unexpected outcome of running the script.

To test the location-independent installation, remove the chdir from the
zuul job tasks that execute kayobe commands.

Change-Id: I59194952901fa648382489f48dc7aafb03d3a682
Story: 2004252
Task: 29347

The mirror caches images from Dockerhub, and should reduce load on the
cloud and improve download times.

Change-Id: I2b52fa6371118e28c1d601e8b90065897f24f41d

tenks-deploy.sh deploys a minimal virtualized baremetal test cluster
locally. It also adds it to the overcloud-base CI job. To make the new
CI job work, we need to configure the firewall on the test machine to
allow the baremetal machines to communicate with the openstack services.

Change-Id: I7487a2606cf0bac71c5c63db41b2b518a6f6398b
Depends-On: https://review.openstack.org/#/c/615939
Depends-On: https://review.openstack.org/#/c/618003
Story: 2004297
Task: 27850

There is currently no coverage of upgrades in CI, which leaves us open
to regression in this infrequently tested but crucial area. This change
adds the required scripts and Zuul configuration.

A control plane is first deployed using the Kayobe stable/pike branch
and associated default configuration. The control plane is tested by
deploying then deleting a server instance. An upgrade to Queens is
performed, using the Kayobe master branch, or code in review if
applicable. The upgraded control plane is tested by deploying then
deleting a second server instance.

A workaround was required to restart the nova_compute service after the
upgrade, since the SIGHUP sent to it by Kolla Ansible during upgrade
appeared to be putting it into a degraded state.

A future improvement to this test could be to leave a server instance
running during the upgrade.

Change-Id: I0e595524e39d1131fe3ec6aaf2aeec3ff3d6a536
Story: 2003472
Task: 24732

The Kayobe development configuration has moved to
openstack/kayobe-config-dev. It was previously hosted at
stackhpc/dev-kayobe-config on Github.

This change updates the Zuul configuration and development documentation
to use the new location. It also uses a location for the kayobe source
code that works when kayobe is not the repository against which the code
review is targetted. This allows us to run kayobe jobs for
kayobe-config-dev changes.

Configuration for the seed VM and seed hypervisor development
environments is not yet supported by kayobe-config-dev, so the
documentation references a branch in my personal fork on Github.

Change-Id: I621324093b8dff0ac52a648539cd91bc12da4862

Change-Id: I9affd65fea2009404a32825000e74f5edea24832

We no longer need to use devicemapper with a loopback-mounted LVM since
Bifrost can now support the overlay driver.

Change-Id: I043d3be303e337a1fcbbca6e6edc5abc5dcb9c1b

This allows us to use the Depends-On syntax in commit messages to depend
on changes in kolla, kolla-ansible, and requirements.

Change-Id: I0c863d46f65de24c04ba9ea445473436b146c424

This can help in analysing failures.

Change-Id: I9910f89fbeb5698a66659a835041dc564e2b64ae

Adds a new CI job, kayobe-seed-centos, that deploys a single VM as a
seed host. The VM provided by Zuul is configured as the seed, rather
than a seed hypervisor that provisions a seed VM using nested virt.

This approach should be more reliable, faster, and easier to get
working, at the expense of not testing the seed hypervisor code paths.

Currently the seed services are deployed, but not tested. A future
change could test discovery and provisioning of an overcloud host using
the seed.

Change-Id: Iefe16215af6f1a9fb8ec78094cd6bd76a82a040a
Story: 2001655
Task: 6683