This can be advantageous in deployments with a data security
requirement.

Change-Id: I555ee575ccec0cfbcc4c4bcb53677796c83227e3
Story: 2007555
Task: 39410

1. Blacklist Ansible 2.9.8

Ansible 2.9.8 includes a regression on the fileglob plugin [1] that
causes the Kolla Ansible HAProxy role to fail.

This change blacklists Ansible 2.9.8 to work around the issue.

2. Use ensure-docker role instead of install-docker

The install-* roles are being deprecated and renamed to follow the
ensure-* naming convention [2].

[1] https://github.com/ansible/ansible/issues/69450
[2] http://lists.zuul-ci.org/pipermail/zuul-announce/2020-April/000071.html

Change-Id: Iab1d84e6a8c1b3dd81e53279309153687677a061
Story: 2007659
Task: 39748

Ironic inspector rules are registered both with the seed and (if using)
overcloud ironic inspector services. These tasks often show up as
changed even when no configuration changes have been made that would
affect the rules.

This is caused by inspector returning default values for fields that may
be omitted in the requested rule. This change fixes the issue by
including those defaults in the comparison.

Change-Id: Ia24e328d4531201d76a65b6385e4463bb1f3c5c6
Story: 2007399
Task: 38997

Sets 'monasca_install_type: source' to remove need
for kolla-ansible var boilerplate.

Also use default Monasca parameters to configure
Grafana post deploy.

Change-Id: I2b6d62104c9c127cb8f6b4f4930dd695cd00da17
Story: 2007597
Task: 39587

A common failure early on when using Kayobe is during discovery of SSH
known hosts. This happens if a host does not have an IP address
configured on the admin (SSH) network. The failure looks like this:

PLAY [Ensure known hosts are configured]
**********************************************************************
TASK [ssh-known-host : Scan for SSH keys]
**********************************************************************
failed: [compute0 -> localhost] (item=) => {"ansible_loop_var": "item",
    "changed": false, "cmd": ["ssh-keyscan"], "delta": "0:00:00.013855",
    "end": "2020-04-17 10:51:01.857855", "item": "", "msg": "non-zero
        return code", "rc": 1, "start": "2020-04-17 10:51:01.844000",
    "stderr": "usage: ssh-keyscan [-46cDHv] [-f file] [-p port] [-T
        timeout] [-t type]\n\t\t   [host | addrlist namelist]",
    "stderr_lines": ["u sage: ssh-keyscan [-46cDHv] [-f file] [-p port]
        [-T timeout] [-t type]", "\t\t   [host | addrlist namelist]"],
        "stdout": "", "stdout_lines": []}

This happens when ansible_host is an empty string, typically because the
host has no IP address defined in for the admin network in
network-allocation.yml. This is very confusing for a new user. We should
provide a more informative message.

It's not exactly clear how a user gets to this point, since the
ip-allocation.yml playbook runs before ssh-known-host.yml, which should
populate network-allocation.yml.

This change detects this failure mode and provides a message with
information about how to resolve it.

Change-Id: I564b6e4509a30dec7c49a23bb2f75d490be775ed
Story: 2007566
Task: 39456

Kayobe has a workaround for CentOS cloud images which contain a bogus
nameserver entry in /etc/resolv.conf. By setting
overcloud_host_image_workaround_resolv_enabled to true, the entry would
be removed. Previously we removed a specific IP address - 10.0.2.3 -
that was present in the CentOS 7 images. However, it seems that CentOS 8
images have a different IP - 192.168.122.1.

This change fixes the issue and becomes resilient to future changes by
matching any IP address. This should be fairly safe, since this
workaround is opt-in.

Change-Id: I9323a38cb2bb627ff56f5713900be00595ea8d4b
Story: 2006574
Task: 39484

Kayobe generates passwords.yml for Kolla Ansible, and can encrypt it
using the vault password. Previously this was failing on Python 3 due to
passing a string to file.write() which expects bytes.

This change fixes the issue by encoding the password string passed to
file.write().

This allows us to run the ansible role tests under Python 3.

Change-Id: I33813f79984a46f1967ef3aee455dcfbe7eb93da
Story: 2006574
Task: 39481

We can use the Ansible pip module's support for specifying a list of
packages with version constraints.

Change-Id: If5d3c7117175732c54e38025692eb4c036053ebc

Previously, Kayobe used Kolla Ansible's bootstrap-servers command to
create a user account and Python virtual environment for Kolla Ansible.
In order to do this it used the Kayobe Ansible user and Python
interpreter.

This causes problems for Ansible fact caching, which needs separate
caches for Kayobe and Kolla Ansible, since the different users and
Python interpreters used result in different facts. Bootstrapping
servers with the Kayobe user and interpreter resulted in the Kolla
Ansible fact cache being populated with Kayobe's user and interpreter.

This change disables user creation during Kolla Ansible's
bootstrap-servers command, instead creating the user and virtual
environment in Kayobe prior to running the command. This allows the
bootstrap-servers command to be executed using the normal Kolla Ansible
user and interpreter, which results in the correct facts being gathered.

The downside here is some duplication of code and configuration, but a
nice side effect is that we no longer need to dump configuration in the
CLI for host configure in order to fetch the Ansible user and
interpreter.

Change-Id: I85670be7242bc436f73c689f027670b0938ba031
Story: 2007492
Task: 39444

Sync with kolla-ansible multinode inventory.

Change-Id: I30bd5286c4783fce544c41e726efc5f800d6f56a

* Change default seed VM image to CentOS 8
* Change default bifrost deploy image to CentOS 8
* Workaround DIB bug
  https://bugs.launchpad.net/diskimage-builder/+bug/1866847 by setting
  DIB_DISABLE_KERNEL_CLEANUP to 1
* Install iptables on seed for SNAT - missing on CentOS 8
* Fix provider network MTU lookup for empty string
* Bump stackhpc.libvirt-host to 1.7.0 for CentOS 8 support
* Bump stackhpc.libvirt-vm to 1.13.0 for CentOS 8 support
* Bump jriguera.configdrive for Python 3 support

Change-Id: Ie0edf6a924a914395c6502e2d5cf1139bce14a48
Story: 2006574
Task: 39000

Some Ruckus switches, e.g. the Ruckus ICX 7150, advertise switch
interface names as switch port descriptions. Unlike Dell switches, there
is no space character between port type and port number. For example:
GigabitEthernet1/1/9.

Update regular expression to match both styles.

Change-Id: I359b07abadc8665ff0a8c3407ca0fc5effc504cf
Story: 2007532
Task: 39343

The seed VM will fail to provision if the Ansible control host and the
seed hypervisor are not the same hosts.

This is because Kayobe creates the seed-vm-user-data file on the
seed-hypervisor host. It then invokes the jriguera.configdrive role
which uses a copy task without remote_src, which fails to find the
source file locally on the Ansible control host.

Instead we create a local temporary file for seed VM user data.

Change-Id: Iabbe4c624b9ad02bb82c323070f99c16e5822966
Story: 2007530
Task: 39338

One way to improve the performance of Ansible is through fact caching.
Rather than gather facts in every play, we can configure Ansible to
cache them in a persistent store. An example Ansible configuration for
doing this is as follows:

[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./facts
fact_caching_timeout = 86400

While this mostly just works, there are a few places where we
unconditionally gather facts using the setup module. This change
modifies these to only gather facts when necessary.

We no longer execute the MichaelRigart.interfaces role using become:
true, since it may gather facts and we do not want it to do so as root.
The role uses become where necessary.

Change-Id: I9984a187fc6c0496ada489bb8eef36e44d695aac
Story: 2007492
Task: 39216

Adds a new variable, 'kolla_enable_openstack_core', which can be set a
default value for whether the default OpenStack services are enabled.
This includes Glance, Heat, Horizon, Ironic, Keystone, Neutron and Nova.
It is 'true' by default.

Change-Id: I7768d3a92272d4353522dbf1a96f124225f4d73d
Story: 2007524
Task: 39315

Kolla Ansible sets kolla_{external,internal}_fqdn_cacert variables with
default values compatible with the use of `kolla-ansible certificates`.

However, when these variables are left unset in Kayobe, which is
generally the case when using trusted certificates, we end up with
openrc files setting OS_CACERT to a file that does not exist:

    ${KOLLA_CONFIG_PATH}/certificates/haproxy-ca.crt

Instead we allow null cacert variables to be passed to kolla-ansible,
which results in openrc files without the bogus OS_CACERT entry.

Change-Id: Ifa615888b6d8d54c9e6314fd90f3fc4872fc6e5a
Story: 2007516
Task: 39299

There is no activity on the resmo fork of the role and it seems
impossible to get any patches merged.

Change-Id: I1f09f7c11767226e89b34687dab1553e87be76ba
Story: 2005272
Task: 39197

Using become for all Kolla Ansible tasks is not ideal from a security
perspective. It is also incompatible with fact caching, since it causes
facts to be gathered and cached as root, which changes some facts.

This change modifies the default value of kolla_ansible_become to false.

Change-Id: I9ee5c55e59276f70c92e9c698c01123dcf8919a1
Story: 2007492
Task: 39217

This is a minimal fix to support loading dashboards into the Monasca
Grafana fork. It firstly aligns the default Monasca Grafana control
plane organisation and Monasca Grafana local admin username with Kolla
Ansible to make the feature easier to use. Secondly, it extracts the
associated OpenStack project name from this variable by stripping off
the OpenStack domain.

Longer term we may wish to move the dashboard loading functionality into
Kolla Ansible, now that it supports deploying Monasca.

Affects Rocky onwards.

Change-Id: I77c94edf654565a12ce8be681e3c9b16caa55c86
Story: 2007477
Task: 39186

Implemented via 'kolla-ansible stop'.

Change-Id: Iaf8db47e70f023b446c17aa61fc8bb89cf7c2b28
Story: 2007467
Task: 39155

Adds support for configuration of DNF repo mirrors for CentOS and EPEL
repositories, as well as custom repositories.

Adds support for DNF automatic, which is a replacement for yum-cron.

Configuration is backwards compatible, falling back to the equivalent
yum variables when DNF variables have not been overridden.

Change-Id: I8bef5e9c8e1c77c25d6077ff690da8f2cde6a643
Story: 2006574
Task: 38922

It leaves certain ceph mentions in globals.yml.j2 as it needs
syncing with kolla-ansible contents anyways
(these are all comments).

Change-Id: I05e9c6223583e9bb5dc0020edc0b56990275093c
Story: 2007295
Task: 38766

CentOS 8 removes interfaces from their bridge during ifdown, and removes
the bridge if there are no interfaces left. When Kayobe bounces veth
links plugged into the bridge, it causes the bridge which has the IP we
are using for SSH to be removed. Use a dummy interface in CI to avoid
this problem.

Kolla-ansible has dropped all CentOS 7 jobs on master now, and prechecks
only allow CentOS 7 hosts. Drop all CentOS 7 jobs. We will have to run
without upgrade jobs in place until Train supports CentOS 8.

Depends-On: https://review.opendev.org/695881

Change-Id: I7c1a885b36445e33d4db1b1c8533db28a644b4a1
Story: 2006574
Task: 38870

Backport: train

OpenStack Ansible modules were broken in Ansible 2.8.9. This affects
kolla-toolbox, and kayobe's ansible install for tasks that interact with
APIs. See https://github.com/ansible/ansible/issues/68042 and
https://bugs.launchpad.net/kolla/+bug/1866181.

This change blacklists ansible 2.8.9 for Kayobe and Kolla Ansible. A
separate change will be made to kolla to blacklist ansible 2.8.9 from
the kolla-toolbox image.

Depends-On: https://review.opendev.org/711485

Change-Id: I535ab240b7ab8f3ab104b49170e4a9ee01fc482b
Story: 2007383
Task: 38959

CentOS 8 does not provide an ntp package. Instead fall back to using the
chrony container provided by Kolla Ansible by default.

Depends-On: https://review.opendev.org/711511

Change-Id: If5230854d7565c8b3c91a46da4795c63edf095e4
Story: 2006574
Task: 38866

We enable ntpd by default, and provide a variable to disable it -
ntp_service_enabled. It is also automatically disabled if the user
enables the chrony container (kolla_enable_chrony).

However, setting ntp_service_enabled to false will cause the host
configure commands to fail due to a bug in the resmo.ntp role. This is
because it tries to configure the ntpd service in systemd, but it will
not exist so the task fails.

This change fixes the issue by skipping the resmo.ntp role if the NTP
service is disabled.

Change-Id: I640873c11ceae5008030dc03984c089a410a0cee
Story: 2007384
Task: 38968

This is ignored by Libvirt, so there is no need to pass it to the
libvirt-host role.

Note: omitting the capacity argument requires libvirt-host 1.3.1 or
later.

Change-Id: Id94e7b514ed36c8e042e56e8f1891d98f8371e5d
Story: 2007381
Task: 38957

This extends the physical network configuration in Kayobe to configure
Cumulus physical switches using the nclu Ansible module.

Change-Id: I960027ead301c5793a0ada1959a23549a71bdbfb

ncclient 0.6.7 has been released and includes a fix [1] for the host key
checking issue that required us to pin to 0.6.2.

Restrict the package to <0.7.0 to avoid potential breakage from new
releases of ncclient.

[1] https://github.com/ncclient/ncclient/commit/ead7b640921ef9e1af4aab698ba287c3b8a48553

Change-Id: Ia665cffb11253f58bbdce7ea9892766c36f7af40
Story: 2006378
Task: 38765

Updates the minimum version of Ansible from 2.6 to 2.8, and the maximum
supported version from 2.8 to 2.9.

CentOS 8 requires Ansible 2.8.

Change-Id: I3f8f7f8d7d37e3cb851965a491ac9c43030869d5
Story: 2006574
Task: 38826

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: I2a7a82d7f576739c5516a0072f953712ffa5c233
Story: 2004959
Task: 29392

Change-Id: I2547ef2556ca96b614854515069aaca3349cd692

Currently we install python dependencies on the Ansible control host
each time the ip-allocation and console-allocation roles are executed.
This is inefficient, particularly in the case of the ip-allocation role
which is run serially for all hosts. It is also unnecessary since we
have these packages available in the Python environment used to execute
kayobe.

The kolla-ansible role also has an implicit dependency on PyYAML for
managing kolla passwords.

This change uses ansible_playbook_python as the Python interpreter for
the necessary tasks in these roles to avoid installing dependencies on
the system on CentOS 8 and Ubuntu. For CentOS 7 we still need to use the
platform Python, due to needing SELinux bindings.

Change-Id: Ic6a1c69a34241f4fbe617a0b12aec9b1528ba352
Story: 2006574
Task: 38825

Kayobe overcloud introspection data save fails because the dynamic
inventory script siliently breaks causing Ansible to parse it as a
static inventory file. The failure occurs due to OS_TOKEN being set.
This change works around setting OS_TOKEN before running the dynamic
inventory script by setting OS_CLOUD before querying inspector.

Confirmed on Stein and Train, and verified in both environments.

Story: 2007326
Task: 38846
Change-Id: I57fbf91ae3440d3e4e6a64cd7d05151e299c9322

These roles are no longer necessary now that Ansible supports setting
ansible_python_interpreter via a task- or role-scoped variable.

Change-Id: I4121d01dc83ac028350d4d98d3e1158e15fdfd63
Story: 2006574
Task: 38824

Upstream Ansible OpenStack modules now use openstacksdk rather than shade.
Switch local Ansible modules to follow suit. Also switch to use the
stackhpc.os_openstacksdk role from stackhpc.os-shade.

The stackhpc.os-shade role is removed during 'kayobe control host upgrade'.

Change-Id: Id3894c3c36ef99f00ed463de6a3457e11733d6b7
Story: 2007294
Task: 38759

One use case is to use seperate disk for the registry storage. This
can prevent the rootfs from filling up.

Change-Id: I9634ee7f5730e93b8ddd96de04982d638dd4dae2

This reverts commit a93b85ba.

The local Python executable for Kolla Ansible is changed to Python 3
because Kolla Ansible master no longer supports Python 2.

Change-Id: I768ce8db9cec1c70d94f271997bbcc64d370403e

The default is still Python 2. This is a necessary prerequisite for using
the master branch of kolla-ansible, which requires Python 3.

Change-Id: Ida5b60b723c8208bb7305c3d669eafdab6dbbe01
Story: 2004959
Task: 38767

Kolla recently upgraded bifrost from 7.0.0 to 7.1.0 on the stable/train
branch. This switched to IPA builder to build the IPA image, and
introduced a rename of the IPA kernel file from ipa.vmlinuz to
ipa.kernel, which breaks overcloud provisioning. The iPXE kernel
download fails with a 404, since Kayobe introspection rules use
ipa.vmlinuz for the driver_info.deploy_kernel URL.

This change works around the issue by setting two Bifrost variables,
ipa_kernel and ipa_kernel_url, to reference the old kernel filename of
ipa.vmlinuz. This works both in the case where the image is downloaded
from a URL (ipa_kernel sets the destination file name), and where it is
built via 'kayobe seed deployment image build' (kayobe uses the legacy
ironic-agent DIB element rather than IPA builder, which creates a
hardlink to ipa.vmlinuz).

We chose the above approach rather than switching to IPA builder due to
it being a less risky change at a time close to release. A future
release of Kayobe should switch to IPA builder, but this will be a
larger effort.

[1] https://review.opendev.org/#/c/692200/1/playbooks/roles/bifrost-ironic-install/defaults/main.yml

Change-Id: I7f75c25602fd7ae4bfeb6abbdd3b42d8ee465abf
Story: 2007068
Task: 37951