This fixes Debian job failures during image building.

See https://github.com/docker/for-linux/issues/711
for upstream details.

Change-Id: Icf3ffb261605ffe5d8f2618c2ed4cb97db97dd49

Change-Id: Ie7ee70e19536c2d8f4300df55c9c6ca38abb7ae4

Tests the following operations for MariaDB:

* Stop
* Recovery

Backup and restore will be added in a separate change.

Depends-On: https://review.opendev.org/693329
Change-Id: I836d91554715cce0e82c1bbebb7430c457418b2d

This playbook is neither kolla-ansible nor Zuul. Much of what it does is
stale and no longer required. Remove it.

Change-Id: I6fed9d8f2439a53c6ac35c670b56b43579236ee9

Starting with the Train release, kolla_ansible (the package) must be
available in the PYTHONPATH to run most playbooks.
It is documented.
This change makes the error message friendlier.

Change-Id: Icf0399d21b3fde8d530d73e6e7ee4a57665da276
Closes-bug: #1849943

Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689



Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Change-Id: I0628b16e3ebdb3fa8196acdc1bd9c63e75bcfb09

Sometimes as cloud admins, we want to only update code that is running
in a cloud.  But we dont need to do anything else.  Make an action in
kolla-ansible that allows us to do that.

Change-Id: I904f595c69f7276e71692696471e32fd1f88e6e8
Implements: blueprint deploy-containers-action

This is required for the dict2items filter.

Change-Id: I60a04e839bf06506ff36c2631a286130d5fde972

Add Masakari testing into the Gate.

Change-Id: I52df33f963e7d2ae4059887df3d24d9e6642134e
Depends-On: https://review.opendev.org/#/c/615469/
Depends-On: https://review.opendev.org/#/c/615715


Implements: blueprint ansible-masakari
Co-Authored-By: Gaëtan Trellu <gaetan.trellu@incloudus.com>

We install kolla-ansible requirements in Zuul's Ansible playbooks.
This patch cleans up the installation in scripts so that they are
only concerned with auxiliary requirements:
- ansible (since we do not track it in requirements)
- ara (for log summaries)
- openstack clients (for first init and tests after deployment)

Additionally this patch installs openstack clients in a separate
virtualenv.
Note that all kolla-ansible requirements, ansible and ara are still
installed system-wide.

Change-Id: Iac04082ad39a9d823c515ba11c5db9af50ed225f
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Skip creation by setting ENABLE_EXT_NET to 0.

Since adding errexit we are failing in kayobe CI, since we have a
conflicting flat network on physnet1.

Change-Id: I88429f30eb81a286f4b8104d5e7a176eefaad667

We have a minimum supported version of Ansible, currently 2.5. We should
test this in addition to the latest version. This change tests latest on
Ubuntu, and minimum on other distros.

Change-Id: I45a7173139f057177a71e919ad3e718a99d9f87b

Otherwise ara had only the stderr part and logs only the
stdout part which made ordered analysis harder.

Additionally add -vvv for the bootstrap-servers run.

Change-Id: Ia42ac9b90a17245e9df277c40bda24308ebcd11d
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

This performs the same as a deploy-bifrost, but first stops the
bifrost services and container if they are running.

This can help where a docker stop may lead to an ungraceful shutdown,
possibly due to running multiple services in one container.

Change-Id: I131ab3c0e850a1d7f5c814ab65385e3a03dfcc74
Implements: blueprint bifrost-upgrade
Closes-Bug: #1834332

Some kolla-ansible jobs failed due to using external mirrors
instead of local ones.
This was due to not using the template override provided by kolla.
This patch fixes that.

Depends-On: https://review.opendev.org/668226


Change-Id: I27f714fdf05e521aa8ce25c5683a452ceb35eeb8
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

This is necessary for some Ansible tests which were renamed in 2.5 -
including 'version' and 'successful'.

Change-Id: Iacf88ef5589c7571fcf56ba8b99d3dbe76975195

Previously we sourced this script in tests/deploy.sh, but this was
recently changed. Following that change we lost the errexit setting,
meaning we ignore errors in init-runonce.

Adding errexit in the script itself means that all callers get error
handling.

Also log init-runonce output.

TrivialFix

Change-Id: I9b35bd5f0f76eec26ddd968d093a3a5fd55a7ce2

This patch removes old, unused setup scripts.
They belong to kolla anyway.

Change-Id: Ia8dcd39991177153af61c638c7ae05123da08e82
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Adds four new CI jobs for testing centos/ubuntu binary/source deploys
with ironic enabled. These are run only when there are changes to the
ironic role.

Performs some simple testing by creating a node using the fake-hardware
hardware type and creating a server.

Change-Id: Ie669e57ce2af53257b4ca05f45193cb73f48827a
Depends-On: https://review.opendev.org/664011

We should use become: true for all tasks that use Docker.

Change-Id: I5ce06cc6f2c7403a1c36aadf9e55068c780f05ac

Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496

We made it default in Queens.

Change-Id: I52de6e041bbc4c316d332cbedeca924a7f3c7346

Add CI jobs for testing an upgrade of a multinode system with Ceph
enabled. As for the existing upgrade job, we upgrade from the previous
release to the current release.

Change-Id: I931772ca4c63757769467a57c80dc0726a11167a
Depends-On: https://review.opendev.org/658163

ARA 1.0 will be released in the near future and isn't backwards
compatible. Pin it so it doesn't break things unexpectedly.

ARA ships simple setup modules to help figure out the paths
to where things are located.
These are backwards compatible from ARA 1.0 to 0.x.

Change-Id: I3fe3f4082279c2fd9a629605619a97aa5f5b0b73

'Bootstrapping' was spelt with one p - added the second p so the
word becomes a verb nicely.

TrivialFix

Change-Id: I126a5c253408af70d6d0a3be6e59270f385a00e3

This patch adds two new jobs:

* kolla-ansible-centos-source-upgrade
* kolla-ansible-ubuntu-source-upgrade

These jobs first deploy a control plane using the previous release of
Kolla Ansible, then upgrade to the current release.

Because we can't change the branch of the git repository on the Zuul
executor, we change the branch of the kolla-ansible repository on the
primary node to the branch of the previous release, in this case
stable/rocky. A new remote-template role has been added that supports
generating templates using a remote template source, to generate config
files using the previous kolla-ansible branch.

If the change being tested depends on a kolla change for the current
branch, then we build images. Rather than using the current
kolla-ansible version to tag the images, we now tag them with
change_<gerrit change ID>. This is because the version of kolla-ansible
will change from the previous release to the current one as we upgrade
the system.

Finally, it should be noted that the 'previous_release' variable in the
Zuul config needs to be updated with each release, since this sets the
release of kolla-ansible that is installed initially.

Depends-On: https://review.openstack.org/645089/
Depends-On: https://review.openstack.org/644250/
Depends-On: https://review.openstack.org/645816/
Depends-On: https://review.openstack.org/645840/
Change-Id: If301e0affcd55360fefe3b105f023ae5c47b0853

Depends-On: https://review.openstack.org/#/c/642958
Depends-On: https://review.openstack.org/642984
Change-Id: If795a9eb3ec92f75867ce3f755d6b832eba31af9

Tests Cinder with an LVM/iSCSI backend, including attaching a volume to
an instance.

Change-Id: I6963b5e81d7ac88dd1bdfb94b62d9d63028b9786

Change-Id: Id8276448c6e779b2b4a0aafee45d953c4f009fc1

The path /var/lib/docker/volumes/kolla_logs/_data/ is too long
shorter log path will help to debug from log.
The volume path is compatible with docker-engine and docker-ce.

Change-Id: I9195d5f24d938f5060fe748aac3ae58c79ec5abf

If building images, kolla currently uses its own version for their tag
by default.  This might not match the version of kolla-ansible, which is
what is used by default in kolla-ansible. This leads to the image not
being found.

This change uses kolla-ansible's version to tag locally built images.

Change-Id: I49c7afb9167d1e3f8a4470b6ebbb6ad6798d4996

Change-Id: I8d21546847a2f5fa418e07cfe79016a349af4783
Signed-off-by: ZijianGuo <guozijn@gmail.com>

Change-Id: I33484d0961f1cd17ed05dd124eba68d99b4db83f

Change-Id: If5b4ba975a65e07d2704eb6bdb9d841d6a9c3d42

trivial modify

Change-Id: I27d5b85d2c745fee5ff0643e7771b46faebd23a6

With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers

blueprint database-backup-recovery

Introduce a new option, mariadb_backup, which takes a backup of all
databases hosted in MariaDB.

Backups are performed using XtraBackup, the output of which is saved to
a dedicated Docker volume on the target host (which defaults to the
first node in the MariaDB cluster).

It supports either full (the default) or incremental backups.

Change-Id: Ied224c0d19b8734aa72092aaddd530155999dbc3

Change-Id: I7887ff3317f9eac068fee117b493db10b0e97862

Improve where openstack services are deployed
to a variable, so we will avoid many conditions
when more CI jobs are added with different configs.
Heat is enabled by default in globals.yml, lets deploy it
at least.

Change-Id: If0f009cf434c3ed5874b3171d8c75911ef8e4be1