Since Zuul was restarted after applying [1], we can keep relevant
files listings in the base jobs per scenario without blocking
periodic job runs.
This greatly simplifies the jobs and project definitions.

[1] https://review.opendev.org/678273

Change-Id: If2e4fff6a514e6174709c2fc9f76b103f81a54ae

* Adding zuul centos-source/ubuntu-source ceph-ansible jobs
* Jobs will deploy all Ceph integrated OpenStack components, i.e.
  cinder, glance, nova
* Will utilize core openstack testing script

Depends-On: https://review.opendev.org/685032
Depends-On: https://review.opendev.org/698301

Implements: blueprint ceph-ansible
Change-Id: I233082b46785f74014177f579aeac887a25b2ae2

Debian/source to match Kolla CI test.

Change-Id: Ica45d5673db8ba1ebd4bfeb6e3d37b5ad5dd412b

Partially-implements: blueprint kolla-deployment-scenario-testing
Change-Id: I82266829dba47eac81f440a53706c2e9064b8e62

Separate upgrade logic to is_upgrade job var and rename
scenarios to match.

Rename "ACTION" to "SCENARIO" (as it is a scenario).

Separate testing of dashboard (aka Horizon) and increase
its timeout to 5 minutes (CentOS 7 slow as always).

Separate initialization of core OpenStack.

Use gate setup script from ./tests/

Remove useless tox setupenv.

Do not deploy Heat when not really necessary.

Change-Id: I4fca319ccc3de7188f8b7b44c9c71321e3899467

Change-Id: Ie42e4f96df38888b3f64515cfe84504727d3def3

Now that the stable/train branch has been cut, we can set the previous
release to Train. This is done in kolla-ansible for rolling upgrades,
and in CI configuration for upgrade tests.

Change-Id: I9d903543936e59aeeee939b32afce3e63b8c4394

Exempt files like tox.ini or unit tests from triggering ansible
runs.

Change-Id: I9c6c1d55e8df6aae782a804432374b2747fbd18e

Tests the following operations for MariaDB:

* Stop
* Recovery

Backup and restore will be added in a separate change.

Depends-On: https://review.opendev.org/693329
Change-Id: I836d91554715cce0e82c1bbebb7430c457418b2d

Debian source images are published now.

Change-Id: Id611fd2fa71eb54ca08d1f68de9505d28ad4ea40

This patch adds initial support for deploying multiple Nova cells.

Splitting a nova-cell role out from the Nova role allows a more granular
approach to deploying and configuring Nova services.

A new enable_cells flag has been added that enables the support of
multiple cells via the introduction of a super conductor in addition to
cell-specific conductors. When this flag is not set (the default), nova
is configured in the same manner as before - with a single conductor.

The nova role now deploys the global services:

* nova-api
* nova-scheduler
* nova-super-conductor (if enable_cells is true)

The nova-cell role handles services specific to a cell:

* nova-compute
* nova-compute-ironic
* nova-conductor
* nova-libvirt
* nova-novncproxy
* nova-serialproxy
* nova-spicehtml5proxy
* nova-ssh

This patch does not support using a single cell controller for managing
more than one cell. Support for sharing a cell controller will be added
in a future patch.

This patch should be backwards compatible and is tested by existing CI
jobs. A new CI job has been added that tests a multi-cell environment.

ceph-mon has been removed from the play hosts list as it is not
necessary - delegate_to does not require the host to be in the play.

Documentation will be added in a separate patch.

Partially Implements: blueprint support-nova-cells
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Change-Id: I810aad7d49db3f5a7fd9a2f0f746fd912fe03917

Upgrade jobs like to timeout in the 2-hour window when they must
build their images.
This increase is already applied in ceph jobs.

Change-Id: Ic1118760d9192cc15e1ebf37fb8adf3440f18a78

Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689



Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Attempts affect pre failures.
This means we can increase stability of jobs by rejecting nodes
that fail pre without failing runs at the same time (unless we
are really unlucky and hit b0rken nodes 5 times in a row).

Change-Id: I17b7f878c742fa8db66f738526855a02ab9f1905

Change-Id: I0628b16e3ebdb3fa8196acdc1bd9c63e75bcfb09

VXLAN is necessary to run HA in CI (due to floating VIP
address handled by keepalived).
It also turned out to be required to have private
IPv6 address assignments.
This patch is based on linux bridge rather than OVS
to avoid problems with OVS deployed in containers.

This patch enables haproxy in multinode jobs.

Includes saving of linux networking details.

Makes DASHBOARD_URL agree with OS_AUTH_URL - properly uses the
pre-upgrade value for testing.

Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Depends-on: https://review.opendev.org/683068
Depends-on: https://review.opendev.org/682957
Change-Id: I66888712da80c3d6f84ee4949762961664d3adea

- Test Zun on CentOS too
- Make etcd change also trigger Zun jobs (like kuryr and zun)
- Test multinode Zun deployments instead of AIO
  (more likely to break)
- In Zun scenario, stop configuring docker for legacy swarm mode
  (Zun is no swarm)
- Separate test-zun.sh testing script
- Show appcontainer to see which node it has been started on

Change-Id: I289b1009fe00aedb9b78cbd83298b14da5fd9670
Depends-On: https://review.opendev.org/676736


Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Add Masakari testing into the Gate.

Change-Id: I52df33f963e7d2ae4059887df3d24d9e6642134e
Depends-On: https://review.opendev.org/#/c/615469/
Depends-On: https://review.opendev.org/#/c/615715


Implements: blueprint ansible-masakari
Co-Authored-By: Gaëtan Trellu <gaetan.trellu@incloudus.com>

Additionally simplifies regexps.

Change-Id: I3b027aeb330630a2ea90fc3e2c4b70c7fb38f058
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Change-Id: I9773a7c4f7a5d31a83c10562057ce772439b9693
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

This is to ensure that any Depends-On does not cause Zuul not to pick up
the change for gating due to no notifications between queues.
Previously W+1-ing a change which depended on non-merged change from
the other project caused it to remain in the same state.

Change-Id: Ib2d88471ac5730c00b5a9721066d1fb3f2998c9c
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Kolla has it already and kolla-ansible should.
Patch to backport as far as pike.
Affects only stable branches.

Change-Id: Iecc46b364ad9fc69fe67dd09ee1b4e3c5511f01c
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

openstack/requirements is not used.
openstack/kolla is used conditionally when Zuul itself adds it
due to dependency.
No need to always clone the two repositories.

Change-Id: Iabfc25e2b4227ae227f06ab196ca544b02e7808f
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

The file is not used in Zuul jobs. It is templated independently.

Change-Id: Ia39898cdb165e30dab74de727048be2ebb2290c8
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Adds four new CI jobs for testing centos/ubuntu binary/source deploys
with ironic enabled. These are run only when there are changes to the
ironic role.

Performs some simple testing by creating a node using the fake-hardware
hardware type and creating a server.

Change-Id: Ie669e57ce2af53257b4ca05f45193cb73f48827a
Depends-On: https://review.opendev.org/664011

We have agreed to remove support for Oracle Linux, starting by
disabling
the CI jobs.

http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006896.html

Change-Id: I79f6ec98383b1f59b8317a3720f6f9923d6bb21d

Add CI jobs for testing an upgrade of a multinode system with Ceph
enabled. As for the existing upgrade job, we upgrade from the previous
release to the current release.

Change-Id: I931772ca4c63757769467a57c80dc0726a11167a
Depends-On: https://review.opendev.org/658163

It shouldn't trigger kolla-ansible-base job if changing deploy guide.

Change-Id: I8cb5d3ae7825690279f78ec847b940110bd60216

Following the process set down in the TC resolution on updating Python
versions:

https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html

Change-Id: If36099eb2ab0c34c18544a0996d2bbaea255b539

This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.

We run these jobs in kolla, we should also run them in kolla-ansible.

Change-Id: Ica530a57d0e6dcfd5c50fd6f8796cd65510a4409

Now that the stable/stein branch has been cut, we can set the previous
release to Stein. This is done in kolla-ansible for rolling upgrades,
and in CI configuration for upgrade tests.

Change-Id: I87269738db9521fc22a6ce3aee67d9ab00d47e2a

This patch adds two new jobs:

* kolla-ansible-centos-source-upgrade
* kolla-ansible-ubuntu-source-upgrade

These jobs first deploy a control plane using the previous release of
Kolla Ansible, then upgrade to the current release.

Because we can't change the branch of the git repository on the Zuul
executor, we change the branch of the kolla-ansible repository on the
primary node to the branch of the previous release, in this case
stable/rocky. A new remote-template role has been added that supports
generating templates using a remote template source, to generate config
files using the previous kolla-ansible branch.

If the change being tested depends on a kolla change for the current
branch, then we build images. Rather than using the current
kolla-ansible version to tag the images, we now tag them with
change_<gerrit change ID>. This is because the version of kolla-ansible
will change from the previous release to the current one as we upgrade
the system.

Finally, it should be noted that the 'previous_release' variable in the
Zuul config needs to be updated with each release, since this sets the
release of kolla-ansible that is installed initially.

Depends-On: https://review.openstack.org/645089/
Depends-On: https://review.openstack.org/644250/
Depends-On: https://review.openstack.org/645816/
Depends-On: https://review.openstack.org/645840/
Change-Id: If301e0affcd55360fefe3b105f023ae5c47b0853

Currently ceph jobs are often getting TIMED_OUT, increasing limit.

Change-Id: I3c6684984930d55a56da846bd8c3f19df2754b06

Migrate to the latest Ubuntu LTS release 18.04 aka Bionic.  See [0] for
the big picture.

Also test running tox jobs on Bionic.

[0] https://etherpad.openstack.org/p/devstack-bionic

Change-Id: I96e7b8d17bc1e92716c04fdcf362c2adb08a2212

Change-Id: I1f17c504e265e127409b108d2cc53ef6e8c6b887

Tests Cinder with an LVM/iSCSI backend, including attaching a volume to
an instance.

Change-Id: I6963b5e81d7ac88dd1bdfb94b62d9d63028b9786

Python 3.5 was the target runtime for the Rocky release. The current
target py3 runtime for Stein is Python 3.6, so there is no reason to
keep testing against the older version.

https://governance.openstack.org/tc/reference/runtimes/stein.html#python-runtime-for-stein

Change-Id: I0db6c9cbf2af9ed9c701755a26f721aeac97d4cd

This is a mechanically generated patch to add a unit test job running
under Python 3.7.

See ML discussion here [1] for context.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135626.html

Change-Id: Ice5e11c244b48812e786ef188b4110e6d57b5445
Story: #2004073
Task: #27423

Zun depends on Kuryr so let's trigger the Zun job on kuryr's change.

Change-Id: If95a6bfb5f305c2f69808a781ff9d8513bf2006c