Also slightly refactor test-openstack.sh script.

Change-Id: I7f10f073e89d2b66367bbb700201b3cd412fc433
Depends-On: https://review.opendev.org/#/c/674241
Depends-On: https://review.opendev.org/#/c/668410
Depends-On: https://review.opendev.org/#/c/668409

After the integration with placement [1], we need to configure how
zun-compute is going to work with nova-compute.

* If zun-compute and nova-compute run on the same compute node,
  we need to set 'host_shared_with_nova' as true so that Zun
  will use the resource provider (compute node) created by nova.
  In this mode, containers and VMs could claim allocations against
  the same resource provider.
* If zun-compute runs on a node without nova-compute, no extra
  configuration is needed. By default, each zun-compute will create
  a resource provider in placement to represent the compute node
  it manages.

[1] https://blueprints.launchpad.net/zun/+spec/use-placement-resource-management

Change-Id: I2d85911c4504e541d2994ce3d48e2fbb1090b813

Instead of changing Docker daemon command line let's change config
for Docker instead. In /etc/docker/daemon.json file as it should be.

Custom Docker options can be set with 'docker_custom_config' variable.

Old 'docker_custom_option' is still present but should be avoided.

Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I1215e04ec15b01c0b43bac8c0e81293f6724f278

- Test Zun on CentOS too
- Make etcd change also trigger Zun jobs (like kuryr and zun)
- Test multinode Zun deployments instead of AIO
  (more likely to break)
- In Zun scenario, stop configuring docker for legacy swarm mode
  (Zun is no swarm)
- Separate test-zun.sh testing script
- Show appcontainer to see which node it has been started on

Change-Id: I289b1009fe00aedb9b78cbd83298b14da5fd9670
Depends-On: https://review.opendev.org/676736


Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Add Masakari testing into the Gate.

Change-Id: I52df33f963e7d2ae4059887df3d24d9e6642134e
Depends-On: https://review.opendev.org/#/c/615469/
Depends-On: https://review.opendev.org/#/c/615715


Implements: blueprint ansible-masakari
Co-Authored-By: Gaëtan Trellu <gaetan.trellu@incloudus.com>

This actually replaces two ad-hoc fixes with a more unified
solution (with comment for posterity).

Change-Id: I62f57cb489c900f68a0c7aeb3e20e4715c0e2661
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Docker has no restart policy named 'never'. It has 'no'.
This has bitten us already (see [1]) and might bite us again whenever
we want to change the restart policy to 'no'.

This patch makes our docker integration honor all valid restart policies
and only valid restart policies.
All relevant docker restart policy usages are patched as well.

I added some FIXMEs around which are relevant to kolla-ansible docker
integration. They are not fixed in here to not alter behavior.

[1] https://review.opendev.org/667363



Change-Id: I1c9764fb9bbda08a71186091aced67433ad4e3d6
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Depends-On: https://review.opendev.org/669315
Change-Id: I6946290cd890f74c59ed5394e8382a8b75c0c4cd

Change-Id: Ifc898015b9b523ef4c50fc969e464f05762f2151
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

This reverts commit 8ce5ffd0.

Change-Id: I81ce7c007ff267ebbbb721bcdb7eebc0dd575bf8

Docker registry being insecure is handled by docker_registry_insecure
which is set to true by default when docker_registry is set.
The removed code had no effect because docker_registry is not changed
anyway for base (pre-upgrade) install.

This change makes config more readable and also prevents a potential
conflict with the zun profile if ever used in upgrade mode.

Change-Id: I9b5ae8c5b534fa6cce9dbaca8af191e2ca79d19f
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Adds four new CI jobs for testing centos/ubuntu binary/source deploys
with ironic enabled. These are run only when there are changes to the
ironic role.

Performs some simple testing by creating a node using the fake-hardware
hardware type and creating a server.

Change-Id: Ie669e57ce2af53257b4ca05f45193cb73f48827a
Depends-On: https://review.opendev.org/664011

Add CI jobs for testing an upgrade of a multinode system with Ceph
enabled. As for the existing upgrade job, we upgrade from the previous
release to the current release.

Change-Id: I931772ca4c63757769467a57c80dc0726a11167a
Depends-On: https://review.opendev.org/658163

Periodic jobs don't have zuul.change defined, since there is no change
being tested. This causes an early failure when referencing zuul.change
to set the image tag for built images. In periodic jobs we'll never need
to build images because there is no dependent kolla change under test.

Change-Id: I6d9d81cf17b7d0d7aaf87cd96418c904c46681f2

Make an early start on the TODOs for the Train cycle.

1. Remove the task that removes the vitrage_collector container, which
   was added in the Stein cycle to clean up this container which is no
   longer deployed.

2. Remove globals.yml configuration in CI to disable Heat for upgrade
   jobs. Heat is now enabled in the previous release (Stein).

3. Remove the deprecated variable cinder_iscsi_helper, which was renamed
   to cinder_target_helper in Stein.

Change-Id: I774bf395e0bdd4db9c20c6289a22cf059fa42e1a

This patch adds two new jobs:

* kolla-ansible-centos-source-upgrade
* kolla-ansible-ubuntu-source-upgrade

These jobs first deploy a control plane using the previous release of
Kolla Ansible, then upgrade to the current release.

Because we can't change the branch of the git repository on the Zuul
executor, we change the branch of the kolla-ansible repository on the
primary node to the branch of the previous release, in this case
stable/rocky. A new remote-template role has been added that supports
generating templates using a remote template source, to generate config
files using the previous kolla-ansible branch.

If the change being tested depends on a kolla change for the current
branch, then we build images. Rather than using the current
kolla-ansible version to tag the images, we now tag them with
change_<gerrit change ID>. This is because the version of kolla-ansible
will change from the previous release to the current one as we upgrade
the system.

Finally, it should be noted that the 'previous_release' variable in the
Zuul config needs to be updated with each release, since this sets the
release of kolla-ansible that is installed initially.

Depends-On: https://review.openstack.org/645089/
Depends-On: https://review.openstack.org/644250/
Depends-On: https://review.openstack.org/645816/
Depends-On: https://review.openstack.org/645840/
Change-Id: If301e0affcd55360fefe3b105f023ae5c47b0853

Tests Cinder with an LVM/iSCSI backend, including attaching a volume to
an instance.

Change-Id: I6963b5e81d7ac88dd1bdfb94b62d9d63028b9786

Change-Id: I33484d0961f1cd17ed05dd124eba68d99b4db83f

Change-Id: If5b4ba975a65e07d2704eb6bdb9d841d6a9c3d42

Improve where openstack services are deployed
to a variable, so we will avoid many conditions
when more CI jobs are added with different configs.
Heat is enabled by default in globals.yml, lets deploy it
at least.

Change-Id: If0f009cf434c3ed5874b3171d8c75911ef8e4be1

Introduce a job 'kolla-ansible-ubuntu-source-zun' to test kolla
with Zun enabled. To reduce CI resource, this job will be triggered
only if there are changes on the Zun's ansible roles.

Change-Id: I0ba207e1d3761da2d6992c5834d4f59e7e1d6628

Depends-On: I75e00312b36e1678b90a42cf58d24652323eff27
Change-Id: Ia716fabffca41eff816e59bbf9f4cab79ee8b72f

In order to migrate to the latest release of rabbitmq (3.7), we need to
first remove this deprecated plugin which is no longer supported (the
problems it solved are now addressed in rabbitmq itself).

This avoids a circular dependency in CI where the new images depend on
the new clustering and the new clustering depends on the new images.

Change-Id: I921459f3e40b9e0d4af9497384e49aabf0abe79b

Deploys a bifrost container using kolla-ansible bifrost-deploy.

IPA and disk images are downloaded rather than built to improve
reliability.

Currently only minimal testing of the deployment is performed, creating
and deleting an ironic node. Ideally we would perform a bare metal node
deployment.

The job is based on CentOS, as Ubuntu bifrost deployment is currently
failing with a python-MySQLdb error.

Change-Id: Ic45094594c21116b5b0d6a606f568fc7954175e3

Apply Glance database upgrade procedure:

https://docs.openstack.org/glance/latest/admin/zero-downtime-db-upgrade.html

Rolling upgrade in Glance requires that glance-registry is not deployed and run

Depends-On: https://review.openstack.org/532128
Depends-On: https://review.openstack.org/566804

Change-Id: Ia5d0cee2385facefda9d423051b74ab51d075f13
Implements: blueprint apply-service-upgrade-procedure

refer to https://review.openstack.org/#/c/564169/

Change-Id: I093b0b8fd3b34c9b35149c9e4ae945a8e0e67f53

This reverts commit 2a4ce0b8.

Change-Id: I52efd29791a230df1ce7e3acde4b7184aec34150

We have pin keyston to queens release which supports UUID token through
https://review.openstack.org/546475, let us use UUID in queens and
migrate to fernet in rocky cycle.

This reverts commit df0bf190.

Change-Id: Ifb0112315b5047461ce0bf02c754cc0beac52d9a

we have gates currently broken due using rocky keystone code.
we have no way to test our release until we branch out queens.
this change doesnt change globals default, only our gates tests.

Change-Id: I189aa8cafc73a365668d79e4b186aac21da63a6f

Use the same router will result in conflict in the CI environment.

Change-Id: Id1ba69ebbf67b4cead855adbe440343c05ea3d1e
Closes-Bug: #1744913

Another step to migrate to new CI

Change-Id: Id9bf9b4b1114f5be66f6c028a0db16d869efeceb
Depends-On: Iaacdca31b27b7033ec08499bcea7061b9d3ec279

Change-Id: Id71c1ecda9f8647a0eb290fe02a12afb3df09988

Partial-Bug: #1720601
Change-Id: Ibc20a6ae8c645ff82f3c14a6286073dffd4cfae2

First non-trivial scenario we'll be testing in gates will be cinder +
ceph. This patchset is supposed to create framework to easily add new
testing scenarios.

Change-Id: I6790eaf5cf3297af04a64e4169a39b6e2cd1890f

This reverts commit 94a882ba.

Upstream has fix this issue by Ice89ad9dd486ad5fcac534ef5f7d8aae3b6b0f97

Change-Id: Ib17f93e4030c6f7da4e32ee84fcc5f7139e57895
Closes-Bug: #1694420

This patches changes deploy_gate quite a bit so in reality all
deployments will now assume multinode (even if it's single node). After
that we will refactor it even further to enable easy addition of new
scenerios.

Change-Id: I1faada46e6a7aa026128b2f01d77eabb04759439