Debian defaults to Python2 which is not complete in aarch64 images.
This patch changes CI to always use Python3.

We need to install several Python modules to have working ussuri jobs.

"Failed to import the required Python library (setuptools) on primary's Python /usr/bin/python3."

And then several Python2 ones for train->ussuri upgrade jobs:

"Unable to find any of pip2 to use. pip needs to be installed."

Change-Id: Ia0d3ff15d97d1cabbb0b8e7f32e8712ca3f94732

Removes and/or replaces all mentions of py27.

Cleans up obsolete requirements and their lower-constraints.

Separates test-requirements.

Makes lower-constraints pass outside of CI (MarkupSafe).

Adds FIXMEs about some hacky Mocks that may misbehave.

Change-Id: Ifc090bf3c1db17d8542ee591c91e8225a597bfe2

Bashate warned on init-swift.sh because of E043 -- arithmetic
compound has inconsistent return semantics: (( next_port++ ))
New Zuul reported that on every change as a warning.
This is fixed here.

This change makes Bashate always produce errors so that we do not
introduce such warnings again.

Change-Id: I40166b377ec2580e17901375b636183bca492d3a

Change-Id: I150c8952aad7b6fdb1260b343c26a96fb2360bce

Drops support for creating Python 2 virtualenvs in bootstrap-servers,
and looking for a python2 interpreter in the kolla-ansible script.

Also forces the use of Python 3 as the remote interpreter in CI on
Debian and Ubuntu hosts, since they typically symlink the unversioned
interpreter to python2.7.

Change-Id: Id0e977de381e7faafed738674a140ba36184727e
Partially-Implements: blueprint drop-py2-support

Followup of last reviews from [1].

[1]: https://review.opendev.org/#/c/696841

Change-Id: I7085093b20e8848e09dc521ae9fbf120e909470d

We publish those images for a while.

Change-Id: Ifc157b43e87e4f77a1c70d98343bd0ef9cc0de79

AArch64 does not have a way to get cpu features from libvirt.

Change-Id: Ieed404e17e8a9829f38c03f7ee7fdb3caa3919e8

AArch64 needs a bit more time to boot testing instance. So give it more
time by doubling amount of connect attempts.

Change-Id: I87ca65691dfbac84349e8af24d2f36f1db1c8be1

This patch introduces an optional backend encryption for Keystone
service. When used in conjunction with enabling TLS for service API
endpoints, network communcation will be encrypted end to end, from
client through HAProxy to the Keystone service.

Change-Id: I6351147ddaff8b2ae629179a9bc3bae2ebac9519
Partially-Implements: blueprint add-ssl-internal-network

Implement OVN Ansible role.

Implements: blueprint ovn-controller-neutron-ansible

Depends-On: https://review.opendev.org/713422
Change-Id: Icd425dea85d58db49c838839d8f0b864b4a89a78

CentOS 8 support is now fairly complete - time to drop CentOS 7.

Partially-Implements: blueprint centos-rhel-8

Change-Id: I940b1d3eceb98e16fa366c243672f588b1412d70

The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found by updated hacking version.

Remove hacking and friends from lower-constraints, they are not needed
during installation.

Change-Id: I7ef5ac8a89e94f5da97780198619b6facc86ecfe

We don't need tox nor tell our users to use it when doing kolla
builds.
Tox is going away from base infra images.
It's already gone from aarch64 (arm64) ones.

Change-Id: I2eb5203ad93c011b8806f4b6fb56db081c14a2cb

Now that py2 is gone, oslotest dropped dependency on mock and will
soon affect Ussuri CI [1], let's use unittest.mock built in py3.

This also fixes py38 jobs and proactively prevents py36 and py37
failing due to [1]. This is because we never included mock in
test-requirements (but in lower-constraints where it does not
really belong at all) and instead relied on oslotest to bring
it in.

[1] https://review.opendev.org/716322

Change-Id: I30e82e2d87418272a71c7ee089a8acdaf8872158

In stable branches we are getting hit by more py2-incompats.
Let's pin u-c in all CI pip invocations.

Change-Id: Ie2bcc7c115cd2aaf4639d90803216011b346daf3

Sometimes ping & ssh to the instance are failing - outputting instance console
log can help in the case when there are metadata access issues (or other
issues).

Change-Id: I8437300d621448782e964d877b2614ca606f5849

Since fluentd is disabled in MariaDB jobs - haproxy logs are not getting
populated.

Change-Id: I56b3fc1be6940d97905cdb2c4452b846f106c071
Depends-on: https://review.opendev.org/713704

ceph-ansible by default uses "latest" tag for ceph Docker Hub images,
but recently latest tag has been upgraded to be Octopus release,
not Nautilus.

Change-Id: I5247c10079ab91cce130cd5ba403f25ccaf7c1fb

tox will be removed from the base image. Install it before that happens.

This change is made in a simple way that can be easily backported.

Depends-On: https://review.opendev.org/713386

Change-Id: I4181654c88554c81940f0d079cf1d64326cdec79

We are getting this randomly on neutron-server shutdown
for upgrade.
As it does not affect real operations (and if it did,
we would definitely see it now), let's ignore it.

Change-Id: Ibe561517d44a1108e8223442a71fab36b69c2258
Related-bug: #1863579

Test upgrade from CentOS 8 train to CentOS 8 master.

Change-Id: Ibff2c7f8844dec4758945cbc7aa8df80d70a3dfd
Partially-Implements: blueprint centos-rhel-8

Following I21dd51c82534704f31ca8d3f72cb2587ee216cd9, the test inventory
was synced with the multinode inventory. This removed some temporary
ceph groups used by the ceph-ansible-upgrade jobs, and broke them. This
change adds the groups back.

Change-Id: I37379258447ffde6b083f4e8d9a1644bc17cd165

TrivialFix

Change-Id: I0e6924b5dc86e90c10037d96ab871bd981365cc9

Also adds gawk for timestamping.

This helps to correlate init events with failures elsewhere.

Change-Id: I22fdb683ecf9870b2d66fedd6b40b7004317130a

Inspired by pending work of mnasiadka [1] and mgoddard [2].

[1] https://review.opendev.org/708250
[2] https://review.opendev.org/707604

Depends-on: https://review.opendev.org/709362
Depends-on: https://review.opendev.org/709671
Depends-on: https://review.opendev.org/709830
Change-Id: Ifd7bde60881a4b644fc3f18dd9fcb7f8b02d3a00

This is to allow CI testing of network connectivity.

Note only "primary" node gets an address on it.
Hence, "primary" becomes our fake external router.

Depends-on: https://review.opendev.org/709361

Change-Id: I05592888796107d6de95b940c42b2bff73ac0669

Fix the upgrade TLS scenario in zuul to generate self signed
certificates and to configure TLS to be enabled in the open stack
deployment.

Change-Id: Icacc28eed6ad5b81fc3954db80486d9d7f24c082
Partially-Implements: blueprint custom-cacerts

Clients are starting to release versions that don't support Python 2.
The ironic scenario is currently failing on stable branches for this
reason.

Use upper constraints to avoid installing these new versions on stable
branches.

Change-Id: I4f91b53cbf2297d70da4b54d6c402c1427aacdd9

Change-Id: I21dd51c82534704f31ca8d3f72cb2587ee216cd9

This was never necessary because C7 IPv6 CI was not enabled at
the time and later we fixed IPv6 in C7 to avoid this issue.
In Ussuri C7 is going away so even more reasons to drop it. :-)

Change-Id: I4066c9cd86ff892d78f6713589f9afffc611dcc1

This allows for some real testing, especially in a multinode
environment.

Change-Id: Ic96819fefe460e14c8460e52a78d9b0d034edc80

In some resource-constrained environments, particularly during service
bootstrap Galera cluster nodes can experience timeouts in inter-node
communication.

This change sets the gmcast.peer_timeout based on the galera cluster
documentation:
https://galeracluster.com/library/documentation/galera-parameters.html

We are observing peer timeout issues on some CI runs - therefore raising
it to PT15S as in similar Ubuntu charms jobs.

Change-Id: Id036e41b62a88bab486c35a5f1fde5cfc2fa4803

Reverts temporary.

This reverts commit 241e3474.

Change-Id: I4af6b9c57e594fce407b9524f04cfa83479e16f0

Change-Id: I8b4e649e24871b1191dba39d766b6c133c7e8f24

global_physnet_mtu needs to be set in neutron.conf, because linuxbridge-agent
discovers underlying vxlan0 interface mtu and returns an error when creating
vxlan port

CentOS8 job will not be added, because CentOS 8 iptables-ebtables package
is missing broute (--among-src) tables support required for linuxbridge agent,
see [1].

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1720637

Change-Id: I6b12f7ba95401d3342359c57ceeee8bec8aefe49

This makes it cleaner, allows reuse and outsourcing to zuul jobs
and enables us to create multiple of these overlay networks for
testing of more advanced scenarios.

Change-Id: Id557c81f68a7f34556854e7d6efc6eddfd2e7216

Change-Id: If354fbdeb2fd8d026faa0210b84da0a688c3ae0a

Since move to Swift ARA database does not render server-side,
let's make it render HTML locally as suggested.

Change-Id: I1190526c02d1f312d5284544d6e5be433dd839fa

Since virtualenv 20.0 (amongst other changes) six version >1.12.0 is required.
This change adds upgrade of virtualenv and six in pre - to be reverted once
infra CentOS images are sorted out.

Change-Id: I0ca0347bb6ebc5d8f5d22f708211e01221165262