Change-Id: Idf94a64b70d4684a75ec8e8ccc38d64fa6274431

Include information on how to test bare metal instance creation.

Change-Id: Ie40f258a31d4b72cd9867d321a3ff712522daec5

Currently we configure daemon.json to use either devicemapper or overlay
as the storage driver. We should simply pass through whatever is
configured for docker_storage_driver.

Change-Id: Id4423030b5483fe4ecd4f324bc25800e5444fd63
Story: 2005649
Task: 30932

In [files] section of setup.cfg there is not allowed comment, which
breaks installation with setup.py.

Change-Id: I38fce0a6c155e4b7d2ff7a2535f18e8adc2566e9
Story: 2005546
Task: 30692

Change-Id: Ibf099c6fe4ad516f29d1f4f500f381f4f1baa212

The project infrastructure is now hosted at opendev.org.

Kayobe projects have been moved from the openstack/ namespace to x/.

Change-Id: I16b523285eb34c1cd0eaf920aa4512d0ae1b4f05

In a deployment that has both Ceph or Swift deployed it can be useful to seperate the network traffic.
This change adds support for dedicated storage networks for both Ceph and Swift. By default, the storage hosts are
attached to the following networks:

* Overcloud admin network
* Internal network
* Storage network
* Storage management network

This adds four additional networks, which can be used to seperate the storage network traffic as follows:

* Ceph storage network (ceph_storage_net_name) is used to carry Ceph storage
  data traffic. Defaults to the storage network (storage_net_name).
* Ceph storage management network (ceph_storage_mgmt_net_name) is used to carry
  storage management traffic. Defaults to the storage management network
  (storage_mgmt_net_name).
* Swift storage network (swift_storage_net_name) is used to carry Swift storage data
  traffic. Defaults to the storage network (storage_net_name).
* Swift storage replication network (swift_storage_replication_net_name) is used to
  carry storage management traffic. Defaults to the storage management network
  (storage_mgmt_net_name).

This change also includes several improvements to Swift device management and ring generation.

The device management and ring generation are now separate, with device management occurring during
'kayobe overcloud host configure', and ring generation during a new command, 'kayobe overcloud swift rings generate'.

For the device management, we now use standard Ansible modules rather than commands for device preparation.
File system labels can be configured for each device individually.

For ring generation, all commands are run on a single host, by default a host in the Swift storage group.
A python script runs in one of the kolla Swift containers, which consumes an autogenerated YAML config file that defines
the layout of the rings.

Change-Id: Iedc7535532d706f02d710de69b422abf2f6fe54c

This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.

Adds two new variables, 'openstack_release' and 'openstack_branch',
in ${KAYOBE_CONFIG_PATH}/openstack.yml for setting the current OpenStack
release and branch in a single place.

Also uses the master branch for the molecule tox job.

Change-Id: I851a1e6eddc6738005c1813599445b38c0ae0d57

Use master branch of:

* Kolla
* Kolla ansible
* Bifrost
* IPA
* OpenStack services
* Requirements

Also, use the infra docker registry mirror at port 8082 which does not
have any path component in its URL, and so works with Docker CE.

Depends-On: https://review.openstack.org/#/c/651254/
Change-Id: Icdb62aa0083d9272a08421138719cf5f6b52ec5a
Story: 2001864
Task: 14316

This is supported in kolla-ansible via the ironic_inspector_pxe_filter
variable, which can be added to globals.yml. The default value for that
variable changed in the Stein release from 'iptables' to 'dnsmasq',
since the iptables filter does not work with Docker CE [1].

This change removes the inspector_manage_firewall variable.

This change also adds an iptables rule in CI tests to allow DHCP packets
to be forwarded, to ensure bare metal servers can be deployed.

[1] https://bugs.launchpad.net/kolla-ansible/+bug/1823044

Depends-On: https://review.openstack.org/649673
Change-Id: Idac6777b4d97fbd17698fc2086ceb068d7b2e326
Related-Bug: #1823044

If you enable monasca via kolla_enable_monasca, grafana is automatically
enabled too. Monasca provides its own forked version of grafana, so we
don't need the standard one.

This change brings the logic into line with kolla-ansible.

Change-Id: I5be1f269547d9c547211b675814829a665685485
Story: 2005399
Task: 30393

Change-Id: I0a92c60000dbfaa1a6a841dcad66f118226d9ace

Allow users to use the custom config mechanism for Keepalived
provided by Kolla Ansible.

Change-Id: I052bd8283944197cd2b13747e7a7c32fbe06c045
Story: 2005211
Task: 29989

Currently nested virtualisation under KVM does not seem to be working in
CI. This breaks the 'bare metal' deployment testing using Tenks, which
lead us to disable it in 749ef824.

This commit forces Tenks to use QEMU for its VMs, allowing us to revert
commit 749ef824..

Change-Id: Id382c218f3b37979341f0d96718a6011a1d9da37
Story: 2005316
Task: 30223

When first added, the bare metal deployment tests using Tenks were
fairly reliable (although not 100%). Recently though, jobs seem to be
failing at this step quite frequently, making it difficult to merge
patches. Looking at the historical zuul jobs, the failure rate per job
seems to be about 1/3:
http://zuul.openstack.org/builds?job_name=kayobe-overcloud-centos. With
two overcloud jobs and check and gate required to pass, the merge
success rate is around 20%.

Let's skip this testing for now, until we can make it more reliable.

Change-Id: Id0a05f8c5d227f8cd9424c7a5b138eddee93f827
Story: 2005316
Task: 30224

When generating or updating the passwords.yml file for kolla-ansible,
kayobe writes out various stages of the process to temporary files
in /tmp, in plain text. One of these files can be left in place if
there are no changes to apply to the file.

This change ensures that we always remove temporary files containing
passwords. We also switch from shutil.copy2 to shutil.copyfile, to
keep the permissions of the destination rather than applying those of
the source, which are typically more open (644 vs 600).

Depends-On: https://review.openstack.org/647858
Change-Id: Icb290fd22dc01567a4297a42f5e4d765e3b57d37
Story: 2005299
Task: 30187

Change-Id: I647d93bf40d42a1b86b06170c8b4cc96972051c4
Story: 2005017
Task: 29495

Change-Id: I0b364a77231e1b88dd9471e6f3893626c125d510

There is an issue with the default YAML loader, which allows arbitrary
code execution, as documented here:
https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation.

This can be avoided by using yaml.safe_load. We don't require Python
object serialisation, so safe_load is sufficient.

Change-Id: I09190766066ab56d04b1317a4022782160d60528
Story: 2005253
Task: 30050

Also change to use the py3 tox environment rather than specifying a
version explicitly.

Change-Id: I05d5d6f5718b39dc95b40efe44e9ebb3fb5b1311
Story: 2004959
Task: 30035

In python 3.6, os.path.join does not accept Mock objects, giving the
following error:

TypeError: expected str, bytes or os.PathLike object, not MagicMock

Since the switch to use ubuntu bionic for the base image for jobs,
python 3.6 is the default for python 3.

Change-Id: I79bec585361a54b8303c130a363c62d9a4b7faae
Story: 2004959
Task: 30035

Change-Id: I4c6ddc8cfc3f3cef7e63f6363d9db55f0ea1faff

Change-Id: I8a9d1db4cba381e44718f663347efa8eba854a63

Also updates the release.sh script to support tagging release candidates
and development milestones. Adds zuul configuration to trigger a
Readthedocs webhook for the release notes.

Change-Id: Ia784b34fb0b740998de72599d4921f9303e41cac