Change-Id: I02eca49fed19686551990fc125a7025d54c5d6e3

As a first step towards supporting multiple overcloud disk images, this
change introduces a new command to build a disk image directly with DIB:
`kayobe overcloud host image build`.

It also disables building a root disk image during Bifrost bootstrap if
overcloud_dib_build_host_images is set to true.

Change-Id: I93d242889e225b4e60254f6b9cc5eeb457294ac8
Story: 2002098
Task: 41693

Change-Id: I914c22c6338421bcd0f37277253fe42572300612

This change allows you to define additional VMs to deploy
on the seed-hypervisor.

Co-authored-by: Piotr Parczewski <piotr@stackhpc.com>
Co-authored-by: Will Szumski <will@stackhpc.com>
Co-authored-by: Mark Goddard <mark@stackhpc.com>
Story: 2008741
Task: 42095
Change-Id: I8055fc5eb0a9edadcb35767303c659922f2d07ca

Ansible facts can have a large impact on the performance of the Ansible
control host. This patch introduces some control over which facts are
gathered (kayobe_ansible_setup_gather_subset) and which facts are stored
(kayobe_ansible_setup_filter). By default we do not change the default
values of these arguments to the setup module. The flexibility of these
arguments is limited, but they do provide enough for a large performance
improvement in a typical moderate to large OpenStack cloud.

In particular, the large complex dict fact for each interface has a
large effect, and on an OpenStack controller or hypervisor there may be
many virtual interfaces. We can use the kayobe_ansible_setup_filter
variable to help:

    kayobe_ansible_setup_filter: 'ansible_[!qt]*'

This causes Ansible to collect but not store facts matching that
pattern, which includes the virtual interface facts. Currently we are
not referencing other facts matching the pattern within Kayobe.
Note that including the 'ansible_' prefix causes meta facts module_setup
and gather_subset to be filtered, but this seems to be the only way to
get a good match on the interface facts. To work around this, we use
ansible_facts rather than module_setup to detect whether facts exist in
the cache.

The exact improvement will vary, but has been reported to be as large as
18x on systems with many virtual interfaces.

This change also introduces a new command to gather facts for Kayobe &
Kolla Ansible on demand, 'kayobe overcloud facts gather'. This can be
used to populate a fact cache.

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/794610
Story: 2007993
Task: 42586

Change-Id: I5ce3c734433e1682ee942867505468c57440e689

Runs 'kolla-ansible prechecks' without deploying services.

Change-Id: I3806b707593fc6ac9d7125e1b12d716ad40a2808
Story: 2008912
Task: 42503

Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb

Change-Id: I831f08154dad5a80e8f34aeb9ae161d2de5ffa57

Change-Id: I2f33bda5f2f84bbbb0f40240d2575a0b69ddceea
Story: 2002009
Task: 40038

Change-Id: Ib869ae5146948abcb11eb20691ccc2311db24bb1
Story: 2008458
Task: 41485

Change-Id: I7f757098211745671ec891fdd4e0b52dc1fe7602

Sometimes there is a need to develop site specific playbooks. Currently,
it is necessary to manually invoke these at the right point during the
deployment. Adding the ability to automatically run these custom
playbooks will reduce the chance of running these playbooks at the wrong
point or forgetting to run them at all.

Change-Id: I1ae0f1f94665925326c8b1869dd75038f6f1b87d
Story: 2001663
Task: 12606

Removes and/or replaces all mentions of py27.

Cleans up obsolete requirements and their lower-constraints.

Update cliff minimum to 3.1.0 in requirements.txt, which has a fix for
story 2005891.

Change-Id: I52cffa2f1aee944f79c4618ea20b779755792f2a

Implemented via 'kolla-ansible stop'.

Change-Id: Iaf8db47e70f023b446c17aa61fc8bb89cf7c2b28
Story: 2007467
Task: 39155

OpenStack is dropping the py2.7 support in ussuri cycle.

kayobe is ready with python 3 and ok to drop the
python 2.7 support.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: Idbbdd96ff23846d29833c5997d1e32e65b5169f2

Kolla ansible train adds a new command, kolla-ansible deploy-containers,
that will only deploy containers, and skips registration, bootstrapping
and configuration.

This change adds a new 'kayobe overcloud service deploy containers'
command to make use of the new kolla-ansible command.

Change-Id: I999dfe4b1d082bc88148f54b236644abcbd96a5a
Story: 2006951
Task: 37633

Change-Id: I92f1e5b2482da8aa16133844e3a78f6bbc225291

Uses the support [1] for database backups added to Kolla Ansible in the
Stein release.

Adds the following new commands:

kayobe overcloud database backup [--incremental]
kayobe overcloud database recover [--force-recovery-host <host>]

The recovery command is for recovering a clustered database that has
lost quorum.

[1] https://docs.openstack.org/kolla-ansible/latest/admin/mariadb-backup-and-restore.html

Change-Id: Ie16354cd01ea7dd83cd3d4058dd8451b8387600b
Story: 2005015
Task: 29493

In [files] section of setup.cfg there is not allowed comment, which
breaks installation with setup.py.

Change-Id: I38fce0a6c155e4b7d2ff7a2535f18e8adc2566e9
Story: 2005546
Task: 30692

In a deployment that has both Ceph or Swift deployed it can be useful to seperate the network traffic.
This change adds support for dedicated storage networks for both Ceph and Swift. By default, the storage hosts are
attached to the following networks:

* Overcloud admin network
* Internal network
* Storage network
* Storage management network

This adds four additional networks, which can be used to seperate the storage network traffic as follows:

* Ceph storage network (ceph_storage_net_name) is used to carry Ceph storage
  data traffic. Defaults to the storage network (storage_net_name).
* Ceph storage management network (ceph_storage_mgmt_net_name) is used to carry
  storage management traffic. Defaults to the storage management network
  (storage_mgmt_net_name).
* Swift storage network (swift_storage_net_name) is used to carry Swift storage data
  traffic. Defaults to the storage network (storage_net_name).
* Swift storage replication network (swift_storage_replication_net_name) is used to
  carry storage management traffic. Defaults to the storage management network
  (storage_mgmt_net_name).

This change also includes several improvements to Swift device management and ring generation.

The device management and ring generation are now separate, with device management occurring during
'kayobe overcloud host configure', and ring generation during a new command, 'kayobe overcloud swift rings generate'.

For the device management, we now use standard Ansible modules rather than commands for device preparation.
File system labels can be configured for each device individually.

For ring generation, all commands are run on a single host, by default a host in the Swift storage group.
A python script runs in one of the kolla Swift containers, which consumes an autogenerated YAML config file that defines
the layout of the rings.

Change-Id: Iedc7535532d706f02d710de69b422abf2f6fe54c

Change-Id: I019fc3f5c59c383215febb958f9c4cf8c8b8e4a6
Story: 2003312
Task: 24270

This adds the ansible playbooks required by kayobe to the manifest by
using the data_files option in setuptools. When using pip to install
kayobe into a virtualenv, these files will be placed in
<venv>/kayobe/share/.

In an editable install, e.g using `pip install -e .`, data_files are not
installed into the virtualenv. Instead, we must follow the egg-link file
to find out the actual location.

Story: 2004252
Task: 27787
Change-Id: Ibef040eceb547476007f83c0d5dcdb2bc6986d1e

Adds a new command to upgrade containerised services on the seed:

kayobe seed service upgrade

This is largely the same as the deploy command, since there is no
kolla-ansible bifrost-upgrade command yet.

A limitation in Bifrost requires us to remove IPA images before
upgrading the bifrost container.

Change-Id: Ibbe01e972487e69dbb753a67efcf13e3ae0800de
Story: 2004308
Task: 27872

Mailinglists have been updated. Openstack-discuss replaces openstack-dev.

Change-Id: I8eb686e42cb88cd5f47f0e1af20e746baa0e50c0

This allows you to access the serial console from within
Horizon.

Change-Id: Id40e72047174fc0c0c565871f24b775b30e83825
Story: 2004192
Task: 27682

Without this change twine check dist/* would fail as this was
producing a double new line. The rest of the PKG-INFO would then
be interpreted as the message body and become the long
description. See https://github.com/pypa/twine/issues/422 for
more details.

This fixes the test-release-openstack-python3 zuul job.

TrivialFix

Change-Id: If1330c4824e949aa427475dfcc26218fb398a66e

This will update the the deploy_ramdisk and
deploy kernel properties of  'Driver info'
field on Ironic nodes if the locally built
or externally referenced images are updated.

Change-Id: Id3997db452dde6e6e242a9b1091cb219c53ebda1

Just performs a yum update.

Change-Id: I4cdc23a3e491c5a2f92e8beeb7eaebf2df818df3
Story: 2003305
Task: 24257

Change-Id: I2140926828366b86a1cb423275330ae17c55fdcd

Story: 2002176
Task: 20042
Change-Id: I2174a4719aaff63fff24e35ce62eab57b369b457

For compliance with the Project Testing Interface [1] as described in
[2].

[1]
https://governance.openstack.org/tc/reference/project-testing-interface.html
[2]
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: I03d36b2545f69c4ca3ca1ced73d179e81b7b8811

Kayobe is to become an OpenStack-related project, and as such will use
Gerrit for code reviews and Storyboard for issue and feature tracking.
The README and CONTRIBUTING documents have been updated to reflect that,
and a standard OpenStack HACKING document has been added.

Change-Id: Id03bcb5183bfbf6e84bb4d9c22f85d9418b6b4fc
Story: #2001637
Task: #6644

Python Build Reasonableness (PBR) [1] is used by most OpenStack projects
for building python packages, so we should fall into line here.

[1] https://pypi.python.org/pypi/pbr

Change-Id: Ic74a62eeb65e7158ce75a082e68652c2e50a2a91
Story: #2001637
Task: #6648