If any nova compute service fails to register itself, Kolla Ansible will
fail the host that queries the Nova API. This is the first compute host
in the inventory, and fails in the task:

    Waiting for nova-compute services to register themselves

Other hosts continue, often leading to further errors later on. Clearly
this is not idea.

This change modifies the behaviour to query the compute service list
until all expected hosts are present, but does not fail the querying
host if they are not. A new task is added that executes for all hosts,
and fails only those hosts that have not registered successfully.

Alternatively, to fail all hosts in a cell when any compute service
fails to register, set nova_compute_registration_fatal to true.

Change-Id: I12c1928cf1f1fb9e28f1741e7fe4968004ea1816
Closes-Bug: #1940119

Config plays do not need to check containers. This avoids skipping
tasks during the genconfig action.

Ironic and Glance rolling upgrades are handled specially.

Swift and Bifrost do not use the handlers at all.

Partially-Implements: blueprint performance-improvements
Change-Id: I140bf71d62e8f0932c96270d1f08940a5ba4542a

Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. For unconditionally included tasks, switching to
import_tasks provides a clear benefit.

Benchmarking of include vs. import is available at [1].

This change switches from include_tasks to import_tasks where there is
no condition applied to the include.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md#task-include-and-import

Partially-Implements: blueprint performance-improvements

Change-Id: Ia45af4a198e422773d9f009c7f7b2e32ce9e3b97

Change-Id: Iea3f4f3d2e5c6040c1e0bc7bfae8719cc7d8ac55

Deploy a small cloud. Add one host to the compute group in the
inventory, and scale out:

$ kolla-ansible deploy --limit <new compute host>

The command succeeds, but creating an instance fails with the following:

    Host 'compute0' is not mapped to any cell

This happens because we only discover computes on the first host in the
cell's nova conductor group. If that host is not in the specified limit,
the discovery will not happen.

This change fixes the issue by running compute discovery when any ironic
or virtualised compute hosts are in the play batch, and delegating it to
a conductor.

Change-Id: Ie984806240d147add825ffa8446ae6ff55ca4814
Closes-Bug: #1869371

Currently there are a few services that perform host configuration
tasks. This is done in config.yml. This means that these changes are
performed during 'kolla-ansible genconfig', when we might expect not to
be making any changes to the remote system.

This change separates out these host configuration tasks into a
config-host.yml file, which is included directly from deploy.yml.

One change in behaviour is that this prevents these tasks from running
during an upgrade or genconfig. This is probably what we want, but we
should be careful when any of these host configuration tasks are
changed, to ensure they are applied during an upgrade if necessary.

Change-Id: I001defc75d1f1e6caa9b1e11246abc6ce17c775b
Closes-Bug: #1860161

This patch adds initial support for deploying multiple Nova cells.

Splitting a nova-cell role out from the Nova role allows a more granular
approach to deploying and configuring Nova services.

A new enable_cells flag has been added that enables the support of
multiple cells via the introduction of a super conductor in addition to
cell-specific conductors. When this flag is not set (the default), nova
is configured in the same manner as before - with a single conductor.

The nova role now deploys the global services:

* nova-api
* nova-scheduler
* nova-super-conductor (if enable_cells is true)

The nova-cell role handles services specific to a cell:

* nova-compute
* nova-compute-ironic
* nova-conductor
* nova-libvirt
* nova-novncproxy
* nova-serialproxy
* nova-spicehtml5proxy
* nova-ssh

This patch does not support using a single cell controller for managing
more than one cell. Support for sharing a cell controller will be added
in a future patch.

This patch should be backwards compatible and is tested by existing CI
jobs. A new CI job has been added that tests a multi-cell environment.

ceph-mon has been removed from the play hosts list as it is not
necessary - delegate_to does not require the host to be in the play.

Documentation will be added in a separate patch.

Partially Implements: blueprint support-nova-cells
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Change-Id: I810aad7d49db3f5a7fd9a2f0f746fd912fe03917

The idea is to factor out a role for deploying Nova related services
to cells. Since all deployments use cells, this role can be used
in both regular deployments which have just cell0 and cell1,
and deployments with many cells.

Partially Implements: blueprint support-nova-cells
Change-Id: Ib1f36ec0a773c384f2c1eac1843782a3e766045a

When nova-api group have no hosts, we don't have to run create_cells
and discover_computes. Add conditional blocks to prevent to run them.

Change-Id: Ia1ba058c1b74b06b678f45544883e567e2b4eb55
Closes-Bug: #1843235

Nova services may reasonably expect cell databases to exist when they
start. The current cell setup tasks in kolla run after the nova
containers have started, meaning that cells may or may not exist in the
database when they start, depending on timing. In particular, we are
seeing issues in kolla CI currently with jobs timing out waiting for
nova compute services to start. The following error is seen in the nova
logs of these jobs, which may or may not be relevant:

No cells are configured, unable to continue

This change creates the cell0 and cell1 databases prior to starting nova
services.

In order to do this, we must create new containers in which to run the
nova-manage commands, because the nova-api container may not yet exist.
This required adding support to the kolla_docker module for specifying a
command for the container to run that overrides the image's command.

We also add the standard output and error to the module's result when a
non-detached container is run. A secondary benefit of this is that the
output of bootstrap containers is now displayed in the Ansible output if
the bootstrapping command fails, which will help with debugging.

Change-Id: I2c1e991064f9f588f398ccbabda94f69dc285e61
Closes-Bug: #1808575

If upgrading the nova, cinder or manila services via 'kolla-ansible
upgrade', the Ceph config files are not generated. Users will expect
that these files are generated, to pull in any changes from their
configuration or the base kolla configuration.

This change moves Ceph tasks inside config.yml to ensure that they are
performed during deploy, reconfigure and upgrade. This has been done for
nova, cinder, gnocchi and manila - glance already does this.

Change-Id: Ic75692c2bcba9b81dee922ff6fbbccd160e7fa19
Closes-Bug: #1794275

Add a possibility to mount sources as volumes to containers,
in "more than documentation" way. That will let us to use kolla
as a replacement for devstack.

Partially implements: blueprint mount-sources
Co-Authored-By: zhulingjie <easyzlj@gmail.com>

Change-Id: I10677e5ad22f2107a0657feeeaf32287ab9f8e28

include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated



Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a

It is not necessary to enable the nova cephfs backend to make use
of cinder cephfs volumes.

Change-Id: I35c3d2e49769962e5c47f585d91d1efd492a53d6
Closes-Bug: #1778107

When the virt dirver is xenapi, we should explicitly specify host
with dom0's hostname for nova-compute. The host field should be same
as neutron-openvswitch-agent-xenapi.

In order to use dom0's hostname, we need pull the bootstrap_xenapi
to before running nova config tasks. The dom0's hostname will be
included in the XenAPI facts.

Change-Id: Ibfc964bbd6c569062ad33c1399f9a4612baf5b92
blueprint: xenserver-support

It is external_ceph.yml for cinder/glance/gnocchi.

Change-Id: I534bffe5fc914e2850fb5cb5719fd99afa06c6c4

* "bool" filter is removed when not required
* 'not' is used instead of '== False' check

Change-Id: I85a5bb9a5ea874ac1c397cbf8de416147d2424c3

Change-Id: I64dd52d909a884b2b9aea6f016d789d99caa3f17
Closes-Bug: #1652580

Change-Id: Ida7c955d9ddf7b7dc3289001958326c38c62d0c9
Partially-implements: blueprint better-reconfigure

Change-Id: Ifd40ca95c82fafacaf91291511f7048351d1bf42
Partially-Implements: blueprint nova-proxies

Introduced nova backend selection flag for Ceph and priority if
multiple backends are configured

Add mechanism to deploy arbitrary ceph.conf and keyring files into
nova-compute and nova-libvirt containers

Added documentation

Change-Id: Id010ca9cc2d914e5358ef79edeb600a28220dd4b
Implements: blueprint external-ceph

Due to changes with the drop-root work, we lost the ability to write
to /var/lib/nova/*. This fixes those permissions and ensures cross
container talk works properly between nova_libvirt and nova_compute

Additionally, this fixes another issue introduced which saw that
nova-compute could not run sudo commands as it did not have a proper
sudoers entry

Testing from previous deploys means you need a fresh environment. You
have to remove all of the named volumes that kolla created in docker.
Check these with `docker volume ls`

Signed-off-by: Hui Kang <kangh@us.ibm.com>
Signed-off-by: Artur Zarzycki <azarzycki@mirantis.com>
Co-Authored-By: Sam Yaple <sam@yaple.net>
Co-Authored-By: Hui Kang <kangh@us.ibm.com>
Closes-Bug: #1533350
Change-Id: I7f864c448a2414e0b5d89f48337be411b891df35

Change-Id: Ic20c31efea461f30917a94546db6e261a8b5166f
Partially-Implements: blueprint pre-pull-images

- See instructions in doc/nova-fake-driver.rst

Implements: blueprint nova-fake-driver

Change-Id: I553a40c2df39bdcc391eb1b8b2b8fd5f4ed48c33
Signed-off-by: Hui Kang <kangh@us.ibm.com>
Signed-off-by: Marcio D. Silva <marcios@us.ibm.com>

The bootstrap tasks could be started on any node listed in the
site.yml

The issue is that all the tasks must run on the same node, and the
only node that all the tasks can run on is the 'api' node due to
needing to start the bootstrap container which binds in the configs.

delegate_to is required to ensure that the proper node gets the task.

Co-Authored-By: Sam Yaple <sam@yaple.net>
Change-Id: Ica04ab366777a571a92212ac22c482f1881d4ecd
Closes-Bug: #1513598
Related-Bug: #1513439

Adjust all the configs to list all the rabbitmq hosts rather than
running rabbitmq through the VIP. This is made possible by clusterer
which has already merged.

Change-Id: I5db48f5f10ec68f4c8863a29bc13984f6845a4f9
Partially-Implements: blueprint rabbitmq-clusterer

Change-Id: I902e771a6e377f9ce3e452c123b2bbf65e3a2d7f
Implements: blueprint ceph-container

This implements all the openstack pieces needed to make ceph work.

DocImpact
Change-Id: I1d24476a966602cf955e5ef872b0efb01319894a
Partially-Implements: blueprint ceph-container
Implements: blueprint kolla-live-migration

Initial support for Nova in Ansible

Partially-Implements: blueprint ansible-service

Change-Id: I4b0a74bd3e5daa5664f5e1e622bfb40c3285949e

Due to the licensing issues and the modules not existing in upstream
Ansible yet, I have written a simple module to fill the gaps.

This also uses Keystone v2.0 for all create of users, roles and
endpoints. The implementation of Keystone v3 must be discussed after the
new modules arrive.

Partially-Implements: blueprint ansible-service

Change-Id: I389edd56741360dd26fbbc0a982f365ca27ff446

Adds support for glance in ansible

Partially-Implements: blueprint ansible-service
Change-Id: I2d162e79f85877116b0e33b1843de6ccb62a445a

Add set_configs function that implements the flow from the proposed
ansible-multi spec. Move start.sh to config-internal.sh to preserve existing
behaviour.

config-externall.sh copies the appropriate configs in from the bind'd
location and sets permissions and ownership appropriately.

Partially Implements: blueprint ansible-multi

Change-Id: I53fca0660451087f273fefc3c63e0d8cf1a2c096

Add a auth role so ansible can deploy
keystone.

Change-Id: I3f41fe150654451aae6271cce59ddeb807945043