hostnqn is generated using to_uuid filter

Usually "nvme gen-hostnqn" command is used to generate
hostnqn, and it has the format of:
nqn.2014-08.org.nvmexpress:uuid:67dc8c8e-0262-4d81-ac51-ace7c25e4daa

"nqn.2014-08.org.nvmexpress:uuid:" is always static

Closes-Bug: #2035975

Change-Id: I6ece4fe8c18c0167a2707c24693fbe39ed15cdba

When using the 'q35' machine type, by default, it allows only
a single PCIe device to be hotplugged. Nova currently sets
'num_pcie_ports' to "0" (defaults to libvirt's "1"), which is
not sufficient for hotplug use.

The max value is 28, but 16 is enough in most cases.

More info please check

* [0] https://libvirt.org/pci-hotplug.html
* [1] https://marcin.juszkiewicz.com.pl/2018/02/19/hotplug-in-vm-easy-to-say/
* [2] https://review.opendev.org/c/openstack/nova/+/545034



Co-Authored-By: Bartosz Bezak <bartosz@stackhpc.com>

Change-Id: I7b8063566837aeb947927b7d6858417982b63deb
Closes-Bug: 1909063

For example, an operator may wish to customise the
nova-compute-ironic service configuration without affecting
other Nova services.

Closes-Bug: #2034949

Change-Id: If8648d8e85ab3dbcbb4ecba674b2e34b06898327

With the libvirt driver, during live migration,skip comparing guest CPU
with the destination host. When using QEMU >= 2.9 and libvirt >= 4.4.0,
libvirt will do the correct thing with respect to checking CPU
compatibility on the destination host during live migration.[1]

[1] https://opendev.org/openstack/nova/commit/267a40663cd8d0b94bbc5ebda4ece55a45753b64

Change-Id: I947c94b59368c7a2740583bf57e407296473d75e

Use case: exposing single external https frontend and
load balancing services using FQDNs.

Support different ports for internal and external endpoints.

Introduced kolla_url filter to normalize urls like:
- https://magnum.external:443/v1
- http://magnum.external:80/v1



Change-Id: I9fb03fe1cebce5c7198d523e015280c69f139cd0
Co-Authored-By: Jakub Darmach <jakub@stackhpc.com>

This patch is adding a feature for an option to copy different
ceph configuration files and corresponding keyrings for cinder,
glance, manila, gnocchi and nova services.

This is especially useful when the deployment uses availability
zones as below example.

  - Individual compute can read/write to individual ceph
    cluster in same AZ.
  - Cinder can write to several ceph clusters in several AZs.
  - Glance can use multistore and upload images to
    several ceph clusters in several AZs at once.

Change-Id: Ie4d8ab5a3df748137835cae1c943b9180cd10eb1

As of I3629b84d3255a8fe9d8a7cea8c6131d7c40899e8 nova
now requires the service_user section to be configured
to address CVE-2023-2088. This change adds
the service user section to the nova.conf template in
the nova and nova-cell roles.

Related-Bug: #2004555
Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: I2189dafca070accfd8efcd4b8cc4221c6decdc9f
(cherry picked from commit a77ea13ef1991543df29b7eea14b1f91ef26f858)
(cherry picked from commit 03c12abbcc107bfec451f4558bc97d14facae01c)
(cherry picked from commit cb105dc293ff1cdb11ab63fa3e3bf39fd17e0ee0)
(cherry picked from commit efe6650d09441b02cf93738a94a59723d84c5b19)

Add checking for container readiness before create sasl user

Closes-Bug: #2015589
Change-Id: Ic650ba6be1f192e3cbeaa94de3d00507636c1c92

Closes-Bug: #2005119
Change-Id: I542f7ae19b4400355b04854f42a1d1802a6efeea

When generating config, we do not want to restart any services.

TrivialFix

Change-Id: Ice13759c406ff7cec91c9d740b0e0c43eda99510

Previously, when running one of the following commands:

  kolla-ansible deploy --check
  kolla-ansible genconfig --check

deployment or configuration generation fails for various reasons.

MariaDB fails to lookup the existing cluster.

Keystone fails to generate cron config.

Nova-cell fails to get the cell settings.

Closes-Bug: #2002661
Change-Id: I5e765f498ae86d213d0a4379ca5d473db1499962

As per the RBAC new direction in Zed cycle, we have dropped the
system scope from API policies and all the policies are hardcoded
to project scoped so that any user accessing APIs using system scope
will get 403 error. It is dropped from all the OpenStack services
except for the Ironic service which will have system scope and to
support ironic only deployment, we are keeping system as well as project
scope in Keystone.

Complete discussion and direction can be found in the below gerrit
change and TC goal direction:

- https://review.opendev.org/c/openstack/governance/+/847418
- https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#the-issues-we-are-facing-with-scope-concept

As phase-2 of RBAC goal, services will start enabling the new
defaults and project scope by default. For example: Nova did in
- https://review.opendev.org/c/openstack/nova/+/866218

Kolla who start accessing the services using system scope token
- https://review.opendev.org/c/openstack/kolla-ansible/+/692179

This commit partially revert the above change except keeping
system scope usage for Keystone and Ironic. Rest all services are changed
to use the project scope token.

And enable the scope and new defaults for Nova which was disabled
by https://review.opendev.org/c/openstack/kolla-ansible/+/870804

Change-Id: I0adbe0a6c39e11d7c9542569085fc5d580f26c9d

In order to honour configured max number of attempts
it has to be presented in nova.conf inside of
nova_conductor container, otherwise the default value
of 3 will be used

Closes-Bug: #2003587
Change-Id: I928af332b8658223444594f96417830233057284

A combination of durable queues and classic queue mirroring can be used
to provide high availability of RabbitMQ. However, these options should
only be used together, otherwise the system will become unstable. Using
the flag ``om_enable_rabbitmq_high_availability`` will either enable
both options at once, or neither of them.

There are some queues that should not be mirrored:
* ``reply`` queues (these have a single consumer and TTL policy)
* ``fanout`` queues (these have a TTL policy)
* ``amq`` queues (these are auto-delete queues, with a single consumer)
An exclusionary pattern is used in the classic mirroring policy. This
pattern is ``^(?!(amq\\.)|(.*_fanout_)|(reply_)).*``

Change-Id: I51c8023b260eb40b2eaa91bd276b46890c215c25

When running in check mode, some prechecks previously failed because
they use the command module which is silently not run in check mode.
Other prechecks were not running correctly in check mode due to e.g.
looking for a string in empty command output or not querying which
containers are running.

This change fixes these issues.

Closes-Bug: #2002657
Change-Id: I5219cb42c48d5444943a2d48106dc338aa08fa7c

The ``[oslo_messaging_rabbit] heartbeat_in_pthread`` config option
is set to ``true`` for wsgi applications to allow the RabbitMQ
heartbeats to function. For non-wsgi applications it is set to ``false``
as it may otherwise break the service [1].

[1] https://docs.openstack.org/releasenotes/oslo.messaging/zed.html#upgrade-notes

Change-Id: Id89bd6158aff42d59040674308a8672c358ccb3c

Regularly, we experience issues in Kolla Ansible deployments because we
use wrong options in OpenStack configuration files. This is because
OpenStack services ignore unknown options. We also need to keep on top
of deprecated options that may be removed in the future. Integrating
oslo-config-validator into Kolla Ansible will greatly help.

Adds a shared role to run oslo-config-validator on each service. Takes
into account that services have multiple containers, and these may also
use multiple config files. Service roles are extended to use this shared
role. Executed with the new command ``kolla-ansible validate-config``.

Change-Id: Ic10b410fc115646d96d2ce39d9618e7c46cb3fbc

See https://github.com/ansible/ansible/pull/76677

Change-Id: If822dcfc4c1abf7a22be35ffd90fd05ee46cb0d9

Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

THis change adds container_engine to module parameters
so when we introduce podman, kolla_toolbox can be used
for both engines.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: Ic2093aa9341a0cb36df8f340cf290d62437504ad

Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

This change adds container_engine variable to kolla_container_facts
module, this prepares module to be used with docker and podman as well
without further changes in roles.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I9e8fa30646844ab4a288555f3aafdda345b3a118

First part of patchset:
 https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

This implements kolla_container_engine variable
in command calls of docker,so later on it can be
also used for podman without further change.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Change-Id: Ic30b67daa2e215524096ad1f4385c569e3d41b95

In the Victoria cycle, Nova merged improved support for
managing resource providers:
https://review.opendev.org/q/topic:bp%252Fprovider-config-file

See the blueprint for more details:
https://docs.openstack.org/nova/latest/admin/managing-resource-providers.html

This change allows us to copy the necessary configuration.

Change-Id: I0a3caaad73bc6fe27380e7f6bf6b792aca51c84c

Kolla Ansible stopped setting them as they turned out to be
unnecessary for its operations, yet may have conflicted with
security policies of the hosts. [1] [2]

[1] https://launchpad.net/bugs/1837551
[2] https://launchpad.net/bugs/1945453

Change-Id: Ie8ccd3ab6f22a6f548b1da8d3acd334068dc48f5

mainly jinja spacing and jinja[invalid] related

Change-Id: I6f52f2b0c1ef76de626657d79486d31e0f47f384

This patch adds loadbalancer-config role
which is "wrapper" around haproxy-config
and proxysql-config role which will be added
in follow-up patches.

Change-Id: I64d41507317081e1860a94b9481a85c8d400797d

Depends-On: https://review.opendev.org/c/openstack/kolla/+/769385
Depends-On: https://review.opendev.org/c/openstack/kolla/+/765781

Change-Id: I3c4182a6556dafd2c936eaab109a068674058fca

ansible-lint introduced var-spacing - let's fix our code.

Change-Id: I0d8aaf3c522a5a6a5495032f6dbed8a2be0251f0

Render {{ openstack_service_workers }} for workers
of each openstack service is not enough. There are
several services which has to have more workers because
there are more requests sent to them.

This patch is just adding default value for workers for
each service and sets {{ openstack_service_workers }} as
default, so value can be overrided in hostvars per server.
Nothing changed for normal user.

Change-Id: Ifa5863f8ec865bbf8e39c9b2add42c92abe40616

Per comments on [1].

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/843727

Change-Id: I60162b54bc06e158534d29311d4474b34750c64d

Following up on [1].
The 3 variables are only introducing noise after we removed
the reliance on Keystone's admin port.

[1] I5099b08953789b280c915a6b7a22bdd4e3404076

Change-Id: I3f9dab93042799eda9174257e604fd1844684c1c

Change-Id: Ib4b15ed4feac82d8492b1c0f0238a752eac668e6

We run some nova tasks once per cell, using a condition to match a
single host in the cell. In other similar tasks, we use run_once, which
will fail all hosts if the task fails. Typically these tasks are
critical, and that is desirable. However, with the approach used in
nova-cell to support multiple cells, if a once-per-cell task fails, then
other hosts will continue to execute, which could lead to unexpected
results.

This change adds any_errors_fatal to the plays or blocks that run these
tasks.

Closes-Bug: #1948694

Change-Id: I2a5871ccd4e8198171ef3239ce95f475f3e4b051

We have only one value for install_type now and it gets removed from
image names.

Change-Id: I8bf95fd7aa9dd26b80d618ca0fcb097003b4cb0a

This change addresses an issue in the nova-libvirt-cleanup command,
added in I46854ed7eaf1d5b5e3ccd8531c963427848bdc99.

Check for rc=1 pgrep command, since a lack of matches is a pass.

Also, use bash for set -o pipefail.

Change-Id: Iffda0dfffce8768324ffec55e629134c70e2e996

If any nova compute service fails to register itself, Kolla Ansible will
fail the host that queries the Nova API. This is the first compute host
in the inventory, and fails in the task:

    Waiting for nova-compute services to register themselves

Other hosts continue, often leading to further errors later on. Clearly
this is not idea.

This change modifies the behaviour to query the compute service list
until all expected hosts are present, but does not fail the querying
host if they are not. A new task is added that executes for all hosts,
and fails only those hosts that have not registered successfully.

Alternatively, to fail all hosts in a cell when any compute service
fails to register, set nova_compute_registration_fatal to true.

Change-Id: I12c1928cf1f1fb9e28f1741e7fe4968004ea1816
Closes-Bug: #1940119

Designate sink is an optional service that consumes notifications,
users should have an option to disable it when they don't use them.

Change-Id: I1d5465d9845aea94cff39ff5158cd8b1dccc4834

Change Ia1239069ccee39416b20959cbabad962c56693cf added support for
running a libvirt daemon on the host, rather than using the nova_libvirt
container. It did not cover migration of existing hosts from using a
container to using a host daemon.

This change adds a kolla-ansible nova-libvirt-cleanup command which may
be used to clean up the nova_libvirt container, volumes and related
items on hosts, once it has been disabled.

The playbook assumes that compute hosts have been emptied of VMs before
it runs. A future extension could support migration of existing VMs, but
this is currently out of scope.

Change-Id: I46854ed7eaf1d5b5e3ccd8531c963427848bdc99

In some cases it may be desirable to run the libvirt daemon on the host.
For example, when mixing host and container OS distributions or
versions.

This change makes it possible to disable the nova_libvirt container, by
setting enable_nova_libvirt_container to false. The default values of
some Docker mounts and other paths have been updated to point to default
host directories rather than Docker volumes when using a host libvirt
daemon.

This change does not handle migration of existing systems from using
a nova_libvirt container to libvirt on the host.

Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/830504

Change-Id: Ia1239069ccee39416b20959cbabad962c56693cf

Consistently use template instead of copy. This has the added
advantage of allowing variables inside ceph conf files and keyrings.

Closes-Bug: 1959565

Signed-off-by: Imran Hussain <ih@imranh.co.uk>
Change-Id: Ibd0ff2641a54267ff06d3c89a26915a455dff1c1

In Kolla Ansible OpenStack deployments, by default, libvirt is
configured to allow read-write access via an unauthenticated,
unencrypted TCP connection, using the internal API network.  This is to
facilitate migration between hosts.

By default, Kolla Ansible does not use encryption for services on the
internal network (and did not support it until Ussuri). However, most
other services on the internal network are at least authenticated
(usually via passwords), ensuring that they cannot be used by anyone
with access to the network, unless they have credentials.

The main issue here is the lack of authentication. Any client with
access to the internal network is able to connect to the libvirt TCP
port and make arbitrary changes to the hypervisor. This could include
starting a VM, modifying an existing VM, etc. Given the flexibility of
the domain options, it could be seen as equivalent to having root access
to the hypervisor.

Kolla Ansible supports libvirt TLS [1] since the Train release, using
client and server certificates for mutual authentication and encryption.
However, this feature is not enabled by default, and requires
certificates to be generated for each compute host.

This change adds support for libvirt SASL authentication, and enables it
by default. This provides base level of security. Deployments requiring
further security should use libvirt TLS.

[1] https://docs.openstack.org/kolla-ansible/latest/reference/compute/libvirt-guide.html#libvirt-tls

Depends-On: https://review.opendev.org/c/openstack/kolla/+/833021
Closes-Bug: #1964013
Change-Id: Ia91ceeb609e4cdb144433122b443028c0278b71e