* In service_credentials section, project_name should be
admin and auth_url should use keystone public port (5000)
according [1].
* User vitrage should have role admin in project admin [2].

[1] https://github.com/openstack/vitrage/blob/master/devstack/plugin.sh#L164
[2] https://docs.openstack.org/vitrage/latest/install/install-rdo.html#initialize-vitrage

Change-Id: I8bf028c702df5659e4c58df0e17600ab5be14eea

This is required to support execution as a non-root user.

Change-Id: I60d224407c2828d6b9f1701f7637385a25fbcced
Closes-Bug: #1809233

Change-Id: If5b4ba975a65e07d2704eb6bdb9d841d6a9c3d42

In multi controller deployment, kolla will generate
"controller_ip_port_list option" in [health_manager] section with
ONLY IP of that node instead of a list of controller ip.
Therefor, "amphora-agent.conf" file of amphora instance will
contain IP of ONLY ONE controller node.
In case of that node fail, amphora agent won't send heartbeat
message to other health manager node, and the loadbalancer will
go to ERROR state.

Change-Id: I102ed6ba3fff2c12cc6d37f81ad59508eacc859c
Co-Authored-By: Hieu LE <hieulq2@viettel.com.vn>

Update the template so that if 'dns_interface' is set, named listens on
this interface as well as the 'api_interface'.

Change-Id: I986ca46e5599e4767800fcc7f34a1c6e682efb55
Closes-Bug: 1808829

Add missing `prometheus_memcached_exporter` container_fact
Fix conditional container_fact for haproxy_exporter

Change-Id: Id0f3b94af956f51e3c782c0244c6ce7a340119bd
Closes-Bug: #1808820

Add an enable_cinder_backend_quobyte option to etc/kolla/globals.yml to
enable use the Quobyte Cinder backend.
Change the bind mounts for /var/lib/nova/mnt to include the shared
propogation if Quobyte is enabled.
Update the documentation to include a section on configuring the Cinder.

Implements: blueprint cinder-quobyte-backend

Change-Id: I364939407ad244fe81cea40f880effdbcaa8a20d

According [1], vitrage notification has to be configured in Nova,
Neutron, Cinder & Aodh config file.

[1] https://review.openstack.org/#/c/302802/

Change-Id: Iaf8cd7d40e6eb988adf4d208e6ad784f1004caa5

Find module searches paths on managed server. Since role path and custom
Kolla config is located on deployment node and deployment node is not
considered to be a managed server, Monasca plugin files cannot be found.
After the deployment container running Monasca agent collector stucks in
restart mode due to missing plugin files.

The problem does not occur if deployment was started from a managed
server (eg. OSC). The problem occurs if the deployment was started from
a separate deployment server - a common case.

This change enforces running find module locally on deployment node.

Change-Id: Ia25daafe2f82f5744646fd2eda2d255ccead814e
Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>

In multinode deployments creating default Grafana organization failed,
because Ansible attempted to call Grafana API in the context of each
host in the inventory. After creating organization via the first host,
subsequent attempts via the remaining hosts failed due to already
existing organization. This change enforces creating default
organization only once.

Other tasks using Grafana API have been enforced to be ran only once as
well.

Change-Id: I3a93a719b3c9b4e55ab226d3b22d571d9a0f489d
Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>

Nova services may reasonably expect cell databases to exist when they
start. The current cell setup tasks in kolla run after the nova
containers have started, meaning that cells may or may not exist in the
database when they start, depending on timing. In particular, we are
seeing issues in kolla CI currently with jobs timing out waiting for
nova compute services to start. The following error is seen in the nova
logs of these jobs, which may or may not be relevant:

No cells are configured, unable to continue

This change creates the cell0 and cell1 databases prior to starting nova
services.

In order to do this, we must create new containers in which to run the
nova-manage commands, because the nova-api container may not yet exist.
This required adding support to the kolla_docker module for specifying a
command for the container to run that overrides the image's command.

We also add the standard output and error to the module's result when a
non-detached container is run. A secondary benefit of this is that the
output of bootstrap containers is now displayed in the Ansible output if
the bootstrapping command fails, which will help with debugging.

Change-Id: I2c1e991064f9f588f398ccbabda94f69dc285e61
Closes-Bug: #1808575

xtrabackup doesnt work with mariadb 10.3,
need to be changed to mariadb-backup tool.

For now only migrate galera, not kolla-backup tool
to fix the CI.

https://jira.mariadb.org/browse/MDEV-15774

Change-Id: Ie77ae41e419873feed4b036a307887b22455183b
Depends-On: Icefe3a77fb12d57c869521000d458e3f58435374

when using ceilometer+gnocchi, for every notification sample, ceilometer
will update the resource even if is not updated.

We should add [cache] section to make ceilometer cache the resource, and
stop send the useless update request.

Closes-Bug: #1807841
Change-Id: Ic33b4cd5ba8165c20878cab068f38a3948c9d31d

Vitrage has already supported Prometheus as
datasource. Kolla can config it automatically,
just need a little changes, for example in
wsgi config file [1].

Co-Authored-By: Hieu LE <hieulq2@viettel.com.vn>

[1] https://review.openstack.org/#/c/584649/8/devstack/apache-vitrage.template

Change-Id: I64028a0dfd9887813b980a31c30c2c1b1046da61

Prior to this change, when the --limit argument is used, each host in the
limit gathers facts for every other host. This is clearly unnecessary, and
can result in up to (N-1)^2 fact gathers.

This change gathers facts for each host only once. Hosts that are not in
the limit are divided between those that are in the limit, and facts are
gathered via delegation.

This change also factors out the fact gathering logic into a separate
playbook that is imported where necessary.

Change-Id: I923df5af41a7f1b7b0142d0da185a9a0979be543

Currently, every service has a play in site.yml that is executed, and
the role is skipped if the service is disabled. This can be slow,
particularly with many hosts, since each play takes time to setup, and
evaluate.

This change creates various Ansible groups for hosts with services
enabled at the beginning of the playbook. If a service is disabled, this
new group will have no hosts, and the play for that service will be a
noop.

I have tested this on a laptop using an inventory with 12 hosts (each
pointing to my laptop via SSH), and a config file that disables every
service. Time taken to run 'kolla-ansible deploy':

Before change: 2m30s
After change: 0m14s

During development I also tried an approach using an 'include_role' task
for each service. This was not as good, taking 1m00s.

The downsides to this patch are that there is a large number of tasks at
the beginning of the playbook to perform the grouping, and every play
for a disabled service now outputs this warning message:

[WARNING]: Could not match supplied host pattern, ignoring: enable_foo_True

This is because if the service is disabled, there are no hosts in the
group. This seems like a reasonable tradeoff.

Change-Id: Ie56c270b26926f1f53a9582d451f4bb2457fbb67

refer:
https://docs.ansible.com/ansible/2.5/modules/package_module.html

Change-Id: I68a0eb64a61bc6c0f77cbae7e8b4f4c7143202c5

This change adds support to comfigure tty,
it was enabled by default but a recent patch
removed it. Some services such as Karaf in opendaylight
requires a TTY during startup.

Closes-Bug: #1806662
Change-Id: Ia4335523b727d0e45505cbb1efb40ccf04c27db7

When using external ceph, enable_ceph=no and glance_backend_ceph=yes,
glance.conf should enable rbd store.

Change-Id: Ia09cd57c829b00f28674cddf44fb55583e193d0f

Nova allows customisation of various metadata passed through to VMs via
a 'release' file[0]. Allow operators to make use of this.

[0] https://github.com/openstack/nova/blob/master/etc/nova/release.sample

Change-Id: I71569314c8e64320f8ffad79b9273f4d6d903bb6

Remove mode "0660" because mode it's not a supported parameters for kolla_docker

Change-Id: I1e3d690eb3cb5d61b1c88f6da2f9b10e2c5f3603
Closes-Bug: #1804702

With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers

blueprint database-backup-recovery

Introduce a new option, mariadb_backup, which takes a backup of all
databases hosted in MariaDB.

Backups are performed using XtraBackup, the output of which is saved to
a dedicated Docker volume on the target host (which defaults to the
first node in the MariaDB cluster).

It supports either full (the default) or incremental backups.

Change-Id: Ied224c0d19b8734aa72092aaddd530155999dbc3

Glance cache is used to keep a locally cache image
in the glance_api service.
Is an usefull service when an image is commonly used
to speed times between pulling from storage backend
and send to nova.

Change-Id: I8e684cc10e4fee1cb52c17a126e3b11f69576cf6

The configfs kernel module is not mounted by default in Ubuntu 16.04,
leading to the iscsid container failing to start because it bind mounts
/sys/kernel/config. The issue does not apply to Ubuntu 18.04, or other
distros (AFAIK), which load configfs by default.

This change loads the configfs module when the iscsid container is in
use.

Change-Id: I5b521ddca24b919658d2664ede2d878507d6d106
Closes-Bug: #1631072

Refer to neutron docs [1][2]
[1]: https://docs.openstack.org/neutron/pike/admin/archives/adv-config.html#l3-metering-service-driver
[2]: https://wiki.openstack.org/wiki/Neutron/Metering/Bandwidth


Co-Authored-By: ZhijunWei <wzj334965317@outlook.com>

Change-Id: I4a676d041bc2a86497cb139d9347365738b156df

The dnsmasq PXE filter [1] provides far better scalability than the
iptables filter typically used. Inspector manages files in a dhcp-hostsdir
directory that is watched by dnsmasq via inotify. Dnsmasq then either
whitelists or blacklists MAC addresses based on the contents of these
files.

This change adds a new variable, ironic_inspector_pxe_filter, that can
be used to configure the PXE filter for ironic inspector. Currently
supported values are 'iptables' and 'dnsmasq', with 'iptables' being the
default for backwards compatibility.

[1]
https://docs.openstack.org/ironic-inspector/latest/admin/dnsmasq-pxe-filter.html

Implements: blueprint ironic-inspector-dnsmasq-pxe-filter
Change-Id: I73cae9c33b49972342cf1984372a5c784df5cbc2

OpenDaylight logs have different format than openstack,
is a karaf log with java error traces.
This PS add required config to make fluentd parse properly
ODL logs.

Change-Id: I34fb96c8a424679b3b618f2ff6a840b8dc165bec

At the moment the "databases user and setting permissions" task for
designate and nova leaks the database_password because of the use
of with_items:

---snip---
TASK [nova : Creating Nova databases user and setting permissions] *********************************************************
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova', u'database_username': u'nova'})
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova_cell0', u'database_username': u'nova'})
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova_api', u'database_username': u'nova_api'})
---snap---

Change-Id: I141e4153223c8772c82a31d81e58057ce266c0b9
Co-authored-by: Bernd Müller <mueller@b1-systems.de>

add gather_facts to false to avoid duplicated

Change-Id: I633fc9042e7df24f5cdf3a62fd7ec619270469d5
Closes-Bug: #1806443

Tested on Rocky, /v3 needs to be added to the variable auth_url to have
the trust/trustee mechanism to work. All cluster creation would fail
otherwise.

Closes-Bug: #1805896
Change-Id: Ieedac124fa22e5a7ae622c16d47d482007bbec60

We copy-paste the same play into various playbooks to detect
openstack_release. This change factors that code into a separate
playbook that is imported.

Change-Id: I5fea005642b960080bf5e43455618dc24766c386

Tested on Rocky, it seems there is no admin_* variables and some others
are missing (username/password/...) causing keystone to return http code
400 responses.

Change-Id: If4a0919bfcd6b8d8a6bfd5df9001b4967e441e7e
Closes-Bug: #1805714

From Karbor documentation, endpoints should be created with
"%(project_id)s" and not with "%(tenant_id)s".

This is very important because of this commit in Karbor which is
looking for a string "project_id".

Change-Id: I8fc640891d0d58541198cc8f2e942d8db6e8d02f
Closes-Bug: #1805705

region_id has a default value hardcoded in Karbor code equal to
"RegionOne" which could be an issue if a different region is define.

Change-Id: Ia13496156515d0f871e8fa9bd3584940a32759e9
Closes-Bug: #1798125

Change-Id: I9fbde5e26c76d8fc78b51e12e7cb7baf66f8602a
Closes-Bug: #1796366

A spec to Deprecate the Glance Registry Service[0] was accepted in Newton,
but it contained the ambiguous statement, "Mark the service as deprecated
and ready for removal in the Q release." kolla-ansible disable the
glance-registry in Q release[1], and since we are in S now,
remove glance-registry is safe.

[0]: http://specs.openstack.org/openstack/glance-specs/specs/newton/approved/glance/deprecate-registry.html
[1]: https://review.openstack.org/#/c/566804/

Change-Id: I48f794029e97aa6f76bbd500e33f28f51a3f2ac4

The variable {{ node_config_directory }} is used for the configuration
directory on the remote hosts, and should not be used for paths on the
deploy host (localhost).

This changes the default value of the TLS certificate and CA file to
reference {{ CONFIG_DIR }}, in line with the directory used for
admin-openrc.sh (as of I0709482ead4b7a67e82796e17f85bde151e71bc0).

This change also introduces a variable, {{ node_config }}, that
references {{ CONFIG_DIR | default('/etc/kolla') }}, to remove
duplication.

Change-Id: Ibd82ac78630ebfff5824c329d7399e1e900c0ee0
Closes-Bug: #1804025

add monitoring of zookeeper

Change-Id: I1db23b67dd8f83f9341a376d9cd2d9a49abd4427

By default the driver used is elasticsearch in version 2
This change updates the driver with the good one.
It also updates backend with name used in the documentation.

Change-Id: I80f3020cb42903ae48ef65f52f67aae977c5a56b