This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121

Fixed misspelling in both comments and code function names 

Change-Id: Iaed5f246a9b02fd64433671fd0c0ee41b40bd222

Zun use oslo.concurrency to achieve safely running multi-thread,
multi-process applications. The lock_path is directory to use for
lock files. If external locks are used, lock path must be set.

Change-Id: I4f667d9699ec2231722bcfd7c5f84ba47fc8465c

- Heat
- Ironic
- Magum
- Manila
- Mistral

This will copy only yaml or json policy file if they exist.

Change-Id: I1ab71e2758dc99dd6654d433ece79600f0c44ce8
Implements: blueprint support-custom-policy-yaml
Co-authored-By: Duong Ha-Quang <duonghq@vn.fujitsu.com>

Memcached do not support cluster. Then make it work in active-standby
mode. This will be helpful to implement high available when using memcached
as tooz backend.

Change-Id: I13722111d8b8d5b066e9a85d4c8d1679704c8caa

Change-Id: Ia909c62158acbec9d1dd926d45c66ac36e6b4244
Depends-On: I06325d4601128a0b5a2235370fa44012427c29e3
Implements: blueprint ceph-nfs-ganesha

When using XenAPI as the compute virt driver, we need an OVS agent
to manage the OVS running in XenServer dom0. This OVS agent uses
the HIMN(Host Internal Management Network) to communicate with
dom0's OVS. This commit includes the following changes:

* Add a new ovs agent service - neutron-openvswitch-agent-xenapi
  This new agent service will run in the compute hosts and controls
  the OVS running in XenServer dom0; the existing agent service -
  neutron-openvswitch-agent will run in the network hosts and controls
  the OVS running in network hosts.

* It retrieves XenAPI variables from the json file generated at XenAPI
  bootstrap.

* Basing on the XenAPI variables, it will customize relative ml2_conf.ini's
  configure options in a new template which will override the default options.
  e.g.
  * of_listen_address:
  XenAPI use the local himn interface's IP as of_listen_address, so
  that the ovs running dom0 can receive OpenFlow rules from the service
  of neutron-openvswitch-agent-xenapi.
  * ovsdb_connection:
  XenAPI use XenServer dom0's HIMN IP as the OVS DB connection IP, so
  that neutron-openvswitch-agent-xenapi can connect to dom0's OVS DB.
  * host:
  Use the dom0's hostname.

* At the moment, l2_population doesn't for for XenAPI. So disable it.

References:

* XenServer (and other XAPI based Xen variants):
  https://docs.openstack.org/nova/pike/admin/configuration/hypervisor-xen-api.html

* XenCenter HIMN plugin (adding HIMN network which is used by XenAPI driver to
  communicate with XenServer):
  https://github.com/citrix-openstack/xencenter-himn-plugin

* Neutron OVS agent configuration options:
  https://docs.openstack.org/neutron/latest/configuration/openvswitch-agent.html

Change-Id: Iaee0a6c84069b3e6015b00de7aea880cdd33ab09
blueprint: xenserver-support

This is to add a bootstrap task for XenAPI to bootstrap the compute
node. One compute node is composed with a XenServer and a DomU VM
running on the XenServer. Before it's ready for deploying OpenStack
sevices on it, we need bootstrap it firstly. The package of
python-os-xenapi contains a tool and some utils which were developed
to finish all of the needed bootstrap tasks
e.g.:
1. configure himn(host internal management network)
2. configure ipatables to allow traffic
3. install xapi plugins to dom0
4. gather XenAPI facts and save them into a file
...

The task added in this commit is to invoke that tool so that the
bootstrap tasks can be completed automatically during kolla deployment.

Depends-On: Ie2d7d40f2755580aac4a10f3d302190a8bd4fe6f
Change-Id: I1500535a1c9a085bcbfec5db5fbddcf040e6892d
blueprint: xenserver-support

the precheck action should better move in precheck.yml

Change-Id: I233cc73e7c16dd0d7f98338966d964417da7d0ad

Depends-On: I5bf14a668a9e8976a18aa28a5bbfe586690bbd31
Implements: blueprint manila-cephfs-backend

Change-Id: I483be398d929429e35e460b5a7660b82f409b587

when gnocchi use ceph as backend, need set
gnocchi_backend_storage: "ceph", gnocchi will pull keyring
from ceph mon node [0], but the site.yml missing it.

[0]: https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/gnocchi/tasks/ceph.yml#L39

Change-Id: Ib33c26ee1fdfe683e5c9b3ef987d6da2c588116a
Closes-Bug: #1744010

Change-Id: I673c70ad2c0465babd7a39bf3d5fb5fb8252f1e0

https://github.com/openstack/trove/blob/master/trove/common/cfg.py#L85

Change-Id: I51d08d5a57453bb8a8459daea73a17b5e800f45f

through the database_address has beed defined in groups_vars/all.yml, we should
better use it, this way, if we want to use external database, we just need to
redefined in all.yml

refer to https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L83


Co-Authored-By: chenqiaomin <chen.qiaomin@99cloud.net>

Change-Id: Ie559301451954e16347ceaabf02f594c5c5cbe56

By default Trove is looking for "RegionOne", if the region is
different the os_region_name parameter needs to be defined withing
the Trove configuration files.

To solve this issue, we need to set "os_region_name" option in
trove-api, trove-taskmanager and trove-conductor configuration.

os_region_name = {{ openstack_region_name }}

Change-Id: I1397046d2c88ba50d01a65c48e021d3535fe39d2
Closes-bug: #1743402

Presently the taskmanager failed during a creation
of a trove cluster.

During the network ip checks, it didn't match the network.

The idea is to configure it to match all network name.
Finally this configuration is the same as for trove.conf.

Closes-bug: #1743395
Change-Id: I9284501424e6daa7d33d1590994bf231de71edd9

Change-Id: I329307b226757144a493ff21c76125cae8468e91

Add check enable_cinder to iscsi and tgtd deploy logic.

The ironic check is not effected.

Change-Id: If14ae52fbacf106b112e40a813ad6143cb095edc
Closes-Bug: #1742873

Implements: blueprint cephfs
Change-Id: I5bf14a668a9e8976a18aa28a5bbfe586690bbd31

When using server.basePath a dynamic bundle optimization will be started.
This job needs write access to /usr/share/kibana/optimize/bundles.

Change-Id: Ieaae79a5940f3972a8067814a1113ef635970875

Following deployment of Grafana, a check is made to ensure that the
login page is accessible. If the Grafana container has been deployed
to a remote host, but is not directly accessible over the deployment
network then the check will fail because the task is set to run with
connection: local. For example, if Ansible is using a different
network to the network on which grafana is exposed to connect to the
remote host then the task may fail. A similar problem exists for the
remaining two tasks in the post_config file.

Further to this, if Grafana is not accessible, the ansible uri
module does not return a status field, so no retry attempts are made
and the task fails with an obscure message:

TASK [grafana : Wait for grafana application ready] *****************
fatal: [some_remote_node]: FAILED! => {"failed": true, "msg": "The
conditional check 'result.status == 200' failed. The error was: error
while evaluating conditional (result.status == 200): 'dict object' has
no attribute 'status'"}

$ ansible --version
ansible 2.3.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.5 (default, Aug 4 2017, 00:39:18) [GCC 4.8.5 20150623
(Red Hat 4.8.5-16)]

Change-Id: I77e2129b4a1c5f2f0802fb92eb03395ff72cf5f3
Closes-Bug: #1742501

Designate communicate on publicURL (Default) endpoint with Neutron.
When TLS is enabled it generates errors (Missing certificate). We
need to ensure designate communicate with neutron on internalURL.

Change-Id: Ie969fc2d1a2d4241371b459af2fc5b7bdf236bf0
Closes-Bug: #1742625

Change-Id: I419cd02aeb899d93bba5f330d7e4535afd8c0ed8
Closes-bug: #1708655

So it turns out that without 'mode http' redirects and rewrites
will not happen, and we're relying on it for multiple things.

Switch neutron to use http-tunnel mode instead which seems to work,
we've had no errors in our setup.

See:
https://www.haproxy.org/download/1.5/doc/configuration.txt

Quoute:
" - tunnel        : only the first request and response are processed,
                      everything else is forwarded with no analysis."

Fixes: bba80acc
Change-Id: I0d9abe9731fba1e4deb64932e859f991648bb1ec

Provide support fot kolla dev mode in Keystone. When
'kolla_dev_mode' or 'keystone_dev_mode' variables are
enabled, source code of Keystone project is cloned
and bindmounted.

Partially implements: blueprint mount-sources

Change-Id: Ie4cf401ecd9a507e739a53dfdf16f65292ab57e5

Enable docker-ce on AArch64 platform in Debian.
Set the docker daemon name according to the existing of "dockerd" binary.

Closes-Bug: #1734255

Co-authored-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>

Change-Id: Ie0b3bac72b408dccf3fb99912b4fe2d4788705e5
Signed-off-by: Tone.Zhang <tone.zhang@arm.com>

Before cleanup containers and hosts, it should be ensure the
docker serivce running but the destroy action missing it,
this patch to fix it

Change-Id: I85ca806c1fd395daf937b585d466e16fbad4e20c
Closes-Bug: #1741852

Depends-On: I5846b48d336583ae82743f61bfbdadc99c755e8d
Change-Id: Iaf15dad5e4d527b8721409e56fc081043c1cb174
Implements: blueprint vitrage-roles

Current debian stretch use mariadb 10.1.26 which integrates
a backup tool call 'MariaDB Backup' [1]. It is based on
Percona XtraBackup and support full backup capability for
MariaDB Server that includes encrypted and compressed data.

This patch also fixs muti-node deployment failed on Debian
aarch64. Percona's repo has no XtraBackup package for Debian
aarch64. In such case we can use MariaDB builtin backup tool
'MariaDB Backup'.

[1] https://mariadb.com/kb/en/library/mariadb-backup-overview/

Change-Id: I7271d3f93b41d4839670a2c4a358744333411cd7

This will copy both of yaml or json policy file
if it exists

Change-Id: Ie5e03e0307d239f1079d2522e353367f7f954e77
Implements: blueprint support-custom-policy-yaml
Co-authored-By: Duong Ha-Quang <duonghq@vn.fujitsu.com>

Fixes a bug where the Baremetal Introspection service's public endpoint
registered in the Identity service referenced the internal API endpoint.

Also updates keystone endpoints for the Baremetal and Baremetal
Introspection services during reconfigure and upgrade operations.
Previously this was only done during deploy.

Change-Id: I32d475f288bb4a3834c13cc86f0c53b5437c3d25
Closes-Bug: #1738418

Provide support fot kolla dev mode in Magnum. When
'kolla_dev_mode' or 'magnum_dev_mode' variables are
enabled, source code of Magnum project is cloned
and bindmounted.

Partially implements: blueprint mount-sources

Change-Id: Id479bf4dc77ee4a2367c9c7d8f0f00de761b8c85

Instance failed to spawn: libvirtError: unsupported configuration: CPU
mode 'host-model' for aarch64 kvm domain on aarch64 host is not
supported by hypervisor.

Change-Id: Iad530457aef24ee8f561a8f7d2c6c6150c55bc42

Change-Id: Ie6e1d30d2fcbdd7c138c05585f24931e8495fc28

show_multiple_locations is missing for external ceph, and it will
cause tempest run failed.
it means the conditon for show_multiple_locations is
glance_backend_ceph but not enable_ceph, this patch it fix it.

Change-Id: I3c95c3b0a7e34639b376bdfd0205f3930b06e2cd
Closes-Bug: #1741022
Co-Authored-By: chenqiaomin <chen.qiaomin@99cloud.net>

Prevent cinder conf from copy nfs_shares to all nodes instead of
just cinder-volume.

Change-Id: I4f94b6c0ca8b628c1babee7fff0b66fb5a8df257
Closes-Bug: #1713010

Using Docker volumes for persistent data in the bifrost container is
necessary for upgrading the bifrost services.

This change adds the following volumes:

* bifrost_httpboot
* bifrost_ironic
* bifrost_mariadb
* bifrost_rabbitmq
* bifrost_tftpboot

Ironic creates hardlinks between the TFTP master image store and the
HTTP root path when iPXE is enabled. With different Docker volumes
used for these locations we run into
https://bugs.launchpad.net/ironic/+bug/1507894 during deployment. If
we use a directory under /httpboot to store the master images this
issue is avoided.

Change-Id: I8653268d3598e7a59d2eb45c8750d45b6fc9e35f
Partially-implements: blueprint bifrost-upgrade

Using a kolla_logs volume ensures that logs are persistent beyond the
lifespan of the bifrost_deploy container. This is necessary to support
upgrading the bifrost services. It also opens the possibility of log
forwarding, although we do not currently deploy a fluentd container on
the bifrost host.

Change-Id: If3118cd75ccc87b0d003d5f382695aeaa43ed0f8
Partially-implements: blueprint bifrost-upgrade

Kolla-ansible typically configures services to access the internal API
endpoint of other services, rather than the default public endpoint.

This change ensures that this is the case for ironic inspector.

Change-Id: I998f12435fc1bd306444f9a68bd7f99f5b78f6f8
Closes-Bug: #1740591

Previously ironic inspector logs were not being collected, as they are
stored in /var/log/kolla/ironic-inspector/ rather than
/var/log/kolla/ironic/. This change modifies the fluentd input
configuration to use the correct directory.

Also adds the ironic-inspector service to the fluentd programname
rewrite rule.

Change-Id: Ie5d67f395e4c32fef992e674f2d53dbd6a17f238