This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121

- Heat
- Ironic
- Magum
- Manila
- Mistral

This will copy only yaml or json policy file if they exist.

Change-Id: I1ab71e2758dc99dd6654d433ece79600f0c44ce8
Implements: blueprint support-custom-policy-yaml
Co-authored-By: Duong Ha-Quang <duonghq@vn.fujitsu.com>

through the database_address has beed defined in groups_vars/all.yml, we should
better use it, this way, if we want to use external database, we just need to
redefined in all.yml

refer to https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L83


Co-Authored-By: chenqiaomin <chen.qiaomin@99cloud.net>

Change-Id: Ie559301451954e16347ceaabf02f594c5c5cbe56

This commit separates the messaging rpc and notify transports in order
to support separate and different oslo.messaging backends

This patch:
* add rpc and notify variables
* update service role conf templates
* add example to globals.yaml
* add release note

Implements: blueprint hybrid-messaging
Change-Id: I34691c2895c8563f1f322f0850ecff98d11b5185

Add become to only neccesary tasks in roles:
- glance
- heat
- horizon
- keystone
- neutron
- nova
- openvswitch

Gate is also updated to use 'become' feature

Change-Id: I2f3f27306e9f384148e1ad4d54d8da2ebef34d00
Partial-Implements: blueprint ansible-specific-task-become

When deploying with tls enabled in public
endpoints, ansible modules fails due SSL certificates
are self-signed.

This change adds a new variable to allow customization
on which endpoints ansible should connect.
Defaults to admin because admin auth parameters defaults
to admin endpoint.

Change-Id: Ic3ed58cf9c9579cae08a11bbfe6fce983b5a9cbc
Closes-Bug: #1720995

Actually Openstack services configuration can be overriden using many
files:
- /etc/kolla/config/<< service name >>/<< config file >>
- /etc/kolla/config/<< service name >>/<<host>>/<< config file >>
- /etc/kolla/config/global.conf
- /etc/kolla/config/database.conf
- /etc/kolla/config/messaging.conf

Only per-service configuration is actually documented here:
https://github.com/openstack/kolla-ansible/blob/master/doc/advanced-configuration.rst#L164

Allowing to globally modify service configuration can be perform too,
but it can be done in 3 different manners, all not documented:
- /etc/kolla/config/global.conf
- /etc/kolla/config/database.conf
- /etc/kolla/config/messaging.conf

database.conf and messaging.conf seems redundant with global.conf.
In order to simplify codebase it seems logical to remove them.

Documentation has been added for overriding configuration globally and
release note has been added too.

Closes-Bug: #1682479
Change-Id: I5d922dfc0d938173bad34ac64e490b78db1b7e31

Change-Id: I366c2124a778c5b0b22a8fe369d603adad75ebbe

kolla-kubernetes is using its own configuration generation[0], so it is
time for kolla-ansible to remove the related code to simplify the
logical.

[0] https://github.com/openstack/kolla-kubernetes/tree/master/ansible

Change-Id: I7bb0b7fe3b8eea906613e936d5e9d19f4f2e80bb
Implements: blueprint clean-k8s-config

As described here:
https://github.com/openstack/keystone/blob/master/keystone/resource/core.py#L841
https://github.com/openstack/keystone/blob/master/keystone/conf/identity.py#L21

* default project domain name MUST be named 'Default'
* default project domain id MUST be named 'default'
* default project user name MUST be named 'Default'
* default project user id MUST be named 'default'

Change-Id: I610a0416647fdea31bb04889364da5395d8c8d74

In order to speed up deployment time some "local" actions should be run
only once using 'run_once: True'.
This will decrease deployment time in case of multihost configuration.

Change-Id: I6015d772d35c15e96c52f577013b6e41197cb41a

when create database user, it should use
database_user, not database_name.

Change-Id: I4dfa01d1a5a46c5c58f1fc47b0be71b186462764
Closes-Bug: #1698762

Ansible task support vars directive, no need implement another one in
merge_config. This patch remove the vars directive in merge_config
action plugin.

Change-Id: I33648a2b6e39b4d49ce76eb66fbf2522721f8c68

Add a new variable 'kolla_devmode', which when enabled, clones and
bindmounts service source code into the containers.

This commit adds the relevant changes for Heat, more services can be
added and built upon.

Usage:
* Set 'kolla_devmode: yes'

* Code is cloned to /opt/stack/{{ project_name }} on target
  node(s)

* Users can develop in these repos, and simply restart the container to
  pick up / test changes.

Debugging can be done from the host via 'remote_pdb'[0].

[0] https://pypi.python.org/pypi/remote-pdb

Implements: blueprint mount-sources
Change-Id: Ic0431b10d723bf84eeefc72039376fe0058dd902

OSprofile allows user/devs trace OpenStack requests.

Implements: blueprint enable-osprofiler
Co-Authored-By: Bertrand Lallau <bertrand.lallau@gmail.com>
Change-Id: I82ea85d726011ef6cbf99380f395452d6d7f8053

In case Kolla's users want to deploy with both of
binary and source image, we should have a variable
install type that define install type for each project.

We also add specific image tag for each Openstack project.

This commit is implemented for Designate, Gnocchi and Heat projects.

Change-Id: I07451750e70e0e6305dca451422e33cd31ce8a4c
Implements: blueprint mixing-binary-and-source-image

Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455

The Orchestration service automatically assigns the heat_stack_user
role to users that it creates during stack deployment.
To avoid conflicts, do not add this role to users with the heat_stack_owner role.

Closes-Bug: #1690975

Change-Id: I7a4cb7f4a13de7be4fc9ce9c24057ece6a0ced5c

Heat-api-cfn need to point to keystone v3 version.
Otherwise heat fail while authenticating for scaling policies.

``AWS authentication failure.``

Change-Id: I1950cd7359d8ad589feced870de76f02ef2c8a76
Closes-Bug: #1672431

Wait conditions are commonly used by instances to signal various events
back to heat. These instances are unlikely to have access to the
internal API endpoints. OpenStack-Ansible had a similar issue[1] back in
juno and changed to use the public endpoint[2]. The code has now moved
but the default is still in place[3].

This change configures heat to advertise the public API as the endpoint
for wait conditions.

[1] https://bugs.launchpad.net/openstack-ansible/+bug/1459414
[2] https://review.openstack.org/#/c/186221/
[3] https://github.com/openstack/openstack-ansible-os_heat/blob/b1721a7460ba816afabd9bded6e5c79c635bca3a/defaults/main.yml#L48

Change-Id: Id1d66aaa298efa8407db579a899a5aacebe1e6c7
Closes-Bug: #1688331

wait_for module waits 300 seconds for the port started or stopped.  This
is meaningless and useless in precheck. This patch change timeout to 1
seconds.

Change-Id: I9b251ec4ba17ce446655917e8ef5e152ef947298
Closes-Bug: #1688152

Heat can send RPC notifications to Ceilometer as define here:
https://github.com/openstack/ceilometer/blob/master/ceilometer/pipeline/data/event_definitions.yaml#L139

oslo_messaging_notifications section MUST be managed in heat.conf file.

Closes-Bug: #1674935
Change-Id: I314e34a7a7b6caab72b916331f5f20c29afde438

While in openstack deployment guides use Orchestration as
heat-cfn description, heat devstack plugin uses CloudFormation.
I think is more accurate and easier to know which service is.

Change-Id: I760c7e7baa46da57fd2fca9d5409f370a0c9065c
Closes-Bug: #1673421

Change-Id: I5c04712b1ae9ebe36b5c57eed05c34fec09f1fd8
Closes-Bug: #1671784

Add a new subcommand 'check' to kolla-ansible, used to run the
smoke/sanity checks.

Add stub files to all services that don't currently have checks.

Change-Id: I9f661c5fc51fd5b9b266f23f6c524884613dee48
Partially-implements: blueprint sanity-check-container

heat-api kept redirecting clients to use http:// instead of https://
when communicating with our https:// only loadbalancer

Please examine the logic for enabling it carefully, it's hard to know
if it should be enabled or not, potenitially it could be a security
risk.

Based on openstack-ansible-os_heat:
commit 4033a0f854cba6719c61812ef5b553e932a6c6c2
Author: Kyle L. Henderson <kyleh@us.ibm.com>

    Enable oslo_middleware proxy header parsing

"Heat has moved to using oslo_middleware for the http proxy header
parsing, however the default is to not parse the headers.  When
the external protocol differs from the internal protocol this
parsing is required in order for heat to work properly since it
will return 302 redirects to the client during some operations
(such as delete stack).

An example of this is when using haproxy with https configured
for the external protocol and http for the internal protocol.
If the oslo_middleware does not parse the headers, then any
302 redirects would specify a url with http rather than
correctly specifying https and the heat client would fail to
connect on the redirect url."

Change-Id: I38661a0bc2163a7f72febd98b7ae6f51c5d45ad5

- add "item.value.enabled | bool"
- add "| bool" to keystone
- add group check for searchlight

Change-Id: Id4555a0a96ea1670e99c88a1da9d3e07bf253497

Change-Id: I1577cc3afef4dadd3a188c8ba749c9cdfad313ae

Usernames can be configured with variables in
configuration files, but user creation is hardcoded.

Change-Id: I057cfb921d776217db66f59226dcfa79f3eb7368
Closes-Bug: #1661587

Co-Authored-By: Mauricio Lima <mauriciolimab@gmail.com>
Change-Id: I9a4a6b6523dee4b388513386b7d85d421f2b7b89

Change-Id: Icf34357d222cf5eb41e504c3b5e6830c1c6509c5
Closes-Bug: 1659343

A config generation check was added to a few services but the action
name checked was "genconfig" where the kolla-ansible genconfig command
actually uses the action name "config".

Stop run the handlers when action is "config".

Co-Authored-By: Jeffrey Zhang <jeffrey.zhang@99cloud.net>
Partially-implements: blueprint better-reconfigure
Change-Id: I9d3be2f674087f340108b176c8e8e2209ffa8806

Change-Id: I0e8b16bba5c826a97a4e9bd07891f5f3fb549334
Partially-implements: blueprint better-reconfigure

Change-Id: I00d2dcb0895548ba169ab85764cf546c2214cbf5

Making variable name "works" to "workers" for correct configuration
as followed in other services

Closes-Bug: #1655081

Change-Id: I333b7a7a98770e640db49e8103900957c629bad5

Change-Id: I7de1322ae1efe3797ed5be8ebe2f7359142b077f
Partially-implements: blueprint condition-pre-check

Include custom policy.json files in service-api.json.j2 files

Change-Id: Ic55bfc6f61131aa72c3497ce8b2282056bcc7f92
Partially-Implements: blueprint custom-policies

Currently, policy.json is put in
"{{ node_config_directory }}/{{ service_name }}"
in target nodes.

Relocation policy.json to "{{ node_config_directory }}/{{ item }}"
with item is corresponding service compoment config directory.

Currently, the policy.json is copied to all services, but it
should be reviewed and left only in neccesary service
(at many cases, only API service needs that).

Redundant files will be removed in follow up patchset.

Change-Id: I0e997dccf4ec438c9c0436db71ec2fd06650f50d
Closes-Bug: #1639686

* Merge prechecks.yml and site.yml playbook
* Create empty precheck.yml into all roles.

Change-Id: I8a138558a26c0a2a66c5fd48ed37be657c99c1dd
Implements: blueprint condition-pre-check

Allow operators to use their custom policy files.
Avoid maintain policy files in kolla repos, only copying
the files when an operator add their custom config.

Implements: blueprint custom-policies
Change-Id: Icf3c961b87cbc7a1f1dd2ffbfffcf271d151d862