Delegate executing uri REST methods to the current module containers
using kolla_toolbox. This will allow self signed certificate that are
already copied into the container to be automatically validated. This
circumvents requiring Kolla Ansible to explicitly disable certificate
validation in the ansible uri module.

Partially-Implements: blueprint custom-cacerts

Change-Id: I2625db7b8000af980e4745734c834c5d9292290b

When kolla_copy_ca_into_containers is set to "yes", the Certificate
Authority in /etc/kolla/certificates will be copied into service
containers to enable trust for that CA. This is especially useful when
the CA is self signed, and would not be trusted by default.

Partially-Implements: blueprint custom-cacerts

Change-Id: I4368f8994147580460ebe7533850cf63a419d0b4

As part of the effort to implement Ansible code linting in CI
(using ansible-lint) - we need to implement recommendations from
ansible-lint output [1].

One of them is to stop using local_action in favor of delegate_to -
to increase readability and and match the style of typical ansible
tasks.

[1]: https://review.opendev.org/694779/

Partially implements: blueprint ansible-lint

Change-Id: I46c259ddad5a6aaf9c7301e6c44cd8a1d5c457d3

Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689



Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Sometimes as cloud admins, we want to only update code that is running
in a cloud.  But we dont need to do anything else.  Make an action in
kolla-ansible that allows us to do that.

Change-Id: I904f595c69f7276e71692696471e32fd1f88e6e8
Implements: blueprint deploy-containers-action

Some tasks were improperly generalized in change:
I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
(simplify handlers)
This patch reverts that.

Change-Id: I38fbe319da50fc7d5a3fa3c4890f039ae900a60c
Closes-bug: #1845258
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Currently, we have a lot of logic for checking if a handler should run,
depending on whether config files have changed and whether the
container configuration has changed. As rm_work pointed out during
the recent haproxy refactor, these conditionals are typically
unnecessary - we can rely on Ansible's handler notification system
to only trigger handlers when they need to run. This removes a lot
of error prone code.

This patch removes conditional handler logic for all services. It is
important to ensure that we no longer trigger handlers when unnecessary,
because without these checks in place it will trigger a restart of the
containers.

Implements: blueprint simplify-handlers

Change-Id: I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd

Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496

20 seconds may be too short to wait for grafana ready, Let's keep the
check task 60 seconds.

backport: rocky

Change-Id: Ib219ad215d1ef2147ba3591f8c398feb4f3c8888
Closes-Bug: #1821285

Several config file permissions are incorrect on the host. In general,
files should be 0660, and directories and executables 0770.

Change-Id: Id276ac1864f280554e98b937f2845bb424d521de
Closes-Bug: #1821579

With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers

Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b

This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa

include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated



Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a

Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become

Previous change[1] of config for grafana introduced
new bugs.

[1]https://review.openstack.org/#/c/548291/

Change-Id: Ib62f0670003612073ff08f702febe81c753b72e5

- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0

Assuming both Prometheus and Grafana are enabled, this change
configures Grafana such that it can use data collected by the
Prometheus server.

Partially-Implements: blueprint prometheus
Change-Id: I0290806d0c622b1de33cf7047f099795b4c780b9

Add custom configuration for grafana

Change-Id: I03e3ab57ce538c0f86d1442c7bd30d3ce9012637

Add become to only neccesary tasks in roles:

- aodh
- barbican
- bifrost
- ceilometer
- ceph
- chrony
- cinder
- cloudkitty
- collectd
- congress
- designate
- elasticsearch
- etcd
- freezer
- gnocchi
- grafana
- influxdb
- ironic
- iscsi
- karbor
- kibana
- kuryr
- magnum
- manila
- mistral
- mongodb
- multipathd
- murano
- octavia
- panko
- qdrouterd
- rally
- sahara
- searchlight
- senlin
- skydive
- solum
- swift
- swift
- tacker
- telegraf
- tempest
- trove
- vmtp
- watcher
- zun

Change-Id: I6e32d94d4172dd96d09d8609e8a5221ab5586a31
Partial-Implements: blueprint ansible-specific-task-become

The grafana local admin username can be configured by overriding
the admin user field in the grafana.ini file. However, this will
fail when kolla-ansible attempts to configure any enabled
datasources for grafana because the local admin password is
hardcoded to 'admin'. This change allows the grafana local admin
password to be configured via group vars so that the correct
username is used when configuring datasources.

Closes-Bug: #1750408
Change-Id: I0962200894f7a0452da1c249a68f9230b6fab13f

If the wrong status code is returned, ansible sets failed: true.
This means that a retry is never attempted.

Use .get('status') to work around the possibility that a status
may not be returned in some corner cases. For example:

TASK [grafana : Wait for grafana application ready]
***************** fatal: [some_remote_node]: FAILED! =>
{"failed": true, "msg": "The conditional check 'result.status
== 200' failed. The error was: error while evaluating
conditional (result.status == 200): 'dict object' has no
attribute 'status'"}

Change-Id: I14207d1a9e994fa25c1c061025fff5b64a71d690
Closes-Bug: #1742501

This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121

Following deployment of Grafana, a check is made to ensure that the
login page is accessible. If the Grafana container has been deployed
to a remote host, but is not directly accessible over the deployment
network then the check will fail because the task is set to run with
connection: local. For example, if Ansible is using a different
network to the network on which grafana is exposed to connect to the
remote host then the task may fail. A similar problem exists for the
remaining two tasks in the post_config file.

Further to this, if Grafana is not accessible, the ansible uri
module does not return a status field, so no retry attempts are made
and the task fails with an obscure message:

TASK [grafana : Wait for grafana application ready] *****************
fatal: [some_remote_node]: FAILED! => {"failed": true, "msg": "The
conditional check 'result.status == 200' failed. The error was: error
while evaluating conditional (result.status == 200): 'dict object' has
no attribute 'status'"}

$ ansible --version
ansible 2.3.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.5 (default, Aug 4 2017, 00:39:18) [GCC 4.8.5 20150623
(Red Hat 4.8.5-16)]

Change-Id: I77e2129b4a1c5f2f0802fb92eb03395ff72cf5f3
Closes-Bug: #1742501

I think that we do not need this in a production-ready project.

Change-Id: Ief1392c5e8becc90034fca782211a1da30da14ae

Check grafana's /login url for status code 200 instead of root url since the
latter does not return an http 200 code

Change-Id: Ide7da26c0e217a4a60ad3d2ac693d6f7ac7bb722
Closes-Bug: #1705911

Replace grafana home dashboard with file in node_custom_config folder.

Change-Id: I00e3963ee287c526935e34169b63b1a40e6b4025

Make data source creation more dynamic to support a variable number of data
source of different types

Only create data sources actually deployed by kolla

Change-Id: I26a2d3c71b178562b906623ab77eaff673c81dce

when create database user, it should use
database_user, not database_name.

Change-Id: I4dfa01d1a5a46c5c58f1fc47b0be71b186462764
Closes-Bug: #1698762

Condition check are already performed here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/site.yml
In order to simplify codebase, these checks can be removed for
standalone services.

Change-Id: Ib9842cd5363a1c7e56234a5e91dc264f89e2838f

Ansible task support vars directive, no need implement another one in
merge_config. This patch remove the vars directive in merge_config
action plugin.

Change-Id: I33648a2b6e39b4d49ce76eb66fbf2522721f8c68

Sometimes the "Enable influxdb datasource" task failed with "503 Service
Unavailable" error. Waiting for grafana port UP seems not always enough,
it requires to wait for application ready.

Change-Id: I3a27b3086698b89a80a3f585beb810831289ae7a
Closes-Bug: #1697213

Recently added post_config for Grafana does some uri queries,
these can be problematic if run across every node.

Change-Id: I580212474879da18780fc5f66a21952a9e6f8daa

Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455

wait_for module waits 300 seconds for the port started or stopped.  This
is meaningless and useless in precheck. This patch change timeout to 1
seconds.

Change-Id: I9b251ec4ba17ce446655917e8ef5e152ef947298
Closes-Bug: #1688152

In order to automate Grafana configuration, this fix automatically
set the InfluxDB datasource. This avoid doing a annoying manual
configuration after Kolla-ansible deployment.

Change-Id: I2b1a63fd371966192f1df0a82cee4711c6324710

Add a new subcommand 'check' to kolla-ansible, used to run the
smoke/sanity checks.

Add stub files to all services that don't currently have checks.

Change-Id: I9f661c5fc51fd5b9b266f23f6c524884613dee48
Partially-implements: blueprint sanity-check-container

Kolla use kolla_toolbox container to interact
with the database.
Grafana fails to deploy if mysql-python is not installed
in the host.

Change-Id: I033f1828ff027532de9be94aaf7dee6dbeab8fb6
Closes-Bug: #1661646

Change-Id: I672d6fea935cbe7416ebd0cfff5006fddb9cd534
Partially-implements: blueprint better-reconfigure

Change-Id: I00d2dcb0895548ba169ab85764cf546c2214cbf5