When the internal VIP is moved in the event of a failure of the active
controller, OpenStack services can become unresponsive as they try to
talk with MariaDB using connections from the SQLAlchemy pool.

It has been argued that OpenStack doesn't really need to use connection
pooling with MariaDB [1]. This commit reduces the use of connection
pooling via two configuration options:

- max_pool_size is set to 1 to allow only a single connection in the
  pool (it is not possible to disable connection pooling entirely via
  oslo.db, and max_pool_size = 0 means unlimited pool size)
- lower connection_recycle_time from the default of one hour to 10
  seconds, which means the single connection in the pool will be
  recreated regularly

These settings have shown better reactivity of the system in the event
of a failover.

[1] http://lists.openstack.org/pipermail/openstack-dev/2015-April/061808.html

Change-Id: Ib6a62d4428db9b95569314084090472870417f3d
Closes-Bug: #1896635

The use of default(omit) is for module parameters, not templates. We
define a default value for openstack_cacert, so it should never be
undefined anyway.

Change-Id: Idfa73097ca168c76559dc4f3aa8bb30b7113ab28

Include a reference to the globally configured Certificate Authority to
all services. Services use the CA to verify HTTPs connections.

Change-Id: I38da931cdd7ff46cce1994763b5c713652b096cc
Partially-Implements: blueprint support-trusted-ca-certificate-file

Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689



Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

This change defaults freezer to use mariadb as default backend for database
and adds elasticsearch as an optional backend due to the requirement of
freezer to use elasticsearch version 2.3.0. The default elasticsearch in
kolla-ansible is 5.6.x and that doesn't work with freezer.

Added needed options to the elasticsearch backend like:
 - protocol
 - address
 - port
 - number of replicas

Change-Id: I88616c285bdb297fd1f738846ddffe1b08a7a827
Signed-off-by: Marek Svensson <marek@marex.st>

We're duplicating code to build the keystone URLs in nearly every
config, where we've already done it in group_vars. Replace the
redundancy with a variable that does the same thing.

Change-Id: I207d77870e2535c1cdcbc5eaf704f0448ac85a7a

By default the driver used is elasticsearch in version 2
This change updates the driver with the good one.
It also updates backend with name used in the documentation.

Change-Id: I80f3020cb42903ae48ef65f52f67aae977c5a56b

Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.

[1]https://review.openstack.org/#/c/508522/



Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ifd8527d404f1df807ae8196eac2b3849911ddc26
Closes-Bug: #1761907

If SSL is enabled, api of multiple services returns
wrong external URL without https prefix.

Removal of condition for deletion of http  header.

Change-Id: I4264e04d0d6b9a3e11ef7dd7add6c5e166cf9fb4
Closes-Bug: #1749155
Closes-Bug: #1717491

Change-Id: I469800239bfc7f3bb5acc899edf7a971b792655c
Implements: blueprint kolla-freezer-scheduler-container
Co-Authored-By: wu.chunyang <wu.chunyang@99cloud.net>

- Freezer
- Gnocchi
- Kuryr
- Murano
- Panko

This will copy only yaml or json policy file if they exist.

Change-Id: I5450839cb06c515f2be445883421f8f987ca834d
Implements: blueprint support-custom-policy-yaml
Co-authored-By: Duong Ha-Quang <duonghq@vn.fujitsu.com>

storage settings for elasticsearch are not correct.
Need a backend and expecific options for the backend.

Change-Id: I71e35644389b6f97a20daf0f1dec29a30c3c34e3
Closes-Bug: #1713693

As described here:
https://github.com/openstack/keystone/blob/master/keystone/resource/core.py#L841
https://github.com/openstack/keystone/blob/master/keystone/conf/identity.py#L21

* default project domain name MUST be named 'Default'
* default project domain id MUST be named 'default'
* default project user name MUST be named 'Default'
* default project user id MUST be named 'default'

Change-Id: I610a0416647fdea31bb04889364da5395d8c8d74

Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455

Freezer doesn't use notification

Change-Id: If28e6940310096221c0f645995e96f7d34dd4f3e

freezer dont't user rabbitmq

https://github.com/openstack/freezer-api/blob/master/etc/freezer/freezer-api.conf.sample

Change-Id: I90a76275ab256ba8fc1c6d332e67af29fcbc22ef

Change-Id: I14ba906cb688c935351ef296badccf720b0267f0
Closes-Bug: #1665195

Co-Authored-By: caoyuan <cao.yuan@99cloud.net>
Co-Authored-By: zhubingbing <zhubingbing10@gmail.com>

Change-Id: I419f7ceb219ea9643cfd225c130018f967ddd860
Partially-implements: bp freezer-ansible-role

* Add api_workers option in murano group.
* engine workers moved from workers to engine_workers option in engine
  group.

Change-Id: I746a4e3c69acfd809e167e14a30cc8ed6b0512fb
Closes-Bug: #1638793

According to installation guide from murano[1] and devstack sample config[2],
some of the configurations for keystone auth need to be changed.

[1]: http://docs.openstack.org/developer/murano/install/manual.html#install-the-api-service-and-engine
[2]: https://github.com/openstack/murano/blob/master/devstack/plugin.sh#L187-L194

Change-Id: I01f9fcf3d6eb849392c4717f8e4c78be069351a2
Closes-bug: #1624957
Closes-bug: #1621693

Change below parameters name right <-- wrong

identity_uri <-- auth_url
admin_tenant_name <-- project_name
admin_user <-- username
admin_password <-- passowrd

http://docs.openstack.org/developer/murano/install/manual.html

Change-Id: I9a687567597c3d9b0c04db98d048bfedff4f9438
Closes-Bug: #1620379

rabbit_hosts, rabbit_userid and rabbit_password are deprecated for
removal.[0]

rpc_backend is deprecated for removal.[1]

rabbit_ha_queues is deprecated. it is useless when using RabbitMQ >= 3.0
[2]

[0] https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/impl_rabbit.py#L112,#L134
[1] https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/transport.py#L46
[2] https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/impl_rabbit.py#L163,L174

Closes-Bug: #1614082
Change-Id: I05d318ba6c11c5dbfa9fbf67d088a43ab465be30

Notification driver should be configured to avoid timeout failures of
murano app deployments while waiting notifications which will never be
sent.

The required driver is "messagingv2".

TrivialFix
Change-Id: Id0c753f50d93c81eedb2455a7323d86c08873c5f

Use a lower number of workers rather than the default value, which is
equal to the number of the cpu. Otherwise, in a multi cpu environment,
the number of the processes will very high.

In this PS, we use min(5, << number of cpu >>) as the default worker
count.

Closes-Bug: #1582254
Change-Id: I1c32cf0db794b43b8fb8be18f39190422ca5846f

Make sure that all the sevices will attempt to
connect to the database an infinite about of times.
If the database ever disappears for some reason we
want the services to try and reconnect more than just
10 times.

Closes-bug: #1505636
Change-Id: I77abbf72ce5bfd68faa451bb9a72bd2544963f4b

The in-process cache for keystone tokens has been deprecated due to
"incosistent results and high memory usage" with the expectation we
switch to memcached_servers if we want to stay performant.

Add memcache_servers [cache] section to the appropriate servers as the
[DEFAULT]\memcache_servers options was deprecated.

TrivialFix
Related-Id: Ied2b88c8cefe5655a88d0c2f334de04e588fa75a

Change-Id: Ic971bdddc0be3338b15924f7cc0f97d4a3ad2440

Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.

This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.

kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn

The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.

Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla

Partially implements: blueprint heka
Change-Id: I17fc4b838d6ba8b6fcfc5c08314fef5fac1c7aff

To allow for TLS to protect the service endpoints, the protocol
in the URLs for the endpoints will be either http or https.

This patch removes the hardcoded values of http and replaces them
with variables that can be adjusted accordingly in future patches.

Change-Id: Ibca6f8aac09c65115d1ac9957410e7f81ac7671e
Partially-implements: blueprint ssl-kolla

Config file use *_logging_debug as debug default value

Change-Id: I41102fff9056a82f7307694252adff0aedcf2658

TrivialFix
https://review.openstack.org/#/c/249508/

Change-Id: Ie2732ee116b6b4b3e0210eb1f62fc72878d8c277

Option "verbose" from group "DEFAULT" is deprecated for removal.
Its value may be silently ignored in the future.

If this option is not set explicitly, there is no such warning.
Furthermore, the default value of verbose is true, so there is
no need to set this value in config files.

TrivialFix

Change-Id: I3ec2a8900c984a64bc0645672ef89a63975f7f4e

TrivialFix

Change-Id: I33924d6de43126ff8523883eebce703c976f9a1a

Closes-Bug: #1528432
Change-Id: I6e56f283521b29678964cb655ac3cff2d13b8246

In heterogeneous environment, api_interfaces are different each other.
So we should specify it from hostvars.

Implements: bp configure-network-interface
Change-Id: Id15d70bfb9ebb62a64a3847a6b77407efb171dbe

Adjust all the configs to list all the rabbitmq hosts rather than
running rabbitmq through the VIP. This is made possible by clusterer
which has already merged.

Change-Id: I5db48f5f10ec68f4c8863a29bc13984f6845a4f9
Partially-Implements: blueprint rabbitmq-clusterer

Configure murano to use rsyslog container as logging mechanism

Change-Id: I98416f4dac7176539576e865c249adca04379251
Implements: bp logging-container

Deploy the core services for murano-api and murano-engine containers.

Implements: bp ansible-murano

Change-Id: Ibcc618665a3509465ba8f9249a417e8848087eef