Currently there are a few services that perform host configuration
tasks. This is done in config.yml. This means that these changes are
performed during 'kolla-ansible genconfig', when we might expect not to
be making any changes to the remote system.

This change separates out these host configuration tasks into a
config-host.yml file, which is included directly from deploy.yml.

One change in behaviour is that this prevents these tasks from running
during an upgrade or genconfig. This is probably what we want, but we
should be careful when any of these host configuration tasks are
changed, to ensure they are applied during an upgrade if necessary.

Change-Id: I001defc75d1f1e6caa9b1e11246abc6ce17c775b
Closes-Bug: #1860161

One way to improve the performance of Ansible is through fact caching.
Rather than gather facts in every play, we can configure Ansible to
cache them in a persistent store. An example Ansible configuration for
doing this is as follows:

[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./facts
fact_caching_timeout = 86400

This does not affect Kolla Ansible however, since we use the setup
module which unconditionally gathers facts regardless of the state of
the cache. This gets worse with large inventories limited to a small
batch of hosts via --limit or serial, since the limited hosts must
gather facts for all others.

One way to detect whether facts exist for a host is via the
'module_setup' variable, which exists only when facts exist. This change
uses the 'module_setup' fact to determine whether facts need to be
gathered for hosts outside of the batch. For hosts in the batch, we
switch from using the setup module to gather_facts on the play, which
can use the 'smart' gathering logic.

Change-Id: I04841fb62b2e1d9e97ce4b75ce3a7349b9c74036
Partially-Implements: blueprint performance-improvements

In [1] only neutron-openvswitch-agent was fixed and not xenapi.
That merged in Ussuri and went cleanly into Train.
In Stein and Rocky, the backport was not clean and
accidentally fixed xenapi instead of the regular one.

Neither the original bug nor its incomplete fix were released,
except for Rocky. :-(
Hence this patch also removes the confusing reno instead of
adding a new one.

[1] https://review.opendev.org/713129

Change-Id: I331417c8d61ba6f180bcafa943be697418326645
Closes-bug: #1869832
Related-bug: #1867506

Not everyone wants Kafka data stored on a Docker volume. This
change allows a user to flexibly control where the data is stored.

Change-Id: I2ba8c7a85c7bf2564f954a43c6e6dbb3257fe902

This patch fix creating statck resource failure in heat.

Change-Id: I00c23f8b89765e266d045cc463ce4d863d0d6089
Closes-Bug: #1869137

Change-Id: I9395ae32378f4ff1fd57be78d7daec7745579e04
Closes-Bug: #1869133

Deploy HAProxy on one or more servers. Add another server to the
inventory in the haproxy group, and run the following:

kolla-ansible prechecks --limit <new host>

The following task will fail:

    TASK [haproxy : Checking if kolla_internal_vip_address and
    kolla_external_vip_address are not pingable from any node]

This happens because ansible does not execute on hosts where
haproxy/keepalived is running, and therefore does not know that the VIP
should be active.

This change skips VIP prechecks when not all HAProxy hosts are in the
play.

Closes-Bug: #1868986

Change-Id: Ifbc73806b768f76f803ab01c115a9e5c2e2492ac

The 'kolla-ansible stop' command can be used to stop the services
running on hosts. However, if you run this command in an environment
with heterogeneous nodes (most real world scenarios have at least
control/compute), then it fails. This is because it only checks
whether a container is enabled, and not whether the host is in the
correct group. For example, it fails with nova-libvirt:

    No such container: nova_libvirt to stop.

This change fixes the issue by only attempting to stop containers on
hosts to which they are mapped.

Change-Id: Ibecac60d1417269bbe25a280996ca9de6e6d018f
Closes-Bug: #1868596

This is useful to people who manage their Prometheus Server
externally to Kolla Ansible, or want to use the exporters with
another framework such as Monasca.

Change-Id: Ie3f61e2e186c8e77e21a7b53d2bd7d2a27eee18e

Fluentd cannot accept empty 'path' parameter.

I refactored the service list following the general pattern
we have.

Change-Id: I83d820efcc7e86bac9f8bda26a8f8bece72159e6
Closes-bug: #1867953

ovs-ofctl is still being run by neutron-openvswitch-agent.
Potential removal is scheduled for Victoria.
Until then, we have to mount /run/openvswitch in there.

Change-Id: Ia73b5665cece523bb822f6a223335f6fae94fb6a
Closes-bug: #1867506

Fix elasticsearch schema in fluentd when kolla_enable_tls_internal is
true.

Change-Id: I51286d2def7a762d569740c1abc5b924b682ad9d
Closes-Bug: #1867481

Change-Id: I29f65c83b9bd45e463d868cf9a55611f33fe3177
Closes-bug: #1867179

grafana not support ipv6 in grafana.ini.j2.

Closes-Bug: #1866141

Change-Id: Ia89a9283e70c10a624f25108b487528dbb370ee4
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>

This should help to ensure that users are running tested and supported
host OS distributions.

Change-Id: I6ee76463d284ad4f3646af1c7ec2b7e50e2f3b15
Partially-Implements: blueprint improve-prechecks

If haproxy is running somewhere in the cluster and listening on the VIP,
but not running locally, then the following precheck may fail:

   TASK [haproxy : Checking free port for HAProxy monitor (vip interface)]

   msg: Timeout when waiting for 192.0.2.10:61313 to stop.

This change fixes the issue by skipping the check if HAProxy is running
on any host.

Change-Id: I831eb2f700ef3fcf65b7e08382c3b4fcc4ce8d8d
Closes-Bug: #1866617

We already only include .conf files in fluent.conf:

(fluentd)[fluentd@cpu-e-1041 /etc/fluentd]$ cat fluent.conf
@include input/*.conf
@include filter/*.conf
@include format/*.conf
@include output/*.conf

so this change should not cause ill effect. This works because of the
merge option in config files:

merge: merges the source directory into the target directory instead of
replacing it. Boolean, defaults to false.

see https://docs.openstack.org/kolla/latest/admin/kolla_api.html#kolla-api-external-config

Change-Id: I28f63ec81f1ea5bc4a213d053bfb2c04388d5925
Closes-Bug: #1862211

Closes-Bug: #1866727
Change-Id: I455ef6026b39110791cd38dac053205550af1ce2

The variable enable_cadf_notifications is deprecated and marked
for removal during the U cycle.

Change-Id: I5e4d20d112db2392b55a0788f4d704ab6ca6112f

Change-Id: I2257dedb21f335666695648bd4bbd725e7b082b3

to backport before releases

Change-Id: I3ed8b56232e9fa99f7a0bb262d57a2d13cb18999

Change-Id: I68a40bebc174e8ebdaea36a0689b34cadb9009d2
Closes-bug: #1865840

The logrotate rotation interval and count are not configurable.
Currently, the configuration is a "default" that keeps 6 weeks of logs.

Change-Id: I4f55ee2a98f7861cb8de2724f5edc32da6d2f9ee

Change-Id: I1439aa923a9129b752abb4acfb88559930b2178a

Closes-Bug: #1864856
Change-Id: I725eeb18a22b3fa7838f16761d19f7e699ab5e82

Service REST API urls should be constructed using the
{{ internal_protocol }} and {{ external_protocol }} configuration
parameters.

Change-Id: Id1e8098cf59f66aa35b371149fdb4b517fa4c908
Closes-Bug: 1862817

Backport to: Train.

Change-Id: Ide96ea43739d47e623026f0aecd4163f3a2abe7f
Closes-bug: #1864615

Reno was missing in change https://review.opendev.org/#/c/708114/

Change-Id: I909ddf1f33634e4fe1fd05931eee69b12d57778a

Service configuration urls should be constructed using
kolla_internal_fqdn instead of kolla_internal_vip_address. Otherwise SSL
validation will fail when certificates are issued using domain names.

Change-Id: I21689e22870c2f6206e37c60a3c33e19140f77ff
Closes-Bug: 1862419

The kibana, elasticsearch and monasca roles all use the uri module to
perform Elasticsearch configuration tasks via its API. The body of the
request should be JSON formatted, but these tasks now fail because it is
not.

The following error is seen:

TASK [monasca : Create default control plane organisation if it doesn't
exist]

invalid character '\\'' looking for beginning of object key string

The 'JSON' body in this case was:

{'name': 'monasca_control_plane@default'}

This was probably caused by the recent change to execute these tasks in
the kolla_toolbox container, but may also be caused by an Ansible
version bump (or something else).

This change fixes the issue by ensuring that the body is JSON-encoded in
all cases.

Change-Id: I7acc097381dd9a4af4e014525c1c88213abbde93
Closes-Bug: #1864177

Change-Id: I9aa211ceefe7ad3524323be837ec090969f94557

Per http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012646.html

Deprecates support for deploying with Hyper-V integrations.
In Victoria support for these will be removed from Kolla Ansible.

This is dictated by lack of interest and maintenance.

Change-Id: Ic214699e0e501868e453c76929eef740e92ab90f

Currently we have a very wide /run mount for all Neutron/OVS services,
which allows sudo/rootwrap to contact with the hosts dbus - all symptoms
are documented in the related bug.

Since we use tcp connections to OVS from Neutron agents - removing
bind mounts.

Closes-Bug: #1861792

Change-Id: Ifee4bec7b2e9ef4e2d624b1411f1a9e6332325c6

This daemon is an additional piece of functionality supported by Gnocchi
and the general pattern in KA is to disable such things unless the user
explicitly wants them. This also helps avoid having to set the
resource_id, user_id, and project_id variables for Gnocchi if you don't
care about this daemon.

Change-Id: I5f14cee4b0bb0d781b1ff53200d11de972d20c82

This allows you to tune the performance of InfluxDB by locating the
volume on a drive that is separate to the default docker storage.

Change-Id: Iea555a2702b225b30f5d7035b8a703d4f3376ee7

Change-Id: I26206bece95d31c0182e75f2a585c50d6f0fad6f

Make it require uniqueness of resolution as well to avoid later
issues with RabbitMQ going crazy.

Change-Id: I000ba6c62ab44eac0abdf8d5d1f069adfbc6552f
Closes-bug: #1863363

It looks like the only missing part was the actual mount of
/lib/modules

Now Cinder Backup volumes differ from Cinder Volume volumes only
by /etc/target which is not relevant (Cinder Backup does not
provide a target).

Change-Id: Iccf4298c4f9306eb0a95b6712815778555ef44fc
Closes-bug: #1863094

Change-Id: I4f553bd0888e200ddf744604c5029e67a95ee2cd
Closes-bug: #1863041

line from /etc/hosts

Ubuntu always uses 127.0.1.1 for that with some tricky sauce
around `hostname` depending on whether it contains '.' or not.
And when I mean `hostname` it's the one returned by `hostname`
command with no arguments.

ansible_hostname is always a single word so we can match on that.

I did not want to remove just any 127.0.1.1 in case someone
is using it for other purposes. :-)

Change-Id: I8bd3d42a5e3bd0f63336ed60a0af90d52b1650d6
Closes-bug: #1862739