We previously used ElasticSearch Curator for managing log
retention. Now that we have moved to OpenSearch, we can use
the Index State Management (ISM) plugin which is bundled with
OpenSearch.

This change adds support for automating the configuration of
the ISM plugin via the OpenSearch API. By default, it has
similar behaviour to the previous ElasticSearch Curator
default policy.

Closes-Bug: #2047037

Change-Id: I5c6d938f2bc380f1575ee4f16fe17c6dca37dcba

This change replaces ElasticSearch with OpenSearch, and Kibana
with OpenSearch Dashboards. It migrates the data from ElasticSearch
to OpenSearch upon upgrade.

No TLS support is in this patch (will be a followup).

A replacement for ElasticSearch Curator will be added as a followup.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/830373



Co-authored-by: Doug Szumski <doug@stackhpc.com>
Co-authored-by: Kyle Dean <kyle@stackhpc.com>
Change-Id: Iab10ce7ea5d5f21a40b1f99b28e3290b7e9ce895

This makes it possible for services to fetch the Elasticsearch endpoint
from Keystone. It is useful for both operators and Monasca Tempest.

Change-Id: Id60298582496a8959e82b970676669ca17e2e9d4

Config plays do not need to check containers. This avoids skipping
tasks during the genconfig action.

Ironic and Glance rolling upgrades are handled specially.

Swift and Bifrost do not use the handlers at all.

Partially-Implements: blueprint performance-improvements
Change-Id: I140bf71d62e8f0932c96270d1f08940a5ba4542a

Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. For unconditionally included tasks, switching to
import_tasks provides a clear benefit.

Benchmarking of include vs. import is available at [1].

This change switches from include_tasks to import_tasks where there is
no condition applied to the include.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md#task-include-and-import

Partially-Implements: blueprint performance-improvements

Change-Id: Ia45af4a198e422773d9f009c7f7b2e32ce9e3b97

Currently there are a few services that perform host configuration
tasks. This is done in config.yml. This means that these changes are
performed during 'kolla-ansible genconfig', when we might expect not to
be making any changes to the remote system.

This change separates out these host configuration tasks into a
config-host.yml file, which is included directly from deploy.yml.

One change in behaviour is that this prevents these tasks from running
during an upgrade or genconfig. This is probably what we want, but we
should be careful when any of these host configuration tasks are
changed, to ensure they are applied during an upgrade if necessary.

Change-Id: I001defc75d1f1e6caa9b1e11246abc6ce17c775b
Closes-Bug: #1860161

include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated



Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a

Co-Authored-By: caoyuan <cao.yuan@99cloud.net>
Change-Id: Ica989c7f70810d2cf98c0920ec264a311473e5b1
Partially-implements: blueprint better-reconfigure

Part of ELK stack. Includes Dockerfiles for both Centos and Ubuntu.

Change-Id: I9f76adf084cd4f68e29326112b76ffd02b5adada
Partially-implements: blueprint central-logging-service

Change-Id: I3de01770eca6b7e681b7964a6d6e15564065aaea
Partially-Implements: blueprint pre-pull-images

Throughout the project overtime some of these file permissions have
changed to have an executable bit. They should not have this bit set.

TrivialFix

Change-Id: I1748b5bde813a0fcac36aeecdfd83245b8ee5be3

This implements all the openstack pieces needed to make ceph work.

DocImpact
Change-Id: I1d24476a966602cf955e5ef872b0efb01319894a
Partially-Implements: blueprint ceph-container
Implements: blueprint kolla-live-migration

Ansible installation of rsyslog container

Change-Id: I17a7ee679f72e8fe667012d7eb6e07a27098a909
Implements: bp logging-container

Currently we require a slew of deps on each destination node, this
includes a gcc compiler and installing things via pip. We can remove
these dependencies by containerizing them and running and Ansible
inside the container itself. The container would then report back
facts about idempotency.

DocImpact
Closes-Bug: #1481495
Implements: blueprint containerize-dependencies
Change-Id: I3dfccbf9fafc06ffc36e78f3006fe5d3367891df

This commit consists HAProxy ansible bits including config generation,
container deployment and hot reloads.

Closes-Bug: #1477915
Co-Authored-By: Sam Yaple <sam@yaple.net>
Change-Id: Ie93fa68fdb6b2885889c992ff1267d38b68e0cbc
Partially-implements: blueprint ansible-service

Adds initial support for Rabbitmq in Ansible using the CONFIG_EXTERNAL methods.

Additionally, this refactors some of the Rabbitmq config script to allow for
reuse by CONFIG_EXTERNAL.

Partially Implements: blueprint ansible-service

Change-Id: I1765548f7e4f1258eb8a49e2a23242955f52655d

Implements the initial structure for variable substitution in ansible.

Change-Id: I26d82189273be6ebf941b0ab82efc6bf1eebc53d

This ansible module for docker-compose allows for idempotency. I have
submitted a pull request upstream to ansible-modules-extra to include
this new module. When/if the module is accepted upstream if can be
removed from the local module library.

The two playbooks have been updated to use this module. The database
data container does not support idempotency due to the fact that it
exists instead of sleeps. Therefore each time `docker-compose up` is
called, it will start the container and register a change. The
message-broker does not have this issue and will remain unchanged even
repeatedly running these playbooks.

Due to the use of a special branch of docker-compose provided by sdake,
this module requires at least docker-compose==1.2.0rc1

Change-Id: If1644eaa3bff0c2a007fa2d479a95bea941945f6

This adds a very basic structure to begin supporting ansible in Kolla.
Ansible can support complete idempotency, but wrapping docker-compose
does not allow idempotency at this time. These playbooks will be
extended to compensate for that in future patches.

Change-Id: I1c9e8d32321e2733f5a9b752edf74b4fe90317ed