Change-Id: Ief84e093829677c97c8df9a08aefca43b1e51aac

This change allows a user to forward control plane logs
directly to Elasticsearch from Fluentd, rather than via
the Monasca Log API when Monasca is enabled. The Monasca
Log API can continue to handle tenant logs.

For many use cases this is simpler, reduces resource
consumption and helps to decouple control plane logging
services from tenant logging services.

It may not always be desired, so is optional and off by
default.

Change-Id: I195e8e4b73ca8f573737355908eb30a3ef13b0d6

The Monasca alerting pipeline provides multi-tenancy alerts and
notifications. It runs as an Apache Storm topology and generally
places a significant memory and CPU burden on monitoring hosts,
particularly when there are lot of metrics. This is fine if the
alerting service is in use, but sometimes it is not. For example
you may use Prometheus for monitoring the control plane, and
wish to offer tenants a monitoring service via Monasca without
alerting and notification functionality. In this case it makes
sense to disable this part of the Monasca pipeline and this patch
adds support for that.

If the service is ever re-enabled, all alerts and notifications
should spawn back automatically since they are persisted in the
central mysql database cluster.

Change-Id: I84aa04125c621712f805f41c8efbc92c8e156db9

The Log Metrics service is an admin only service. We now have
support in Fluentd via the Prometheus plugin to create metrics
from logs. These metrics can be scraped into Monasca or Prometheus.
It therefore makes sense to deprecate this service, starting by
disabling it by default, and then removing it in the Xena release.
This should improve the stability of the Monasca metrics pipeline
by ensuring that all metrics pass via the Monasca API for
validation, and ensure that metrics generated from logs are
available to both Prometheus and Monasca users by default.

Change-Id: I704feb4434c1eece3eb00c19dc5f934fd4bc27b4

Historically Monasca Log Transformer has been for log
standardisation and processing. For example, logs from different
sources may use slightly different error levels such as WARN, 5,
or WARNING. Monasca Log Transformer is a place where these could
be 'squashed' into a single error level to simplify log searches
based on labels such as these.

However, in Kolla Ansible, we do this processing in Fluentd so
that the simpler Fluentd -> Elastic -> Kibana pipeline also
benefits. This helps to avoid spreading out log parsing
configuration over many services, with the Fluentd Monasca output
plugin being yet another potential place for processing (which
should be avoided). It therefore makes sense to remove this
service entirely, and squash any existing configuration which
can't be moved to Fluentd into the Log Perister service. I.e.
by removing this pipeline, we don't loose any functionality,
we encourage log processing to take place in Fluentd, or at least
outside of Monasca, and we make significant gains in efficiency
by removing a topic from Kafka which contains a copy of all logs
in transit.

Finally, users forwarding logs from outside the control plane,
eg. from tenant instances, should be encouraged to process the
logs at the point of sending using whichever framework they are
forwarding them with. This makes sense, because all Logstash
configuration in Monasca is only accessible by control plane
admins. A user can't typically do any processing inside Monasca,
with or without this change.

Change-Id: I65c76d0d1cd488725e4233b7e75a11d03866095c

Change-Id: I77791d504327ace880d0cc2438af2f8ced66d4eb

trivial fix
see: https://review.opendev.org/c/openstack/kolla-ansible/+/763191

Change-Id: I7f5a5ba5e9a6b3866fc4d2c72b7b4884c85020bd

This change enables the use of Docker healthchecks for magnum
services.
Implements: blueprint container-health-check

Change-Id: I14d862aa599915c781d02b71a0e57d2124de9abc

it was confused to customize opts in trove-conductor.conf
or trove-taskmanager.conf now.
if we want to customize a opts,The operator needs
to know which service is using the configuration opts.
actually trove uses trove.conf is enough for all services
this change combines  all trove config files.

Change-Id: I5a630109e3c4b59bff216146a3ed64c6d47e247f

Change-Id: If42b7324b58851a4f0beb3a80c70ca2c9dd729af

It currently runs on hosts in ovn group, and in case that controllers are not
network gateways - it fails on missing openvswitch.

Change-Id: Ibbf683872337402b4e2a38323bb6a3f35ee4bed4

Update the Monasca docs to improve security considerations.

Trivial-Fix
Change-Id: I97eb8441466f8c6abdbd66068257765bdbe32d4d

This reverts commit ff441c1c.

Since RabbitMQ TLS is still not functional in Victoria, it is
not enabled for upgrade jobs.

Change-Id: I575942c8d90441145de78dcb16a2b4c1f172773b

This pull request adds support for the OpenID Connect authentication
flow in Keystone and enables both ID and access token authentication
flows. The ID token configuration is designed to allow users to
authenticate via Horizon using an identity federation; whereas the
Access token is used to allow users to authenticate in the OpenStack CLI
using a federated user.

Without this PR, if one wants to configure OpenStack to use identity
federation, he/she needs to do a lot of configurations in the keystone,
Horizon, and register quite a good number of different parameters using
the CLI such as mappings, identity providers, federated protocols, and
so on. Therefore, with this PR, we propose a method for operators to
introduce/present the IdP's metadata to Kolla-ansible, and based on the
presented metadata, Kolla-ansible takes care of all of the
configurations to prepare OpenStack to work in a federated environment.

Implements: blueprint add-openid-support
Co-Authored-By: Jason Anderson <jasonanderson@uchicago.edu>
Change-Id: I0203a3470d7f8f2a54d5e126d947f540d93b8210

If kolla-ansible is installed via pip install --user, currently the
kolla-ansible script is unable to locate the installed playbooks.
This leads to a failure when running commands.

This change fixes the issue by checking for the user's .local directory
as a possible installation path.

This fixes some of the scenario tests which were failing after switching
to a user installation in Ifaf1948ed5d42eebaa62d7bad375bbfc12b134d5.
Most tests did not fail since the kolla-ansible script in the source
checkout was used.

Closes-Bug: #1915527

Change-Id: I5b47a146627d06bb3fe4a747c5f20290c726b0f9

Installing ceph-ansible in the virtualenv on CentOS 8 fails with:

    ModuleNotFoundError: No module named 'setuptools_rust'

This error appeared following the release of cryptography 3.4, which now
includes Rust code. It can be installed without Rust using a Python
wheel, but only with more recent pip than version 9.0.3 available as RPM
on CentOS 8. The cryptography bug report [1] recommends pip>=19.1.1.

This change upgrades pip in the virtualenv before installing
ceph-ansible.

[1] https://github.com/pyca/cryptography/issues/5753

Change-Id: I47473de6f71c422db2238d653c2d8f379c55e79b

- Increment retries: waiting 20 seconds (i.e., 10 retries) seem to
  be not enough for monasca-grafana to start on the first node.
  Increasing to 80 seconds (i.e., 40 retries) fixes the issue.

- Prevent the check from running when kolla_action=config. In that
  case, the command would never succeed as the service is not
  deployed yet (similarly to
  https://review.opendev.org/c/openstack/kolla-ansible/+/771237).

Closes-Bug: #1915060
Related-Bug: #1821285
Change-Id: I7b42c51a66caed0eccf118615d841dca97a7af9d

Installing kolla-ansible system-wide on CentOS 8 fails with:

    ModuleNotFoundError: No module named 'setuptools_rust'

This error appeared following the release of cryptography 3.4, which now
includes Rust code. It can be installed without Rust using a Python
wheel, but only with more recent pip than version 9.0.3 available as RPM
on CentOS 8. The cryptography bug report [1] recommends pip>=19.1.1.

This change switches to using pip --user when installing kolla-ansible.

Also fixes an issue with ansible-lint which was failing on
etc/kolla/globals.yml due to a missing space before comments.

[1] https://github.com/pyca/cryptography/issues/5753

Change-Id: Ifaf1948ed5d42eebaa62d7bad375bbfc12b134d5
Closes-Bug: #1915141

One of the renos was causing issues due to a duplicated id.
This change makes tox doc8 env lint renos and fixes
the offending reno.

Change-Id: Id3ae6e144b4261c97726cdec172ea9bef093de9e