This avoids the need to use a proxy, or some other means, to connect to
Prometheus. This is disabled by default and can be enabled by setting
enable_prometheus_server_external to true.

Change-Id: Ia0af044ff436c2a204b357750a16ff49fcdfec45

When running in check mode, some prechecks previously failed because
they use the command module which is silently not run in check mode.
Other prechecks were not running correctly in check mode due to e.g.
looking for a string in empty command output or not querying which
containers are running.

This change fixes these issues.

Closes-Bug: #2002657
Change-Id: I5219cb42c48d5444943a2d48106dc338aa08fa7c

Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/


in which was suggested to split patch into smaller ones.

This change adds container_engine variable to kolla_container_facts
module, this prepares module to be used with docker and podman as well
without further changes in roles.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I9e8fa30646844ab4a288555f3aafdda345b3a118

A follow up patch to I563f2438b3cb0895606b029b5269ce2e50c413e3

Change-Id: I1e4c5db46413668d4b5df2f2dcedc5d9aaecd63a

This can be used to forward Prometheus Alertmanager notifications to
Microsoft Teams.

Change-Id: I563f2438b3cb0895606b029b5269ce2e50c413e3
Depends-On: https://review.opendev.org/c/openstack/kolla/+/812678

Add support for deploying the Kolla Prometheus libvirt exporter image to
facilitate gathering metrics from the Nova libvirt service.

Co-Authored-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: Ib27e60c39297b86ae674297370f9543ab08cda05
Partially-Implements: blueprint libvirt-exporter

This is useful to people who manage their Prometheus Server
externally to Kolla Ansible, or want to use the exporters with
another framework such as Monasca.

Change-Id: Ie3f61e2e186c8e77e21a7b53d2bd7d2a27eee18e

We assume that all groups are present in the inventory, and quite obtuse
errors can result if any are not.

This change adds a precheck that checks for the presence of all expected
groups in the inventory for each service. It also introduces a common
service-precheck role that we can use for other common prechecks.

Change-Id: Ia0af1e7df4fff7f07cd6530e5b017db8fba530b3
Partially-Implements: blueprint improve-prechecks

Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689



Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

This commit follows up the work in Kolla to provide deploy and configure the
Prometheus blackbox exporter.

An example blackbox-exporter module has been added (disabled by default)
called os_endpoint. This allows for the probing of endpoints over HTTP
and HTTPS. This can be used to monitor that OpenStack endpoints return a status
code of either 200 or 300, and the word 'versions' in the payload.

This change introduces a new variable `prometheus_blackbox_exporter_endpoints`.
Currently no defaults are specified because the configuration is heavily
dependent on the deployment.

Co-authored-by: Jack Heskett <Jack.Heskett@gresearch.co.uk>
Change-Id: I36ad4961078d90e2fd70c9a3368f5157d6fd89cd

Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496

This patch implements the support for the elasticsearch-exporter in
kolla-ansible

The configuration and prechecks are reused from the other exporters

Depends-On: Id138f12e10102a6dd2cd8d84f2cc47aa29af3972
Change-Id: Iae0eac0179089f159804490bf71f1cf2c38dde54

This patch implements the initial support for the
openstack-exporter[0] in the kolla-ansible
prometheus monitoring system.

The configuration and prechecks are reused from the other
exporters and a new template is provided for generating
a os-client-config file required by the exporter.

The default scrape interval is 60 seconds, but it can
be extended via a configuration option.

[0] https://github.com/Linaro/openstack-exporter

Change-Id: I4a34c4bb56e74b5cd544972cbd6540d9acb6e4a1

Add missing `prometheus_memcached_exporter` container_fact
Fix conditional container_fact for haproxy_exporter

Change-Id: Id0f3b94af956f51e3c782c0244c6ce7a340119bd
Closes-Bug: #1808820

This patch extends the prometheus role for being able
to deploy the prometheus-alertmanager[0] container.

The variable enable_prometheus_alertmanager
decides if the container should be deployed and enabled.

If enabled, the following configuration and actions are performed:

- The alerting section on the prometheus-server configuration
is added pointing the prometheus-alertmanager host group as targets.

- HAProxy is configured to load-balance over the prometheus-alertmanager
host group. (external/internal).

Please note that a default (dummy) configuration is provided, that
allows the service to start, the operator should extend it via a node custom config

[0] https://github.com/openstack/kolla/tree/master/docker/prometheus/prometheus-alertmanager



Change-Id: I3a13342c67744a278cc8d52900a913c3ccc452ae
Closes-Bug: 1774725
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@linaro.org>

cAdvisor (Container Advisor) provides metrics on resource usage and
performance characteristics of running containers.  This change
deploys a cadvisor container and configures prometheus to scrape
data from it.

Change-Id: I55dd4fee954f9be68efda397746861ddaaa0a565
Partially-Implements: blueprint prometheus

This patch adds the prometheus_memcached_exporter[0] to the
list of  available exporters, following the conventions
used by the previously integrated exporters.

[0] https://github.com/openstack/kolla/tree/master/docker/prometheus-memcached-exporter



Change-Id: I103b0ee19ef2fd17ce19a27d60773675ad234c1c
Closes-Bug: #1773303
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@linaro.org>

the prometheus container is not exits, it should be
prometheus_server[0]

[0]: https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/prometheus/defaults/main.yml#L6

Change-Id: Ib44390af9b8af5156dafbd0b0da6ae061a926ec7

This patch adds the ansible role to deploy the prometheus service which
can be used to collect performance metrics accross the environment

Partially-Implements: blueprint prometheus
Change-Id: I908b9c9dad63ab5c9b80be1e3a80a4fc8191cb9e