Add global_request_id according to oslo.log defaults [1],
also support lines like:

2024-12-23 20:48:29.905 2 ERROR oslo_service.periodic_task ['Traceback (most recent call last):\\n'

[1]: https://docs.openstack.org/oslo.log/latest/configuration/#DEFAULT.logging_context_format_string

Closes-Bug: #2044370

Change-Id: I80c3e20de0b7503de6331b55879e85892323b3d6
(cherry picked from commit 5a3fb2b4)

When merging change [1], fluentd_enable_watch_timer was
unintentionally missed in Let’s Encrypt, as change [2] had been
merged earlier.

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/785309
[2] https://review.opendev.org/c/openstack/kolla-ansible/+/899895

Change-Id: I7c72faecbdb66c7fd196acd3e7b2351851983490

By default, the watch timer in Fluentd is set to True.
To save CPU and I/O consumption this can be set to False, which
kolla-ansible has been hardcoding so far.

When the watch timer is disabled, in_tail relies entirely
on inotify. In certain constellations, this may not work
reliably. In these cases, the watch timer needs to be activated, so this
change adds a variable to make the setting configurable.

Change-Id: Ic8ce6fbc3ed8f31d5d090e114b35703532679729

Also stop using the old config name (td-agent.conf)

Change-Id: Ied2736b891cd8c6dfcc509a8fd6b1fc8bfe21cb4

Change-Id: Ia6db7d6a41ddbda8fcbf563dc55a0c65ef8db9be

Depends-On: https://review.opendev.org/c/openstack/kolla/+/901508
Change-Id: I8c7d3de95d0f1f8e57a993b8c3417d90459e19be

Adds configurations and changes tasks to enable the
systemd plugin. Additionaly, the plugin is set to
read logs from the /var/log/journal directory

Implements: enable-fluent-plugin-systemd
Signed-off-by: Juan Pablo Suazo <jsuazo@whitestack.com>
Change-Id: Ic714a341befa5f906d9c0f78fa86f4c934df87cd

Depends-On: https://review.opendev.org/c/openstack/kolla/+/899773
Change-Id: I016c7a54ee2c43b6bbad0cb3eb3fe216f3d66238

Related-Bug: #2015184

Closes-Bug: #2015184

Change-Id: I6ea0949d58e38e2cda41c9702001e7bd500f23f1

The default value of logging_context_format_string and
logging_user_identity_format has changed [1][2][3].
This is to correct fluentd to use the same format.

[1]https://opendev.org/openstack/oslo.log/commit/ebdee7f39920ad5b4268ee296952432b0d41a375
[2]https://opendev.org/openstack/oslo.log/commit/7ed065f38df0595881d07eb91cec9e5c07cb38be

[3]https://review.opendev.org/c/openstack/oslo.log/+/838185

Closes-Bug: #1872220

Change-Id: Ide0079ef4d28e3a3fa60288a74b936531c52ce60

Their cleanup has been added to monasca cleanup command.

Change-Id: I19a846e2683ae70b33ca64d2aba7ac71eb724588

Kolla Ansible is switching to OpenSearch and is dropping support for
deploying ElasticSearch. This is because the final OSS release of
ElasticSearch has exceeded its end of life.

Monasca is affected because it uses both Logstash and ElasticSearch.
Whilst it may continue to work with OpenSearch, Logstash remains an
issue.

In the absence of any renewed interest in the project, we remove
support for deploying it. This helps to reduce the complexity
of log processing configuration in Kolla Ansible, freeing up
development time.

Change-Id: I6fc7842bcda18e417a3fd21c11e28979a470f1cf

In fluentd v1.0, "format" became a deprecated parameter,
replaced by "<parse>"

Change-Id: I0825e81fcd90fbc7f64c8df3ea9ae199ad79294a

Adds a new configuration file that provides fluentd with an appropiate regex to match with OpenvSwitch logs in both default files.

The regex is segmented with variable as to isolate the relevant parts of each log message.

Closes-Bug: #1965815
Signed-off-by: Juan Pablo Suazo <jsuazo@whitestack.com>
Change-Id: Ife83c50c048d517a5c8a5dee588f8f7846fcee00

In Xena [1] we removed Monasca Grafana service, but some components were left
to support cleanup operations.

[1]: https://review.opendev.org/c/openstack/kolla-ansible/+/788228

Change-Id: Iccc7bc3628bb7cbab1ac28f41c7b7dc7695894c6

We no longer have to support multiple fluentd versions and rely
on the workaround.

Change-Id: Id4083a9f07317fc04467e52176850f3909703169

This change adds the dnsmasq.log for the ironic-dnsmasq container and
also enables more verbose logging when debug logging enabled.
This can be triggered globbaly via 'openstack_logging_debug' or per
service via 'ironic_logging_debug' or 'neutron_logging_debug'.

Change-Id: I0e6b089beb88827effbcc365625eb2df902f5470
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

This patch adds support for integrating Prometheus with Fluentd.
This can be used to extract useful information about the status
of Fluentd, such as output buffer capacity and logging rate,
and also to extract metrics from logs via custom Fluentd
configuration. More information can be found here in [1].

[1] https://docs.fluentd.org/monitoring-fluentd/monitoring-prometheus

Change-Id: I233d6dd744848ef1f1589a462dbf272ed0f3aaae

By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.

This change updates all references to Ansible facts within Kolla Ansible
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.

This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.

[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars

Change-Id: I7e9d5c9b8b9164d4aee3abb4e37c8f28d98ff5d1
Partially-Implements: blueprint performance-improvements

Currently the logs tagged with infra.mariadb.xinetd flow into
elasticsearch with no hostname or programname attach, thus making
navigating the logs very hard.

The quick fix is renaming the tag to infra.mariadb-xinetd, which is just
enought to ensure the logs are processed correctly with the existing
filters.

TrivialFix

Change-Id: Icd72206de7c1f701bdf35c8fb3b128ef2dbe29a8

Time format in Ruby Time.strptime is not accepting padding flags,
therefore we need to remove them for the Fluentd to be able
to parse MariaDB xinetd logs properly.

Change-Id: Iabfa9afdcad505106a5580eb2d058273ee5f7c1f
Closes-Bug: #1886002

The Monasca Log API has been removed and in this change we switch
to using the unified API. If dedicated log APIs are required then
this can be supported through configuration. Out of the box the
Monasca API is used for both logs and metrics which is envisaged to
work for most use cases.

In order to use the unified API for logs, we need to disable the
legacy Kafka client. We also rename the Monasca API config file
to remove a warning about using the old style name.

Depends-On: https://review.opendev.org/#/c/728638
Change-Id: I9b6bf5b6690f4b4b3445e7d15a40e45dd42d2e84

Depends-On: https://review.opendev.org/710217/

Change-Id: I85652f23e487c40192106d23f2cdd45a3077deca

Change-Id: I812665059783617d581d748e619b29426f89b353

Fluentd cannot accept empty 'path' parameter.

I refactored the service list following the general pattern
we have.

Change-Id: I83d820efcc7e86bac9f8bda26a8f8bece72159e6
Closes-bug: #1867953

WSGI log files use a different input configuration than OpenStack log
files. Currently this depends on log files matching either *-access.log
or *-error.log. Some services use *_access.log or *_error.log, so are
not parsed correctly.

This change modifies the fluentd configuration to accept an underscore
or hyphen for WSGI log file names.

Change-Id: I566d6cac0b6749054fd5422ec8f36f99dacb1db7
Closes-Bug: #1720371

Opendaylight support has been deprecated in Train - time to remove it.

Change-Id: I3a61bfbcbf366c327ea3e25d2424bc3fedca29f0

Change-Id: I49b24545501085d5a44f4de73f0c6dd21e06e2a0
Closes-Bug: #1835501

MariaDB logs contain two different log message formats, one output
from mysqld and one from mysqld_safe. This patch splits the message
formats by tag and parses them separately.

Change-Id: I58857be67ae387eeda7487811a6af85b0f95970c
Closes-Bug: #1845629

In order to orchestrate smooth transition to fluentd 0.14.x
aka 1.0 stable branch aka td-agent 3
from td-agent repository - use image labels (fluentd_version
and fluentd_binary).

Depends-On: https://review.opendev.org/676411
Change-Id: Iab8518c34ef876056c6abcdb5f2e9fc9f1f7dbdd

Masakari provides Instances High Availability Service for
OpenStack clouds by automatically recovering failed Instances.

Depends-On: https://review.openstack.org/#/c/615469/


Change-Id: I0b3457232ee86576022cff64eb2e227ff9bbf0aa
Implements: blueprint ansible-masakari
Co-Authored-By: Gaëtan Trellu <gaetan.trellu@incloudus.com>

Change-Id: I6d205fe327f198e699519ebe9d589b9ee77a62d2
Closes-Bug: #1837274
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Monasca Python service logs prior to this change were being dropped
due to missing entries in the Fluent record_transformer config file.
This change adds support for ingesting those logs, and explicitly
removes support for ingesting Monasca Log API logs to reduce the risk
of feedback, for example if debug logging is turned on in the Monasca
Log API.

Change-Id: I9e3436a8f946873867900eed5ff0643d84584358

Presently, errors can appear in Fluentd and Monasca Log API logs due
to log output from some Monasca services, which do not use Oslo log,
being processed alongside other OpenStack logs which do.

This change parses these log files separately to prevent these errors.

Change-Id: Ie3cbb51424989b01727b5ebaaeba032767073462

If Blazar is enabled, ensure that fluentd processes its logs.

Change-Id: If71d5c056c042667388dae8e4ee6d51a5ecab46e

Qinling is an OpenStack project to provide "Function as a Service".
This project aims to provide a platform to support serverless functions.

Change-Id: I239a0130f8c8b061b531dab530d65172b0914d7c
Implements: blueprint ansible-qinling-support
Story: 2005760
Task: 33468

Because kolla-ansible not have cyborg so should add it.

Implements: blueprint add-cyborg-to-kolla-ansible

Depend-On: I497e67e3a754fccfd2ef5a82f13ccfaf890a6fcd

Change-Id: I6f7ae86f855c5c64697607356d0ff3161f91b239

For other architectures, fluentd is not installed using the
td-agent package.

Change-Id: I508bf16a54dcadac4b5cba03e184b4f620a2b395
Depends-On: https://review.openstack.org/638692

OpenDaylight logs have different format than openstack,
is a karaf log with java error traces.
This PS add required config to make fluentd parse properly
ODL logs.

Change-Id: I34fb96c8a424679b3b618f2ff6a840b8dc165bec

There are cases when we can lost original timestamp field given from
logs, like when we send our logs to the next fluentd forwarder in chain
of forwarders, it will rewrite our timestamp by default. Save
`Timestamp` field explicitly to avoid such situation and be able to
reconstruct messages date and time.

Closes-Bug: #1781046
Change-Id: I2b4486aedacbe16dc4c0fb2e4e4984bd80e59f2d