For other architectures, fluentd is not installed using the
td-agent package.

Change-Id: I508bf16a54dcadac4b5cba03e184b4f620a2b395
Depends-On: https://review.openstack.org/638692

The path /var/lib/docker/volumes/kolla_logs/_data/ is too long
shorter log path will help to debug from log.
The volume path is compatible with docker-engine and docker-ce.

Change-Id: I9195d5f24d938f5060fe748aac3ae58c79ec5abf

Vitrage Collector service has been removed from Vitrage in change:
Ie713456b2df96e24d0b15d2362a666162bfb4300.

Change-Id: I45023940c1d2573bfed49d4ce3fac16ed2d559e4
Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>
Co-Authored-By: Kien Nguyen <kiennt65@viettel.com.vn>

With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers

OpenDaylight logs have different format than openstack,
is a karaf log with java error traces.
This PS add required config to make fluentd parse properly
ODL logs.

Change-Id: I34fb96c8a424679b3b618f2ff6a840b8dc165bec

This commit adds some filters which format logs so that they
can be correctly sent to the Monasca Log API by the Monasca
Fluentd plugin. In the future the Fluentd plugin could be
extended and this config could be removed.

Partially-Implements: blueprint monasca-roles
Change-Id: I87b6dfb3052d03f87349d30b66078c39d625195d

A small number of services set the recurse flag when they create
their config directory. This can change permission of files within
the directory, which are later set back to the original state. The
side effect is that the service is then restarted, even though the
net change to the config files amounts to nothing. The expected
behaviour is that a service only restarts if the config *has*
changed. This patch fixes this issue.

Change-Id: Ib6f1ca7b416247f8d455fb25892f4a3b27de03ba
Closes-Bug: 1800480

Change-Id: I0cbc51eedb218f6b2f7d5c50372ad7a8be9bdcc0

When Monasca is enabled disable direct logging to ElasticSearch and
send all logs harvested by Fluentd to the Monasca Log API.

This change also cleans up output files which may be left behind when
the various log forwarding options are enabled / disabled.

Partially-Implements: blueprint monasca-roles
Change-Id: I7197966c5117176407d60c86c08d3bcea5e8131a

Storm is required for running the Monasca thresholder component for
generating alerts.

Change-Id: I5e1ef74dc55a787293abbb3e629b5ab1ce5f4bbb
Partially-Implements: blueprint monasca-roles

Co-authored-by: Mark Goddard <mark@stackhpc.com>
Change-Id: I2e5ecf5b01cc842ec480fc4d883a7d2283fc1c31

This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa

include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated



Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a

There are cases when we can lost original timestamp field given from
logs, like when we send our logs to the next fluentd forwarder in chain
of forwarders, it will rewrite our timestamp by default. Save
`Timestamp` field explicitly to avoid such situation and be able to
reconstruct messages date and time.

Closes-Bug: #1781046
Change-Id: I2b4486aedacbe16dc4c0fb2e4e4984bd80e59f2d

It is not always convenient to use the the given
admin project and admin user; especially when some clouds
use different user and project for there keystone 'admin'

This allows setting the variables for these users to something
else, and defaults them to there current values of 'admin'.

Change-Id: I22b79a30f01c90a92ecc0974886edf3791518f2f

Change-Id: Ib2ca736e08f48da88bb74feb5cd1efce3b860ab7
Partially-Implements: blueprint ansible-check-mode

Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become

In common role, when check common container for cron restart,
we should use cron_container.changed instead of
check_common_containers.changed

Change-Id: Ifaaa437f5e205e970371fbb3660b68df352eb440
Closes-Bug: #1773923

Deploys the Monasca Log API with mod_wsgi + Apache.

Change-Id: I28f0aa31c59b0b6917be2b125b5f8a0d7a7035af
Partially-Implements: blueprint monasca-roles

Deploys the Monasca API with mod_wsgi + Apache.

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Partially-Implements: blueprint monasca-roles
Change-Id: I3e03762217fbef1fb0cbff6239abb109cbec226b

Change-Id: I47b329bb3e16aa33a5e0a156aca515a666cd98e4

- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0

Since the kolla-kubernetes has been separated, and nowhere to
use kubetoolbox, it is safe to remove it from kolla-ansible.

Change-Id: I9aa59d45fa04c13944120f1f99af2361551147ad

Change-Id: Iedf87342eeee90d58734ba21216296f3d94f76c8
TrivialFix

Change-Id: Idb429823d48c4f40cf391151e8bb7478cc7d3631

Projects which may use Kafka include Monasca and Oslo
messaging. Initially it will be possible to
deploy a single Kafka cluster which may be shared
among clients. Support for running multiple Kafka
clusters may be added in a future change. This
change also configures logging for Kafka server,
state-change and controller logs.

Partially-Implements: blueprint monasca-roles
Change-Id: Iab8d200c2450238f3c0c708d1f4184490f6e6284

As rest of the files are follow the same pattern
So in same manner following the same for fluentd format config

Change-Id: I114e3a09e9747085adcc06caa419d5ebe61cec7b

Fluentd custom output and filter configs target directory can have unknown
permissions, so copy these configs as root.

Change-Id: I1649f8cd4a5066245e8f79e8e71b6bbf04dbcb62
Closes-Bug: #1760213

This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799

This picks up the abandoned review:
https://review.openstack.org/#/c/412423



Co-Authored-By: Sam Yaple <sam@yaple.net>
Partially-Implements: blueprint monasca-roles
Change-Id: Idc01afcb125271181ee5e15d327d0929f07b49e9

Add ansible role to deploy blazar
Add nova filters to allow use of blazar

Change-Id: I6742ddc9a4736f256491dd0cfd31904fa8eb5652
Implements: blueprint blazar-ansible-role

Previously ironic inspector logs were not being collected, as they are
stored in /var/log/kolla/ironic-inspector/ rather than
/var/log/kolla/ironic/. This change modifies the fluentd input
configuration to use the correct directory.

Also adds the ironic-inspector service to the fluentd programname
rewrite rule.

Change-Id: Ie5d67f395e4c32fef992e674f2d53dbd6a17f238

Co-Authored-By: caoyuan <cao.yuan@99cloud.net>
Co-Authored-By: Eduardo Gonzalez <dabarren@gmail.com>

Depends-on I7ffb991adbd05275e331401cd73fc84362084ebd
Change-Id: I85afeb5dfce0bf6350a444000bdb885c6fe079c9
Partially-implements: blueprint vitrage-roles

Although there is an option "enable_fluentd", but there are still
some tasks that make fluentd not configurable.

Change-Id: Ic0a9deb36fce154022925d26411e01a8ffe18b50
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>

The owner and permission check for config directories
should be executed only when the service is enabled.

Change-Id: I6dd2a8fcce83cba8a9a79b1c302d6b1ef4305144
Closes-Bug: #1734789

In some scenarios it may be useful to apply custom filters to logs
before forwarding them.  This may be useful to add additional tags to
the messages or to modify the tags to conform to a log format that
differs from the one defined by kolla-ansible.

Configuration of custom fluentd filters is possible by placing filter
configuration files in ``/etc/kolla/config/fluentd/filter/*.conf`` on
the ansible control host.

Change-Id: I29a20efb0df4bf3564ef009616c786e928aa26d9
Implements: blueprint fluentd-custom-filters

- Sets the hostname on rabbitmq and mariadb log messages
  according to the local host.
- Sets the hostname on syslog log messages by extracting it
  from the syslog message. In many cases this will be the
  same as the local host, but it may also be a remote host.

Closes-Bug: #1726596
Change-Id: If7c53916d5a355b3aac33d6a62d89d02474e8b53

The current syntax of Ubuntu fluentd rewrite rules are not supported
anymore, reference to this commit[1].
According to the build of this patch[2], The Centos has no such isuue.
Only Ubuntu needs to be upgrade to use the <rule> section.

* Centos use 01-rewrite-0.12.conf.j2
* Ubuntu use 01-rewrite-0.14.conf.j2

backport: pike

[1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter/commit/248ed8e97def0a60121199c0d0e2fa8785f0d076
[2] https://review.openstack.org/#/c/517907

Depends-on: I47985113fe732569d640a262ca832c6edc8c2bb6
Change-Id: I0b131c2f65652cf3c61b33d8162097b047603923
Closes-Bug: #1730664

Become option is missed in patch #398682, this patchset fix this issue.

Change-Id: I228c31486210ef1de76601d3ea4ee955a628ca73
Partial-Implements: blueprint ansible-specific-task-become

By default the syslog input plugin won't extract the
log level from a syslog message. By setting the priority
key, the log level is extracted.

Partial-Bug: #1725240
Change-Id: I9ad5f5e1c875ef62b39fbbba7b271d98cee68dd6