This patch is moving minsize and maxsize logrotate
options to defaults and use them in template.

Change-Id: Ic2dfc747faedef8a7a4bc5e61db7c0cb5bf93e88

By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.

This change updates all references to Ansible facts within Kolla Ansible
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.

This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.

[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars

Change-Id: I7e9d5c9b8b9164d4aee3abb4e37c8f28d98ff5d1
Partially-Implements: blueprint performance-improvements

Remove rally role as planned

Change-Id: Ic898efe42b21b01c45d4621af2cf90ecd7afc398

the project is deprecated and in the process of being removed
from OpenStack upstream.

Change-Id: I9d5ebed293a5fb25f4cd7daa473df152440e8b50

Co-Authored-By: Bartosz Bezak <bartosz@stackhpc.com>

Depends-On: https://review.opendev.org/c/openstack/kolla/+/782906

Change-Id: I73cd776772a45c9ffcb045f4d0eff8f4bdc997f1

As announced on the openstack-discuss ML[1], Karbor is retiring
this cycle (Wallaby).

Needed-By: https://review.opendev.org/c/openstack/karbor/+/767032

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018643.html

Change-Id: I222cf302e507f6a9de0347c79ec536aa7be22bb6

Searchlight project is retiring in Wallaby cycle[1].
This commit removes the ansible roles of Searchlight project
before its code is removed.

Needed-By: https://review.opendev.org/c/openstack/searchlight/+/764526

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018637.html

Change-Id: I85aab66376ea4f1376c2705066ba3c7e5645644f

Qinling project is retiring in Wallaby cycle[1].
This commit removes the ansible roles of Qinling project
before its code is removed.

Needed-By: https://review.opendev.org/c/openstack/qinling/+/764521

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018638.html

Change-Id: I6543bacff638b1649511f7e779807954c34ef570

Add TLS support for backend Neutron API Server communication using
HAProxy to perform TLS termination. When used in conjunction with
enabling TLS for service API endpoints, network communication will be
encrypted end to end, from client through HAProxy to the Neutron
service.

Change-Id: Ib333a1f1bd12491df72a9e52d961161210e2d330
Partially-Implements: blueprint add-ssl-internal-network

Previously we mounted /etc/timezone if the kolla_base_distro is debian
or ubuntu. This would fail prechecks if debian or ubuntu images were
deployed on CentOS. While this is not a supported combination, for
correctness we should fix the condition to reference the host OS rather
than the container OS, since that is where the /etc/timezone file is
located.

Change-Id: Ifc252ae793e6974356fcdca810b373f362d24ba5
Closes-Bug: #1882553

The common role was previously added as a dependency to all other roles.
It would set a fact after running on a host to avoid running twice. This
had the nice effect that deploying any service would automatically pull
in the common services for that host. When using tags, any services with
matching tags would also run the common role. This could be both
surprising and sometimes useful.

When using Ansible at large scale, there is a penalty associated with
executing a task against a large number of hosts, even if it is skipped.
The common role introduces some overhead, just in determining that it
has already run.

This change extracts the common role into a separate play, and removes
the dependency on it from all other roles. New groups have been added
for cron, fluentd, and kolla-toolbox, similar to other services. This
changes the behaviour in the following ways:

* The common role is now run for all hosts at the beginning, rather than
  prior to their first enabled service
* Hosts must be in the necessary group for each of the common services
  in order to have that service deployed. This is mostly to avoid
  deploying on localhost or the deployment host
* If tags are specified for another service e.g. nova, the common role
  will *not* automatically run for matching hosts. The common tag must
  be specified explicitly

The last of these is probably the largest behaviour change. While it
would be possible to determine which hosts should automatically run the
common role, it would be quite complex, and would introduce some
overhead that would probably negate the benefit of splitting out the
common role.

Partially-Implements: blueprint performance-improvements

Change-Id: I6a4676bf6efeebc61383ec7a406db07c7a868b2a

Currently there is no way to configure a CA certificate bundle file for
fluentd to Elasticsearch communication. This change adds a new variable,
'fluentd_elasticsearch_cacert' with a default value set to the value of
'openstack_cacert.

Closes-Bug: #1885109

Change-Id: I5bbf55a4dd4ccce9fa2635cee720139c088268e3

more info: https://review.opendev.org/#/c/721733/



Depends-On: I561ead226f714d98c8e06e6027715a64c3a8e47e
Depends-On: I21c9ab9820f78cf76adf11c5f0591c60f76372a8
Change-Id: Ic740d090211ee331b374a6dac69dfde466df7200
Co-Authored-By: jacky06 <zhang.min@99cloud.net>

more info: https://opendev.org/openstack/kolla-ansible/commit/a6c97d7284c7de437ebfc9f8ee289244f29e65d7



Change-Id: I778d472cc7f6ca19852482a3e309d793973d75a6
Co-Authored-By: jacky06 <zhang.min@99cloud.net>

Add TLS support for Glance api using HAProxy to perform TLS termination.

Change-Id: I77051baaeb5d3f7dd9002262534e7d35f3926809
Partially-Implements: blueprint add-ssl-internal-network

Some services look for /etc/timezone on Debian/Ubuntu, so we should
introduce it to the containers.

In addition, added prechecks for /etc/localtime and /etc/timezone.

Closes-Bug: #1821592
Change-Id: I9fef14643d1bcc7eee9547eb87fa1fb436d8a6b3

The 'kolla-ansible stop' command can be used to stop the services
running on hosts. However, if you run this command in an environment
with heterogeneous nodes (most real world scenarios have at least
control/compute), then it fails. This is because it only checks
whether a container is enabled, and not whether the host is in the
correct group. For example, it fails with nova-libvirt:

    No such container: nova_libvirt to stop.

This change fixes the issue by only attempting to stop containers on
hosts to which they are mapped.

Change-Id: Ibecac60d1417269bbe25a280996ca9de6e6d018f
Closes-Bug: #1868596

Fluentd cannot accept empty 'path' parameter.

I refactored the service list following the general pattern
we have.

Change-Id: I83d820efcc7e86bac9f8bda26a8f8bece72159e6
Closes-bug: #1867953

Fix elasticsearch schema in fluentd when kolla_enable_tls_internal is
true.

Change-Id: I51286d2def7a762d569740c1abc5b924b682ad9d
Closes-Bug: #1867481

The logrotate rotation interval and count are not configurable.
Currently, the configuration is a "default" that keeps 6 weeks of logs.

Change-Id: I4f55ee2a98f7861cb8de2724f5edc32da6d2f9ee

For the CentOS 7 to 8 transition, we will have a period where both
CentOS 7 and 8 images are available. We differentiate these images via a
tag - the CentOS 8 images will have a tag of train-centos8 (or
master-centos8 temporarily).

To achieve this, and maintain backwards compatibility for the
openstack_release variable, we introduce a new 'openstack_tag' variable.
This variable is based on openstack_release, but has a suffix of
'openstack_tag_suffix', which is empty except on CentOS 8 where it has a
value of '-centos8'.

Change-Id: I12ce4661afb3c255136cdc1aabe7cbd25560d625
Partially-Implements: blueprint centos-rhel-8

Enable reconnect_on_error option so that ES plugin re-establishes
a new session to the ES cluster on errors. Also, enable buffering
to the file, so that the buffer survives container restarts.

Co-Authored-By: Michal Nasiadka <mnasiadka@gmail.com>
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Co-Authored-By: Doug Szumski <doug@stackhpc.com>
Closes-Bug: #1830724
Change-Id: Ia40685b9d4fc02194e03c8791ddeb3d29d7f07f6

We don't add extra volumes support for all services in patch [1].
In order to unify the management of the volume, so we need add extra volumes
support for these services.

[1] https://opendev.org/openstack/kolla-ansible/commit/12ff28a69351cf8ab4ef3390739e04862ba76983



Change-Id: Ie148accdd8e6c60df6b521d55bda12b850c0d255
Partially-Implements: blueprint support-extra-volumes
Signed-off-by: ZijianGuo <guozijn@gmail.com>

Add options for configuring TLS and authentication for elasticsearch
connections in in fluentd.

Change-Id: I936adc2aeaa3c87081be1c44aa0221caf2124e23
Closes-Bug: #1831078

Change-Id: I0cbc51eedb218f6b2f7d5c50372ad7a8be9bdcc0

This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa

Change-Id: I47b329bb3e16aa33a5e0a156aca515a666cd98e4

Since the kolla-kubernetes has been separated, and nowhere to
use kubetoolbox, it is safe to remove it from kolla-ansible.

Change-Id: I9aa59d45fa04c13944120f1f99af2361551147ad

This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799

This change adds enable_fluentd option and enables some other log shippers
to be integrated. When enable_fluentd is "no", syslog server is also disabled.
Then, this change also adds syslog parameters to use a syslog server
prepared by users.

Change-Id: I7c83ef7fe30a6b9ab7385bcee953ad07e96b0a83
Implements: blueprint fluentd-enable-option

SKIP_LOG_SETUP variable was used by Heka stuff.
Heka has been removed from Kolla and Kolla-ansible.

Change-Id: I4448b26ace899eb888d34a12a85b019597e25435

Cron is always restarting due dummy environment
variable is None.
This change adds a useless env value.

Change-Id: Iad435bb0671a25bc12c6a6d0d988d555faba76c3
Closes-Bug: #1697706

Change-Id: I57e3b310c65aabeea9f298db78762e9d535e8703
Partially-implements: blueprint better-reconfigure

* add fluentd role
* remove heka configure

Co-Authored-By: yangzb09 <yangzb09@qq.com>

Partially-Implements: blueprint add-fluentd-role

Change-Id: Ica804a99f5bb8b157f406299c5982b7b6283b3e3

This PS adds kubetoolbox image which is required for Kubernetes
fencing pod. This image includes, kubectl binary, ceph-common and ipmi.

Change-Id: I17b2a47b95ed7dea391f1427a41c92b15c97986d
Partially-Implements: blueprint kubernetes-toolbox

This PS adds Fluentd docker image to Kolla. Kolla
currently does not use Fluentd for log collection, but this image
is required for Kubernetes centralized logging solution.

Change-Id: I719661d75f5044239a1fd52080d322c77490ecae
Partially_Implements: blueprint kubernetes-images

keystone_*_url are cross role variables. They are used in multi roles.
Move them from the common role to the group vars

TrivialFix

Change-Id: If451823ed7612bfec7bc797ec9dd2597164c6804

Note: This should not result in any behavior changes in regular Kolla, just
Kolla-Kubernetes and only when you've overridden stuff in globals.yml

Allows override of interface address and memcached pools, so that Kubernetes
can do the right thing.

There are some significant architectural issues involved in memcached pooling
in the Kolla-kubernetes world.  Avoiding them right now.

Current working with this Kolla-Kubernetes globals.yml file:

api_interface_address: "0.0.0.0"

memcached_servers: "memcached"

keystone_database_address: "mariadb"
keystone_admin_url: "http://keystone-admin:35357/v3"
keystone_internal_url: "http://keystone-public:5000/v3"
keystone_public_url: "http://keystone-public:5000/v3

"

Three tings to note:
 * In Kolla-Kubernetes, the service is not using net=host, so a
   0.0.0.0 interface address is totally OK.  That patch has been merged.
 * In Kolla-Kubernetes, the global.yml file doesn't do var substitution
   so you have to be explicit about the URLs, otherwise Keystone will
   look like it was provisioned but it won't quite be provisioned right.
 * In order to not duplicate tons of code, moved the keystone_admin_url /
   keystone_internal_url / keystone_public_url to the common defaults
   from the keystone defaults.

Co-Authored-By: Ryan Hallisey <rhallise@redhat.com>
Change-Id: I586ce1c6c3300254c4e2a398ff46645df576aeb0
Partially-implements: blueprint api-interface-bind-address-override

All the images follow the same naming schema where the container
name is part of the ansible variable name.  Currently, the
kolla_toolbox container is named ansible_image_full.  Change
this to match what the container name is.

Could impact operators who are specifying ansible_image_full
in globals.yml.

TrivialFix

Change-Id: I9f0cbe4bd69ce17489310a1f5734a254e6ebe27b

Add cron image and playbook for logrotate.

The "common" Ansible playbook includes configuration files for
logrotate. At this point the operator cannot customize/override the
logrotate configuration.

Closes-Bug: #1553244
Change-Id: Ic9fdda9a273c9ccd90502f0acc7614d2c7157dca