Add TLS support for Glance api using HAProxy to perform TLS termination.

Change-Id: I77051baaeb5d3f7dd9002262534e7d35f3926809
Partially-Implements: blueprint add-ssl-internal-network

Some services look for /etc/timezone on Debian/Ubuntu, so we should
introduce it to the containers.

In addition, added prechecks for /etc/localtime and /etc/timezone.

Closes-Bug: #1821592
Change-Id: I9fef14643d1bcc7eee9547eb87fa1fb436d8a6b3

The 'kolla-ansible stop' command can be used to stop the services
running on hosts. However, if you run this command in an environment
with heterogeneous nodes (most real world scenarios have at least
control/compute), then it fails. This is because it only checks
whether a container is enabled, and not whether the host is in the
correct group. For example, it fails with nova-libvirt:

    No such container: nova_libvirt to stop.

This change fixes the issue by only attempting to stop containers on
hosts to which they are mapped.

Change-Id: Ibecac60d1417269bbe25a280996ca9de6e6d018f
Closes-Bug: #1868596

Fluentd cannot accept empty 'path' parameter.

I refactored the service list following the general pattern
we have.

Change-Id: I83d820efcc7e86bac9f8bda26a8f8bece72159e6
Closes-bug: #1867953

Fix elasticsearch schema in fluentd when kolla_enable_tls_internal is
true.

Change-Id: I51286d2def7a762d569740c1abc5b924b682ad9d
Closes-Bug: #1867481

The logrotate rotation interval and count are not configurable.
Currently, the configuration is a "default" that keeps 6 weeks of logs.

Change-Id: I4f55ee2a98f7861cb8de2724f5edc32da6d2f9ee

For the CentOS 7 to 8 transition, we will have a period where both
CentOS 7 and 8 images are available. We differentiate these images via a
tag - the CentOS 8 images will have a tag of train-centos8 (or
master-centos8 temporarily).

To achieve this, and maintain backwards compatibility for the
openstack_release variable, we introduce a new 'openstack_tag' variable.
This variable is based on openstack_release, but has a suffix of
'openstack_tag_suffix', which is empty except on CentOS 8 where it has a
value of '-centos8'.

Change-Id: I12ce4661afb3c255136cdc1aabe7cbd25560d625
Partially-Implements: blueprint centos-rhel-8

Enable reconnect_on_error option so that ES plugin re-establishes
a new session to the ES cluster on errors. Also, enable buffering
to the file, so that the buffer survives container restarts.

Co-Authored-By: Michal Nasiadka <mnasiadka@gmail.com>
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Co-Authored-By: Doug Szumski <doug@stackhpc.com>
Closes-Bug: #1830724
Change-Id: Ia40685b9d4fc02194e03c8791ddeb3d29d7f07f6

We don't add extra volumes support for all services in patch [1].
In order to unify the management of the volume, so we need add extra volumes
support for these services.

[1] https://opendev.org/openstack/kolla-ansible/commit/12ff28a69351cf8ab4ef3390739e04862ba76983



Change-Id: Ie148accdd8e6c60df6b521d55bda12b850c0d255
Partially-Implements: blueprint support-extra-volumes
Signed-off-by: ZijianGuo <guozijn@gmail.com>

Add options for configuring TLS and authentication for elasticsearch
connections in in fluentd.

Change-Id: I936adc2aeaa3c87081be1c44aa0221caf2124e23
Closes-Bug: #1831078

Change-Id: I0cbc51eedb218f6b2f7d5c50372ad7a8be9bdcc0

This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa

Change-Id: I47b329bb3e16aa33a5e0a156aca515a666cd98e4

Since the kolla-kubernetes has been separated, and nowhere to
use kubetoolbox, it is safe to remove it from kolla-ansible.

Change-Id: I9aa59d45fa04c13944120f1f99af2361551147ad

This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799

This change adds enable_fluentd option and enables some other log shippers
to be integrated. When enable_fluentd is "no", syslog server is also disabled.
Then, this change also adds syslog parameters to use a syslog server
prepared by users.

Change-Id: I7c83ef7fe30a6b9ab7385bcee953ad07e96b0a83
Implements: blueprint fluentd-enable-option

SKIP_LOG_SETUP variable was used by Heka stuff.
Heka has been removed from Kolla and Kolla-ansible.

Change-Id: I4448b26ace899eb888d34a12a85b019597e25435

Cron is always restarting due dummy environment
variable is None.
This change adds a useless env value.

Change-Id: Iad435bb0671a25bc12c6a6d0d988d555faba76c3
Closes-Bug: #1697706

Change-Id: I57e3b310c65aabeea9f298db78762e9d535e8703
Partially-implements: blueprint better-reconfigure

* add fluentd role
* remove heka configure

Co-Authored-By: yangzb09 <yangzb09@qq.com>

Partially-Implements: blueprint add-fluentd-role

Change-Id: Ica804a99f5bb8b157f406299c5982b7b6283b3e3

This PS adds kubetoolbox image which is required for Kubernetes
fencing pod. This image includes, kubectl binary, ceph-common and ipmi.

Change-Id: I17b2a47b95ed7dea391f1427a41c92b15c97986d
Partially-Implements: blueprint kubernetes-toolbox

This PS adds Fluentd docker image to Kolla. Kolla
currently does not use Fluentd for log collection, but this image
is required for Kubernetes centralized logging solution.

Change-Id: I719661d75f5044239a1fd52080d322c77490ecae
Partially_Implements: blueprint kubernetes-images

keystone_*_url are cross role variables. They are used in multi roles.
Move them from the common role to the group vars

TrivialFix

Change-Id: If451823ed7612bfec7bc797ec9dd2597164c6804

Note: This should not result in any behavior changes in regular Kolla, just
Kolla-Kubernetes and only when you've overridden stuff in globals.yml

Allows override of interface address and memcached pools, so that Kubernetes
can do the right thing.

There are some significant architectural issues involved in memcached pooling
in the Kolla-kubernetes world.  Avoiding them right now.

Current working with this Kolla-Kubernetes globals.yml file:

api_interface_address: "0.0.0.0"

memcached_servers: "memcached"

keystone_database_address: "mariadb"
keystone_admin_url: "http://keystone-admin:35357/v3"
keystone_internal_url: "http://keystone-public:5000/v3"
keystone_public_url: "http://keystone-public:5000/v3

"

Three tings to note:
 * In Kolla-Kubernetes, the service is not using net=host, so a
   0.0.0.0 interface address is totally OK.  That patch has been merged.
 * In Kolla-Kubernetes, the global.yml file doesn't do var substitution
   so you have to be explicit about the URLs, otherwise Keystone will
   look like it was provisioned but it won't quite be provisioned right.
 * In order to not duplicate tons of code, moved the keystone_admin_url /
   keystone_internal_url / keystone_public_url to the common defaults
   from the keystone defaults.

Co-Authored-By: Ryan Hallisey <rhallise@redhat.com>
Change-Id: I586ce1c6c3300254c4e2a398ff46645df576aeb0
Partially-implements: blueprint api-interface-bind-address-override

All the images follow the same naming schema where the container
name is part of the ansible variable name.  Currently, the
kolla_toolbox container is named ansible_image_full.  Change
this to match what the container name is.

Could impact operators who are specifying ansible_image_full
in globals.yml.

TrivialFix

Change-Id: I9f0cbe4bd69ce17489310a1f5734a254e6ebe27b

Add cron image and playbook for logrotate.

The "common" Ansible playbook includes configuration files for
logrotate. At this point the operator cannot customize/override the
logrotate configuration.

Closes-Bug: #1553244
Change-Id: Ic9fdda9a273c9ccd90502f0acc7614d2c7157dca

Partially implements: blueprint heka

Change-Id: I1322d2dc870e6f8fe052926995d993e8a08a25db

Partially implements: blueprint heka
Change-Id: I9ddad09b3f16b323c7ed535243458b0de6620cdb

This change is needed for clarity. We have a kolla-ansible script.
We have a kolla-mesos repo. We plan to have a kolla-ansible repo.
Already we have had far too much confusion about whether we are
talking about the container or the project. Naming this kolla-toolbox
eliminates all of that confusion and its probably a bit more accurate
of a name too.

Closes-Bug: #1541053
Change-Id: I8fd1f49d5a22b36ede5b10f46b9fe02ddda9007e

By default, if the same role is called in ansible it will only run
once per node. Due to how we have the inventory setup ansible views
every service as a different node and will try to run the common role
tasks again. This causes slowdown in all cases but is particularly
noticable when pulling images. A small change will ensure these tasks
only run once per node per run as originally intended.

TrivialFix

Change-Id: I20b9c46991d10176c8f8645a335eb7a9ed750ee3

This intentionally leaves out rabbitmq from this patchset. It will
require additional work to remove its data container

UpgradeImpact
Partially-Implements: blueprint docker-named-volumes

Change-Id: Id68b8e43a3c077ef4f4f4d67ea34d0692e66eef7

Throughout the project overtime some of these file permissions have
changed to have an executable bit. They should not have this bit set.

TrivialFix

Change-Id: I1748b5bde813a0fcac36aeecdfd83245b8ee5be3

Ansible installation of rsyslog container

Change-Id: I17a7ee679f72e8fe667012d7eb6e07a27098a909
Implements: bp logging-container

Change-Id: If2e6d5cd5530c1e17f34415e169f82465f8362a1
Partially-Implements: blueprint remove-abstraction-ansible

Currently we require a slew of deps on each destination node, this
includes a gcc compiler and installing things via pip. We can remove
these dependencies by containerizing them and running and Ansible
inside the container itself. The container would then report back
facts about idempotency.

DocImpact
Closes-Bug: #1481495
Implements: blueprint containerize-dependencies
Change-Id: I3dfccbf9fafc06ffc36e78f3006fe5d3367891df