Services such as Zookeeper and Kafka use ini files which do not
explicitly specify sections. This change supports merging ini
files with no sections, so that the configuration for these
services follows the example configuration provided for them
as closely as possible.

Closes-Bug: #1756101
Partially-Implements: blueprint monasca-roles
Change-Id: I1061729875e5545c7af7d80779f9c2124b6c7134

Till now we've been flusing iptables in the gates to allow cross node
communication in the multi node ceph jobs. This raised security
concerns, in particular it exposed memcached to the external net.

This patch uses the infra provided role 'multi-node-firewall' in order
to correctly configure iptables. Thanks to Jeremy Stanley and Jeffrey
for help with this.

Closes-Bug: #1749326
Change-Id: Iafaf1cf1d9b0227b0f869969d0bd52fbde3791a0

This reverts commit 2a4ce0b8.

Change-Id: I52efd29791a230df1ce7e3acde4b7184aec34150

We have pin keyston to queens release which supports UUID token through
https://review.openstack.org/546475, let us use UUID in queens and
migrate to fernet in rocky cycle.

This reverts commit df0bf190.

Change-Id: Ifb0112315b5047461ce0bf02c754cc0beac52d9a

we have gates currently broken due using rocky keystone code.
we have no way to test our release until we branch out queens.
this change doesnt change globals default, only our gates tests.

Change-Id: I189aa8cafc73a365668d79e4b186aac21da63a6f

Missing container status check in recreate_or_restart_container,
this causes if the container is not running (kolla-ansible stop),
to not be started with deploy/reconfigure/upgrade if any other param
changes.

Change-Id: I5cff5f367e963ba8b1807ec46469da817e40e468
Closes-Bug: #1714015

- Ceph images are not being built when using depends-on a kolla build
  job.
- Sync inventory files with current ones which causes ceph to fail due
  missing groups.
- Small corrections in ceph config.yml syntax.
- fix preparse the disk only once
- enable ceph nfs only when enable_ceph_nfs is true

Co-Authored-By: Jeffrey Zhang <zhang.lei.fly@gmail.com>
Change-Id: Id0c7963bf59e2af4944834dcd16589a638e78ba5

Use the same router will result in conflict in the CI environment.

Change-Id: Id1ba69ebbf67b4cead855adbe440343c05ea3d1e
Closes-Bug: #1744913

By default, the ansible task will gather facts, it can be configured by
set "father_facts: true", it's unncessary to run setup in tasks, it will
waste much time when hosts is large.

Change-Id: I9462a538c76b6271a34531d8752b72cc695f693e
Closes-Bug: #1743485

Change-Id: Ifdf2ede2cb5f057d1d1f02bfa00ee05872e94b9f

multinode gate change introduced bug that breaks kolla deploy gates.
This is fix.

Change-Id: Ibc5fe3adb3555e177d3a6a64250cbc9bf50430d1

Another step to migrate to new CI

Change-Id: Id9bf9b4b1114f5be66f6c028a0db16d869efeceb
Depends-On: Iaacdca31b27b7033ec08499bcea7061b9d3ec279

Change-Id: Id71c1ecda9f8647a0eb290fe02a12afb3df09988

- move check container failure from post.yml to run.yml
- add binary related jobs
- use static kolla-ansible src dir, which is helpful for kolla project
  to use.
- generate correct /etc/hosts by using private ip address and hostname
- fix the wrong api interface in global.yml file

Change-Id: Idfdee6dfe18f0fa2d4f984df59b57553122ce298

Partial-Bug: #1720601
Change-Id: Ibc20a6ae8c645ff82f3c14a6286073dffd4cfae2

Fluentd send logs to stdout,
this changes creates a file with fluentd logs
output to easy discover missing patterns during fluentd
changes.

Change-Id: I131f95089eac60ccb4c48cf5071c3b44c5ea42ca

Check status of ceph cluster

Change-Id: I4919a32794cc75bd3e8f411a219f778238a334ee

Ceph gates notoriously run out of memory

Change-Id: I5cf5cc5adf5379095ed9e3094494e65b76ff62d8

Docker has an issue[0] when restart container. But stop then start
works.

[0] https://github.com/moby/moby/issues/29704

Change-Id: If0a9c0c257cd72209be8e138a1f0b8871500e089
Closes-Bug: #1707097

Log retrieval was out of sync since repo split
and from multinode gates.

Many useful information retrieved before like
ps, df, docker info, etc is not in kolla-ansible
gates.

Also, his change fix logs visualization to have
colored view, allowing to easily identify errors.

Change-Id: I948233e26ceb6efc58b962bcb4b710b3f006232b

Ansible check if modules parameters are named like
%password% and allow to hidden log param in this case.
This requires adding "no_log" parameters.
This patch just add "no_log" param in order to avoid
this warning.

Change-Id: I9c1df1093e0fd101090292d6e8bf3527f99aeb17
Closes-Bug: #1702244

The pypi package 'docker-py' [1] has been renamed to 'docker' [2].
It is better to move to the new 'docker' package because the old
package will be deprecated and all the new features will go into
the new package only.

Package 'docker' has been added to requirements [3]. The old
package 'docker-py' is still allowed to be in the global requirements
during the transition period but it should be removed after all or
most of the projects finsih the migration.

[1] https://pypi.python.org/pypi/docker-py
[2] https://pypi.python.org/pypi/docker
[3] https://review.openstack.org/#/c/423715/

Change-Id: Ibcd5a57a1fbf55dcc5a690e41f20917f95b63da0

First non-trivial scenario we'll be testing in gates will be cinder +
ceph. This patchset is supposed to create framework to easily add new
testing scenarios.

Change-Id: I6790eaf5cf3297af04a64e4169a39b6e2cd1890f

This reverts commit 94a882ba.

Upstream has fix this issue by Ice89ad9dd486ad5fcac534ef5f7d8aae3b6b0f97

Change-Id: Ib17f93e4030c6f7da4e32ee84fcc5f7139e57895
Closes-Bug: #1694420

In testcase, setUp will be called automatically. This patch used to
remove setUp functions that do nothing. Besides, it will keep code clean.

Change-Id: I60e368dc066af30ce2e53bf7e4cc7bba69387cd8

This patches changes deploy_gate quite a bit so in reality all
deployments will now assume multinode (even if it's single node). After
that we will refactor it even further to enable easy addition of new
scenerios.

Change-Id: I1faada46e6a7aa026128b2f01d77eabb04759439

Currently, when stoping/restarting container, Kolla uses default timeout value
between SIGTERM and SIGKILL provided by docker which is 10 sec. But some
services require more than it to finish graceful shutdown progress.

This patchset adds graceful_timeout to kolla_docker to override the default
one.

Partial Implements: bp signaling-to-container

Change-Id: Ica0b48a53c650cc23dfa1955027d2cf936a5932f

This patch removes very old and unused Keystone test path, we do
this indirectly by our simple sanity tests (and hopefully) by using
Tempest.

Change-Id: I55916d08942df97990515c76914b799d1f327565

At the moment, the process to determine if an image has
changed or not relies on the Docker API which depending
on the Docker release server can return different results.

This patch addresses this issue by grabbing the SHA256
of the image before pulling (defaulting to None if it does
not exist) and then comparing it after the pull is complete
which should always be successful at determining if the
image did change or not.

The test for unknown status images is removed because this
is not a possible scenario as we do not rely on status
anymore except for failures (which are still tested).

Change-Id: Ia60a7f34420b02f50597dddb96a4c36ff3996612
Closes-Bug: #1668059

There is inconsistent use of either `/usr/bin/python` or
`/usr/bin/env python`. This makes for unexpected results when a
user might be using a virtualenv.

Change-Id: Ibb030f920a8869f9113ade70b66a921cc815060d

bypass_checks is disabled by Ib2e19794bbe804470a880253d5870254041358c8 .
But we use pid_mode: "{{ service.pid_mode | default('') }}" pattern in
the yaml file. whereas pid_mode only support "host" value, this patch
add '' as a choice of these two parameters.

Change-Id: Ib83ed5e437ca868f0e42ce740ad1125d2fa48ff4

Recreate and start a new container when container parameter is changed
in COPY_ALWAYS strategy.

Change-Id: I1e45c0dc34a93a18b664109823d6fd5b88c331a2
Closes-Bug: #1659799

Replace the original code with required_if when Ansible 2.0 lands

Change-Id: Ib2e19794bbe804470a880253d5870254041358c8

Change-Id: I7df581243deef7bda4b9b5926addf61421bd42f7

Change-Id: Ibf9ee55c48a5181c5ba3aa46bef485cd3468f14c
Closes-Bug: #1647065

Partically-implements: blueprint better-reconfigure
Change-Id: I89e30e8b87f24a621c521d915842a4af0042d6fe

* rename package name from kolla to kolla-ansible
* remove docker for data_files
* remove kolla docker
* remove kolla-build console_scripts

Change-Id: I53abbf79dffb54eb785a39ba04d375bc4e4f27b0

When using stream=True with docker-py it returns byte arrays rather than
strings.

Change-Id: I8eb6707ba2b122cf779135173abafe874b7223d2
Closes-Bug: #1643740

Change-Id: I0ecb9004941aca9a9520ba8e91b9f1bffaa66b61

Change-Id: Ibf74db39348dfb6f56f4fca0aff142a75ef53d38
Partially-Implements: blueprint add-panko