This reverts commit 5b431f0f.

Reason for revert: the any_errors_fatal play parameter is not templated
by Ansible (tested up to ansible-core 2.15.9). This behaviour is
demonstrated in [1].

This means that "{{ kolla_ansible_setup_any_errors_fatal }}" is always
interpreted as 'true', regardless of the value of
kolla_ansible_setup_any_errors_fatal.  This is particularly bad because
the default value of kolla_ansible_setup_any_errors_fatal is false.

We now have gather_facts_max_fail_percentage which can be set to 0 to
provide the same functionality.

[1] https://github.com/markgoddard/ansible-experiments/tree/master/15-fatal-errors

Change-Id: I2e0ea49701b5900eae26434bcdb6b1bb44507ee7

Previously, during fact gathering when using the '--limit' argument if a
host inside the limit fails to gather facts, this would cause another
host to fail during delegated fact gathering. This is because the
ansible_facts variable would not be populated for the host.

This change fixes the issue by avoiding delegated fact gathering for
hosts inside the limit.

Partial-Bug: #2041860
Change-Id: I8e321469089f0eb80420b788b51e6d028c7fef4d

This allows us to continue execution until a certain proportion of hosts
to fail. This can be useful at scale, where failures are common, and
restarting a deployment is time-consuming.

The default max failure percentage is 100, keeping the default
behaviour. A global max failure percentage may be set via
kolla_max_fail_percentage, and individual services may define a max
failure percentage via <service>_max_fail_percentage.

Note that all hosts in the inventory must be reachable for fact
gathering, even those not included in a --limit.

Closes-Bug: #1833737
Change-Id: I808474a75c0f0e8b539dc0421374b06cea44be4f

If not using --limit, each host runs the setup module for itself only.
Hosts should be able to fail at this point and drop out of execution
without affecting others. However, there is a bug that causes us to hit
the --limit code path for an unreachable host and therefore become
subject to its limitations.  These include the use of delegated fact
gathering, and failing the delegated host when the delegating host is
unreachable.

This change fixes the issue by reversing the order of two tasks in the
gather-facts.yml playbook, ensuring that we check the ansible_play_batch
variable before any hosts have had a chance to fail.

Change-Id: I4b1da63e8f0cc2774f1a9ab2e0414746108f7e12
Closes-Bug: #2041859

These were added in I20e65a6771ebeee462a3aaaabaa5f0596bdd0581 but not
used in the final version of the changeset.

Change-Id: I4fca47e16e87db8b655ff5571fba568ef4edb95c

This commit adds SystemdWorker class to kolla_docker ansible module.
It is used to manage container state via systemd calls.

Change-Id: I20e65a6771ebeee462a3aaaabaa5f0596bdd0581
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>

Kolla Ansible now supports failing execution early if fact collection
fails on any of the hosts. This is to avoid late failures due to missing
facts (especially cross-host).

Change-Id: I7a74b937ded0b9da0621cf413f3a5d0d13a2cd68
Partial-Bug: #1833737

Ansible facts can have a large impact on the performance of the Ansible
control host. This patch introduces some control over which facts are
gathered (kolla_ansible_setup_gather_subset) and which facts are stored
(kolla_ansible_setup_filter). By default we do not change the default
values of these arguments to the setup module. The flexibility of these
arguments is limited, but they do provide enough for a large performance
improvement in a typical moderate to large OpenStack cloud.

In particular, the large complex dict fact for each interface has a
large effect, and on an OpenStack controller or hypervisor there may be
many virtual interfaces. We can use the kolla_ansible_setup_filter
variable to help:

    kolla_ansible_setup_filter: 'ansible_[!qt]*'

This causes Ansible to collect but not store facts matching that
pattern, which includes the virtual interface facts. Currently we are
not referencing other facts matching the pattern within Kolla Ansible.
Note that including the 'ansible_' prefix causes meta facts module_setup
and gather_subset to be filtered, but this seems to be the only way to
get a good match on the interface facts. To work around this, we use
ansible_facts rather than module_setup to detect whether facts exist in
the cache.

The exact improvement will vary, but has been reported to be as large as
18x on systems with many virtual interfaces.

For reference, here are some other tunings tried:

* Increased the number of forks (great speedup depending of the size of
  the deployment)
* Use `strategy = mitogen_linear` (cut processing time in half)
* Ansible caching (little speed up)
* SSH tunning (little speed up)

Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Closes-Bug: #1921538
Change-Id: Iae8ca4aae945892f1dc65e1b10381d2e26e88805

By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.

This change updates all references to Ansible facts within Kolla Ansible
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.

This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.

[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars

Change-Id: I7e9d5c9b8b9164d4aee3abb4e37c8f28d98ff5d1
Partially-Implements: blueprint performance-improvements

This trivial patch is just turning off ansible
changed report for group_by tasks as it could
be confusing for user.

Change-Id: I7512af573782359a6f01290a55291ac7eb0de867

One way to improve the performance of Ansible is through fact caching.
Rather than gather facts in every play, we can configure Ansible to
cache them in a persistent store. An example Ansible configuration for
doing this is as follows:

[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./facts
fact_caching_timeout = 86400

This does not affect Kolla Ansible however, since we use the setup
module which unconditionally gathers facts regardless of the state of
the cache. This gets worse with large inventories limited to a small
batch of hosts via --limit or serial, since the limited hosts must
gather facts for all others.

One way to detect whether facts exist for a host is via the
'module_setup' variable, which exists only when facts exist. This change
uses the 'module_setup' fact to determine whether facts need to be
gathered for hosts outside of the batch. For hosts in the batch, we
switch from using the setup module to gather_facts on the play, which
can use the 'smart' gathering logic.

Change-Id: I04841fb62b2e1d9e97ce4b75ce3a7349b9c74036
Partially-Implements: blueprint performance-improvements

Prior to this change, when the --limit argument is used, each host in the
limit gathers facts for every other host. This is clearly unnecessary, and
can result in up to (N-1)^2 fact gathers.

This change gathers facts for each host only once. Hosts that are not in
the limit are divided between those that are in the limit, and facts are
gathered via delegation.

This change also factors out the fact gathering logic into a separate
playbook that is imported where necessary.

Change-Id: I923df5af41a7f1b7b0142d0da185a9a0979be543

In change I78cb60168aaa40bb6439198283546b7faf33917c, action was changed
to kolla_action, and serial to kolla_serial, to avoid Ansible warnings
due to use of reserved keywords. In that change, some keywords were
missed, and some changes that were merged since then have not switched
to the new variables. This change fixes all current instances of those
issues.

Change-Id: I357dffdfcb2b405e280a962d366ee65eebf0a8d1
Implements: blueprint migrate-to-ansible-2-2-0

This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799

The ansible/kolla-host.yml playbook failed when called with a --limit
flag.

This was due to the fact that only the facts of the subset of hosts
were gathered but the "Generate /etc/hosts for all of the nodes" task
is using the ansible_<IFACE>.ipv4.address variable of all hosts.

The code for this fix is copied from the site.yml playbook
where it was used for the same reason.

Change-Id: I7e70caf750506ad8ab36ec024bd9647dd733fbd0
Closes-Bug: #1713725

Change-Id: I44c2668a8ebb6dd3201a6eb4e47284871380e6d7

Change-Id: Ie70db1b18a73528b1194e4fbcf53d09fed20f4a0
Implements: blueprint kolla-host