The dib_env_vars variable in the Bifrost's dib.yml file can contain
the DIB_BLOCK_DEVICE_CONFIG environment variable which is always the
Multiline-YAML data. By default, the format of the data is not
preserved while the configuration is merged and saved for the
bifrost-deploy container.

This is because Ansible uses the PyYAML library which has a default
80 symbol string length limit. The official Ansible documentation [1]
recommends using to_yaml or to_nice_yaml filters with width parameter.
This change adds the same ability to the merge_yaml Ansible plugin.

1. https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#formatting-data-yaml-and-json



The related change for the diskimage-builder to solve the issue with
incorrect data provided by Kolla-Ansible is also provided:
I3b74ede69eb064ad813a9108ec68a228e549e8bb

Closes-Bug: #2014980
Related-Bug: #2014981
Change-Id: Id79445c0311916ac6c1beb3986e14f652ee5a63c
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

This change fixes the output 'module_args' information of the plugins
'merge_configs' and 'merge_yaml' when Ansible is executed in maximum
verbose mode. Now all the plugin options are displayed instead of
standard 'copy' plugin options only.

Also, this change contains fixes already applied in the Kayobe
project to improve and synchronize the code of the plugins between
projects.

Change-Id: Ie2d9a0501fe29bfd854eb31258f282b197855948
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

See https://github.com/ansible/ansible/pull/76677

Change-Id: If822dcfc4c1abf7a22be35ffd90fd05ee46cb0d9

Dumper and Loader are classes seem to be loaded but not used in the merge_yaml
file. This change removes them for reducing the number of lines.

Change-Id: I87ef305903ab02226fcaa725ece622647d17811c

Change-Id: I0f107d8239ae0cbef3bb4569ec5d77326f8974fc

The merge_configs action plugin is updated with an option to control
whether whitespace should be used around equal signs. The default
remains to use whitespace: only zookeeper.cfg doesn't use them.

This is to avoid issues with the zkCleanup.sh script which expects no
whitespace. The ZooKeeper documentation also uses no whitespace in
configuration file examples.

Change-Id: Ia082a1c002cc4e8b04f7696fdee827b747c6d13f
Closes-Bug: #1917490

set_available_variables is being deprecated.
Use "@available_variables.setter" instead.

Refer to: https://fossies.org/linux/ansible/lib/ansible/template/__init__.py

Change-Id: I874318f78043d21d2aeebe4e0c35a304b5c94029

This allows you to extend lists in yaml config. This is useful, for
example, in prometheus.yml, where it would be nice to be able to
extend the scrape_configs to include exporters that aren't packaged
with kolla-ansible. This would provide a mechanism to do so.

Change-Id: I7a10e363f42e8ffaae3c0d2c2a758853e2cab7e1
Related: blueprint custom-prometheus-targets

These are executed on the local host where we run ansible-playbook,
and we have agreed to drop Python 2 support there.

Partially Implements: blueprint drop-py2-support
Change-Id: Id2190c3a22a56f4f048afbf0f7200daa8f41a292

This mimics behavior of core 'template' module to allow relative
includes from the same dir as merged template, base dir of
playbook/role (usually role for us) and its 'templates' subdir.

Additionally old unused code was removed.

Change-Id: I83804d3cf5f17eb2302a2dfe49229c6277b1e25f
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Fix filemode in the merge_configs and merge_yaml action plugin to
be compatible with python3

Change-Id: Ief64c5bdcd717141281e23c255a49ec02a96aef2
Closes-Bug: #1820134

Use latest release 1.1.0 and compatible changes w.r.t pep8

Change-Id: Ie5e5417a1d8481bb59d516a320bd7cc1dd89591e

Closes-Bug: 1786741
Change-Id: Ic217f3f8194cdcb5e28045e771a0160c1f12280e

move the merge_yaml and merge_config module's DOCUMENTATION and
EXAMPLES into action_plugins.

Change-Id: I84c5b94afb870fc9a25838782389f7b1f8b882fd
Closes-Bug: #1799236

Change-Id: Ib2ca736e08f48da88bb74feb5cd1efce3b860ab7
Partially-Implements: blueprint ansible-check-mode

Services such as Zookeeper and Kafka use ini files which do not
explicitly specify sections. This change supports merging ini
files with no sections, so that the configuration for these
services follows the example configuration provided for them
as closely as possible.

Closes-Bug: #1756101
Partially-Implements: blueprint monasca-roles
Change-Id: I1061729875e5545c7af7d80779f9c2124b6c7134

Ansible task support vars directive, no need implement another one in
merge_config. This patch remove the vars directive in merge_config
action plugin.

Change-Id: I33648a2b6e39b4d49ce76eb66fbf2522721f8c68

There is inconsistent use of either `/usr/bin/python` or
`/usr/bin/env python`. This makes for unexpected results when a
user might be using a virtualenv.

Change-Id: Ibb030f920a8869f9113ade70b66a921cc815060d

It is not safe to call yaml.load with any data received from
an untrusted source, we'd better use yaml.safe_load

Reference:
https://security.openstack.org/guidelines/dg_avoid-dangerous-input-parsing-libraries.html

Change-Id: Ic8bf73bf0f2e2c29eb48094367cf558483be1267

Change-Id: Ibf9ee55c48a5181c5ba3aa46bef485cd3468f14c
Closes-Bug: #1647065

Ansible 2.0 has deprecated the 'ssh' from ansible_ssh_user to become ansible_user.

Change-Id: I15813f638accaa6a31f75dad0b854814264ac493
Closes-Bug: #1635145

Use dict.pop(key, None) instead of del dict[key]. dict.pop will not
raise any error when the key is not exist.

TrivialFix

Change-Id: I3808fc865afe6936a5d93eff792e1d1a93ba2085

In some part in the code we import objects. In the Openstack style
guidelines they recommend to import only modules.

http://docs.openstack.org/developer/hacking/#imports

Change-Id: I4b55cae8c2672f82025a95112b14d441d66f9d6d

- This change indroduces a merge_yaml action_plugin
- This change generates bifrost yaml configs.

Change-Id: I9814e6a5d55cbd46c4b60c06ed70ed54a575bd2f
Implements: blueprint bifrost-support

In links: http://pythonhosted.org/six/, we know 'cStringIO' is
a module of python 2.x, but in python3.x module 'io' has the
same function as 'cStringIO' in python2.x. So here we use
six.stringIO instead.

TrivialFix.

Change-Id: I8022b92b186e239cd76a36dab7766301bf96d2c4

Ansible 2.1.0 has lots of change and the plugin is not compatible
between 2.0.0 and 2.1.0. This change fix the gap.

* fix the signature change for _make_tmp_path in ansible 2.1.0
* fix the common_options in the kolla_docker.py

Change-Id: I05f5f05581c8bd625cd868fa0db549d0c60a7043
Closes-Bug: #1586018

DocImpact

Change-Id: I3cdfbf84919de80f535c030bd146787ecda40dec
partial-Implements: blueprint ansible2

By ignoring the appropriate tests that pep8 does we can properly run
*most* of the pep8 tests on all of our modules allowing for a more
consistent coding style.

Closes-Bug: #1528431
Change-Id: I33f27a250d06d4f044267aa3ad189e092789b8df

Convert config creation from a playbook to an action_plugin. This
reduces the complexity and confusion while retaining the same augment
structure and flexibility.

This allows us to remove the 0-byte files as requirements. They will
still be used if they are present (this means we require additional
documentation around them).

DocImpact
Closes-Bug: #1528430
Change-Id: I2c789f6be9f195c7771ca093a6d59499564b4740