Moving the CLI to python allows for easier
maintenance and larger feature-set.

This patch introduces a few breaking changes!
The changes stem the nature of the cliff package.
- the order of parameters must be
  kolla-ansible <action> <arguments>
- mariadb_backup and mariadb_recovery now are
  mariadb-backup and mariadb-recovery

Closes-bug: #1589020
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>
Change-Id: I9749b320d4f5eeec601a055b597dfa7d8fb97ce2

This avoids the need to use a proxy, or some other means, to connect to
Prometheus. This is disabled by default and can be enabled by setting
enable_prometheus_server_external to true.

Change-Id: Ia0af044ff436c2a204b357750a16ff49fcdfec45

Add support for automatic provisioning and renewal of HTTPS
certificates via LetsEncrypt.

Spec is available at:
https://etherpad.opendev.org/p/kolla-ansible-letsencrypt-https

Depends-On: https://review.opendev.org/c/openstack/kolla/+/887347


Co-Authored-By: Michal Arbet <michal.arbet@ultimum.io>
Implements: blueprint letsencrypt-https
Change-Id: I35317ea0343f0db74ddc0e587862e95408e9e106

The kolla-genpwd, kolla-mergepwd, kolla-readpwd and kolla-writepwd
commands now creates or updates passwords.yml with correct
permissions. Also they display warning message about incorrect
permissions.

Closes-Bug: #2018338
Change-Id: I4b50053ced9150499d1d09fd4a0ec2e243cf938b
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

From:

(kolla) 13:11 (s) marcin@puchatek:kolla-ansible$ kolla-genpwd
Traceback (most recent call last):
  File "/home/marcin/.virtualenvs/kolla/bin/kolla-genpwd", line 8, in <module>
    sys.exit(main())
  File "/home/marcin/.virtualenvs/kolla/lib/python3.10/site-packages/kolla_ansible/cmd/genpwd.py", line 135, in main
    genpwd(passwords_file, length, uuid_keys, ssh_keys, blank_keys,
  File "/home/marcin/.virtualenvs/kolla/lib/python3.10/site-packages/kolla_ansible/cmd/genpwd.py", line 59, in genpwd
    with open(passwords_file, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/etc/kolla/passwords.yml'

To:

(kolla) 13:17 (s) marcin@puchatek:kolla-ansible$ kolla-genpwd
ERROR: Passwords file "/etc/kolla/passwords.yml" is missing

Change-Id: I18a9559daeb3d124a03dcb735ebb01a2cf24f617

This key can be used by users in networking-generic-switch
scenario instead of adding cleartext password in ml2_conf.ini.

Change-Id: I10003e6526a55a97f22678ab81c411e4645c5157

This commit adds two new cli commands to allow an operator
to read and write passwords into a configured Hashicorp Vault
KV.

Change-Id: Icf0eaf7544fcbdf7b83f697cc711446f47118a4d

As announced on the openstack-discuss ML[1], Karbor is retiring
this cycle (Wallaby).

Needed-By: https://review.opendev.org/c/openstack/karbor/+/767032

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018643.html

Change-Id: I222cf302e507f6a9de0347c79ec536aa7be22bb6

this patchset has implemented:
  - network (lb-mgmt-net)
  - security groups and rules (used by amphora and health manager)
  - amphora flavor (used by amphora)
  - nova keypair (used by amphora at the time of debugging)

Add a octavia_amp_listen_port variable which used by amphora
Add amp_image_owner_id in octavia.conf

Implements: blueprint implement-automatic-deploy-of-octavia
Co-Authored-By: zhangchun <zhangchun@yovole.com>

Depends-On: https://review.opendev.org/652030

Change-Id: I67009d046925cfc02c1e0073c80085c1471975f6

The kolla-genpwd and kolla-mergepwd commands can be used to manipulate
the kolla passwords.yml file. The format is a YAML encoded dict of
password variable names to their values. If the format is not a dict,
the error messages are unhelpful.  In particular, this can happen if the
file is encrypted e.g. via Ansible Vault.

For kolla-genpwd:

    AttributeError: 'NoneType' object has no attribute 'items'

For kolla-mergepwd:

    AttributeError: 'NoneType' object has no attribute 'update'

This change adds a more friendly message.

Change-Id: I27f0835b904e05006ae401adf383090322e1b891
Closes-Bug: #1880220

W503 and W504 are incompatible and we need to choose one of them.
Existing codes follows W503, so we disable W504.

Change-Id: Ic745e956dd332eb0fa49b93c1e6acb12f8a7f26c

Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].

This change removes the Ansible code and associated CI jobs.

[1]: https://review.opendev.org/669214

Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2

to clean old keys on merge.

Change-Id: Ifcc99e7c737707eea9e951db066dc94fd85bd9f7

The method `Fernet.generate_key()` generates a binary string in Python 3:
```
>>> Fernet.generate_key()
b'qSMZlOK23pZUw_Uyy-ZRPUfPskMXKGCGmhG6AHCFiV8='
```

Unless properly written as a string to the Kolla `passwords.yml` file,
the Fernet key will end up in the final Barbican config like this:
```
[simple_crypto_plugin]
kek = b'qSMZlOK23pZUw_Uyy-ZRPUfPskMXKGCGmhG6AHCFiV8='
```

Due to the fact that the key is incorrectly written to the barbican
config file (it should be written as a string), every barbican secret
store fails with:

```
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/store_crypto.py", line 83, in store_secret
barbican.api.controllers     encrypting_plugin, context.project_model)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/store_crypto.py", line 290, in _find_or_create_kek_objects
barbican.api.controllers     kek_meta_dto = plugin_inst.bind_kek_metadata(kek_meta_dto)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/crypto/simple_crypto.py", line 104, in bind_kek_metadata
barbican.api.controllers     encryptor = fernet.Fernet(self.master_kek)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/cryptography/fernet.py", line 38, in __init__
barbican.api.controllers     "Fernet key must be 32 url-safe base64-encoded bytes."
barbican.api.controllers ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
```

This commit fixes the issue described above by properly writing
the Fernet key as a string to the Kolla `passwords.yml` file.

Closes-Bug: #1848191
Change-Id: I27fc0159c889bc2e1576fdd69b7d02a320b620f8

When methods for passwords generation and merge are
extracted then external apps and scripts can use
those methods without resolving to subprocess execution
or injecting sys.argv.

Change-Id: I99aff7852180534129fa36859075306eea776ba9
Signed-off-by: Maciej Kucia <maciej@kucia.net>

Sha password is not always valid for barbican cripto key.
Use a fernet key so it always gets valid.

Not need release note for upgrade, users with a working
barbican not regenerate passwords, only new passwords will
get new type.

Change-Id: Ic8c4ca63219295d697062cff9cbf30fadbe49bf3

Due to the changes in hmac.new and how binary strings
are dumped in yaml.safe_dump some changes were needed to
make sure that we dumped only strings, not binary strings.

Change-Id: Ic2fbcf2347023c1e9e666203dfe40dbeaf24ce5f

Change-Id: I87df49939f600cfa1041193808ce6bdcf4620ffc

OSprofile allows user/devs trace OpenStack requests.

Implements: blueprint enable-osprofiler
Co-Authored-By: Bertrand Lallau <bertrand.lallau@gmail.com>
Change-Id: I82ea85d726011ef6cbf99380f395452d6d7f8053

pycrypto is no longer maintained [1]. This patch rewrites functions
using pycrypto and replaces them with the cryptography equivalent

[1] http://lists.openstack.org/pipermail/openstack-dev/2017-March/113568.html

Change-Id: I375b5876ec2f4c4f32b9f6b3f41d209a59a0f615

Remove Python specific types from YAML output
Produce only basic YAML tags

Change-Id: Ib6a4c18663897efb7243ed1ff84df1c9f2abf8bf

Link https://docs.openstack.org/project-install-guide/key-manager/newton/barbican-backend.html#simple-crypto-plugin

Change-Id: I351738c2a98090c56ac69e477fbe5ddec4cc5b26
Closes-Bug: #1672001

Booting from volume require cinder's ceph client secret now. Move cinder
before nova in site.yml, because nova depends on cinder ceph client key
now.

Change-Id: I01c9ed80843d98305b8963894c4917c21a35d3ac
Closes-Bug: #1670676

* Rename kolla namespace to kolla_ansible
* remove oslo.config.opts entry points which is uesless
* delete useless tools/version-check.py script

Change-Id: I005dd7223ff23afbb2ce8cbfd0ebec0969102798