This avoids the need to use a proxy, or some other means, to connect to
Prometheus. This is disabled by default and can be enabled by setting
enable_prometheus_server_external to true.

Change-Id: Ia0af044ff436c2a204b357750a16ff49fcdfec45

Add support for automatic provisioning and renewal of HTTPS
certificates via LetsEncrypt.

Spec is available at:
https://etherpad.opendev.org/p/kolla-ansible-letsencrypt-https

Depends-On: https://review.opendev.org/c/openstack/kolla/+/887347


Co-Authored-By: Michal Arbet <michal.arbet@ultimum.io>
Implements: blueprint letsencrypt-https
Change-Id: I35317ea0343f0db74ddc0e587862e95408e9e106

The kolla-genpwd, kolla-mergepwd, kolla-readpwd and kolla-writepwd
commands now creates or updates passwords.yml with correct
permissions. Also they display warning message about incorrect
permissions.

Closes-Bug: #2018338
Change-Id: I4b50053ced9150499d1d09fd4a0ec2e243cf938b
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>

From:

(kolla) 13:11 (s) marcin@puchatek:kolla-ansible$ kolla-genpwd
Traceback (most recent call last):
  File "/home/marcin/.virtualenvs/kolla/bin/kolla-genpwd", line 8, in <module>
    sys.exit(main())
  File "/home/marcin/.virtualenvs/kolla/lib/python3.10/site-packages/kolla_ansible/cmd/genpwd.py", line 135, in main
    genpwd(passwords_file, length, uuid_keys, ssh_keys, blank_keys,
  File "/home/marcin/.virtualenvs/kolla/lib/python3.10/site-packages/kolla_ansible/cmd/genpwd.py", line 59, in genpwd
    with open(passwords_file, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/etc/kolla/passwords.yml'

To:

(kolla) 13:17 (s) marcin@puchatek:kolla-ansible$ kolla-genpwd
ERROR: Passwords file "/etc/kolla/passwords.yml" is missing

Change-Id: I18a9559daeb3d124a03dcb735ebb01a2cf24f617

This key can be used by users in networking-generic-switch
scenario instead of adding cleartext password in ml2_conf.ini.

Change-Id: I10003e6526a55a97f22678ab81c411e4645c5157

As announced on the openstack-discuss ML[1], Karbor is retiring
this cycle (Wallaby).

Needed-By: https://review.opendev.org/c/openstack/karbor/+/767032

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018643.html

Change-Id: I222cf302e507f6a9de0347c79ec536aa7be22bb6

this patchset has implemented:
  - network (lb-mgmt-net)
  - security groups and rules (used by amphora and health manager)
  - amphora flavor (used by amphora)
  - nova keypair (used by amphora at the time of debugging)

Add a octavia_amp_listen_port variable which used by amphora
Add amp_image_owner_id in octavia.conf

Implements: blueprint implement-automatic-deploy-of-octavia
Co-Authored-By: zhangchun <zhangchun@yovole.com>

Depends-On: https://review.opendev.org/652030

Change-Id: I67009d046925cfc02c1e0073c80085c1471975f6

The kolla-genpwd and kolla-mergepwd commands can be used to manipulate
the kolla passwords.yml file. The format is a YAML encoded dict of
password variable names to their values. If the format is not a dict,
the error messages are unhelpful.  In particular, this can happen if the
file is encrypted e.g. via Ansible Vault.

For kolla-genpwd:

    AttributeError: 'NoneType' object has no attribute 'items'

For kolla-mergepwd:

    AttributeError: 'NoneType' object has no attribute 'update'

This change adds a more friendly message.

Change-Id: I27f0835b904e05006ae401adf383090322e1b891
Closes-Bug: #1880220

W503 and W504 are incompatible and we need to choose one of them.
Existing codes follows W503, so we disable W504.

Change-Id: Ic745e956dd332eb0fa49b93c1e6acb12f8a7f26c

Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].

This change removes the Ansible code and associated CI jobs.

[1]: https://review.opendev.org/669214

Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2

The method `Fernet.generate_key()` generates a binary string in Python 3:
```
>>> Fernet.generate_key()
b'qSMZlOK23pZUw_Uyy-ZRPUfPskMXKGCGmhG6AHCFiV8='
```

Unless properly written as a string to the Kolla `passwords.yml` file,
the Fernet key will end up in the final Barbican config like this:
```
[simple_crypto_plugin]
kek = b'qSMZlOK23pZUw_Uyy-ZRPUfPskMXKGCGmhG6AHCFiV8='
```

Due to the fact that the key is incorrectly written to the barbican
config file (it should be written as a string), every barbican secret
store fails with:

```
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/store_crypto.py", line 83, in store_secret
barbican.api.controllers     encrypting_plugin, context.project_model)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/store_crypto.py", line 290, in _find_or_create_kek_objects
barbican.api.controllers     kek_meta_dto = plugin_inst.bind_kek_metadata(kek_meta_dto)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/crypto/simple_crypto.py", line 104, in bind_kek_metadata
barbican.api.controllers     encryptor = fernet.Fernet(self.master_kek)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/cryptography/fernet.py", line 38, in __init__
barbican.api.controllers     "Fernet key must be 32 url-safe base64-encoded bytes."
barbican.api.controllers ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
```

This commit fixes the issue described above by properly writing
the Fernet key as a string to the Kolla `passwords.yml` file.

Closes-Bug: #1848191
Change-Id: I27fc0159c889bc2e1576fdd69b7d02a320b620f8

When methods for passwords generation and merge are
extracted then external apps and scripts can use
those methods without resolving to subprocess execution
or injecting sys.argv.

Change-Id: I99aff7852180534129fa36859075306eea776ba9
Signed-off-by: Maciej Kucia <maciej@kucia.net>

Sha password is not always valid for barbican cripto key.
Use a fernet key so it always gets valid.

Not need release note for upgrade, users with a working
barbican not regenerate passwords, only new passwords will
get new type.

Change-Id: Ic8c4ca63219295d697062cff9cbf30fadbe49bf3

Due to the changes in hmac.new and how binary strings
are dumped in yaml.safe_dump some changes were needed to
make sure that we dumped only strings, not binary strings.

Change-Id: Ic2fbcf2347023c1e9e666203dfe40dbeaf24ce5f

Change-Id: I87df49939f600cfa1041193808ce6bdcf4620ffc

OSprofile allows user/devs trace OpenStack requests.

Implements: blueprint enable-osprofiler
Co-Authored-By: Bertrand Lallau <bertrand.lallau@gmail.com>
Change-Id: I82ea85d726011ef6cbf99380f395452d6d7f8053

pycrypto is no longer maintained [1]. This patch rewrites functions
using pycrypto and replaces them with the cryptography equivalent

[1] http://lists.openstack.org/pipermail/openstack-dev/2017-March/113568.html

Change-Id: I375b5876ec2f4c4f32b9f6b3f41d209a59a0f615

Remove Python specific types from YAML output
Produce only basic YAML tags

Change-Id: Ib6a4c18663897efb7243ed1ff84df1c9f2abf8bf

Link https://docs.openstack.org/project-install-guide/key-manager/newton/barbican-backend.html#simple-crypto-plugin

Change-Id: I351738c2a98090c56ac69e477fbe5ddec4cc5b26
Closes-Bug: #1672001

Booting from volume require cinder's ceph client secret now. Move cinder
before nova in site.yml, because nova depends on cinder ceph client key
now.

Change-Id: I01c9ed80843d98305b8963894c4917c21a35d3ac
Closes-Bug: #1670676

* Rename kolla namespace to kolla_ansible
* remove oslo.config.opts entry points which is uesless
* delete useless tools/version-check.py script

Change-Id: I005dd7223ff23afbb2ce8cbfd0ebec0969102798

Implement ansible role to deploy designate
and dependencies. The backend used is bind9.

Co-Authored-By: zhubingbing <zhubingbing10@gmail.com>
Co-Authored-By: Eduardo Gonzalez <dabarren@gmail.com>

Depends-On: 6d0dc3e0f931c7c50b64a4659900cc50b0d860a2
Implements: blueprint ansible-designate
Change-Id: I34d8126e0cd8d71d5ced9b62f3776cc354fbb549

Co-Authored-By: zhubingbing <zhubingbing10@gmail.com>
Co-Authored-By: zhangshuai <446077695@qq.com>
Co-Authored-By: Mauricio Lima <mauriciolimab@gmail.com>

Change-Id: I7aebe544e3495767d6389dbf220d633a98c137da
Partially-implements: bp karbor-ansible-role

Closes-Bug: #1082248
Change-Id: I9e52619a6805fa58ffdb396b76a6ca99e2376a7d

Consistently set the search path to prefer PROJECT_ROOT as the source
of packages to import if we are using local tools instead of pip
installed kolla tools.

TrivialFix

Change-Id: Idd1d98bb82a8f1ddb32d1bdb23f346b461534b9f

Partially-Implements: blueprint ansible-gnocchi

Change-Id: I8dd0460bd21ac0a233fab0142ec7b6079459bdc2

- This change extend the genpwd.py
  command to generate an ssh key pair
  bifrost.
- This change bifrost config and bootstrap
  task to install the generated keys.
- This change updates the bifrost guide to
  discribe how to provide your own key.

Change-Id: I05243f58843d9195cace253dff5628fae89c78e8
Implements: blueprint bifrost-support

* Inspected each error and fixed / added nosec where appropriate.
* build-swift-ring.py which was throwing sec errors is no longer used so
  removed it.
* Removed the dev/ directory from being checked.

Closes-Bug: #1617713
Change-Id: I25664cabca4137e5c9f499c1af3f5ce78b86fb56

This addresses the ansible aspects of fernet key bootstrapping as
well as distributed key rotation.

- Bootstrapping is handled in the same way as keystone bootstrap.
- A new keystone-fernet and keystone-ssh container is created to allow
  the nodes to communicate with each other (taken from nova-ssh).
- The keystone-fernet is a keystone container with crontab installed.
  This will handle key rotations through keystone-manage and trigger
  an rsync to push new tokens to other nodes.
- Key rotation is setup to be balanced across the keystone nodes using
  a round-robbin style. This ensures that any node failures will not
  stop the keys from rotating. This is configured by a desired token
  expiration time which then determines the cron scheduling for each
  node as well as the number of fernet tokens in rotation.
- Ability for recovered node to resync with the cluster. When a node
  starts it will run sanity checks to ensure that its fernet tokens
  are not stale. If they are it will rsync with other nodes to ensure
  its tokens are up to date.

The Docker component is implemented in:
  https://review.openstack.org/#/c/349366

Change-Id: I15052c25a1d1149d364236f10ced2e2346119738
Implements: blueprint keystone-fernet-token

Change-Id: Ie70db1b18a73528b1194e4fbcf53d09fed20f4a0
Implements: blueprint kolla-host

partially implements blueprint multiple-clouds

Change-Id: I676c4245e6f058ffbed345970ee78d1750dd0f2f

Add a nova-ssh container to handle the `nova migrate` and
`nova resize` case, in which the nova will use ssh to copy
files between machines.

Change-Id: Ie6675943f3aeabfbba8589d308d55b9c89d732db
Closes-Bug: #1562141

As with all tools, this is a first pass at the generation. Perhaps we
even want to move this into kolla/kolla/cmd and be generated with tox
itself in the future.

This tool, when run, will only populate empty fields that have no
values meaning that it is safe to run repeatedly on the same file.

Of note, there is no way to preserve comments in the file after it has
been processed by the yaml parser in python. Comments and sections
will remain in the passwords.yml template for additional documentation
if the user wishes to populate the file themselves.

Use SystemRandom and clean up the docs a bit to not use pronouns.

Co-Authored-By: Steven Dake <stdake@cisco.com>

Closes-Bug: #1559266
Change-Id: I2932d592df8871f1b7811059206d0b4d0553a687