By resetting image_upload_use_cinder_backend to upstream default.

When uploading volume to glance image, cinder looks at the backend's
image_upload_use_cinder_backend config knob to decide whether to try link
the glance image to a cloned volume made by cinder, i.e. by doing all work
locally and only updating glance's locations for the image (when the knob
is set to True). However, after all [1], [2] and [3], which happens since
Victoria, this option requires further config from user (using volume type
with image_service:store_id property (aka extra spec) set to the desired
glance store (even if there is only one cinder store configured).

Please read the bug report as to why the option removal is the
best option (TL;DR it is the most compatible approach).

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/708114
[2] https://review.opendev.org/c/openstack/glance_store/+/746556
[3] https://review.opendev.org/c/openstack/cinder/+/661676

Closes-Bug: #1991516
Change-Id: Ife87ee0241d907a0c407eb21811a354ed1734408

These are upstream defaults, no need to carry them around.

TrivialFix

Change-Id: I2907d5f38c6a74776961bd473553edf2d83f7257

Change-Id: Ib068117237a199db380fcdfb757d5d0e5d34326b

This allows you to use a more descriptive name if you desire.
For example, when using cinder with multiple ceph backends, rbd-1,
doesn't convey much information. You could include location, disk
technology, etc. in the name.

Change-Id: Icfdc2e5726fec8b645d6c2c63391a13c31f2ce9a

This reverts commit 73fc230f.

Reason for revert: CI jobs failing with "msg": "{{ s3_url }}: 's3_url' is undefined"

Change-Id: Iba7099988cea0c0d8254b9e202309cd9c82a984d

Added options to configure S3 cinder backup driver, so cinder backup
can use S3 storage, for safekeeping backups.

Change-Id: Id6ff6206714581555baacecebfb6d8dd53bed8ac

Render {{ openstack_service_workers }} for workers
of each openstack service is not enough. There are
several services which has to have more workers because
there are more requests sent to them.

This patch is just adding default value for workers for
each service and sets {{ openstack_service_workers }} as
default, so value can be overrided in hostvars per server.
Nothing changed for normal user.

Change-Id: Ifa5863f8ec865bbf8e39c9b2add42c92abe40616

Fixes an issue where access rules failed to validate:

    Cannot validate request with restricted access rules. Set
    service_type in [keystone_authtoken] to allow access rule validation

I've used the values from the endpoint. This was mostly a straight
forward copy and paste, except:

- versioned endpoints e.g cinderv3 where I stripped the version
- monasca has multiple endpoints associated with a single service. For
  this, I concatenated logging and monitoring to be logging-monitoring.

Closes-Bug: #1965111
Change-Id: Ic4b3ab60abad8c3dd96cd4923a67f2a8f9d195d7

This patch is removing api related configuration
from service's config files as we are using
apache mod_wsgi and this configuration is not
used.

Change-Id: I69a1542a6f24214fbf6e703782aefb566de4fb26

Following up on [1].
The 3 variables are only introducing noise after we removed
the reliance on Keystone's admin port.

[1] I5099b08953789b280c915a6b7a22bdd4e3404076

Change-Id: I3f9dab93042799eda9174257e604fd1844684c1c

Add an enable_cinder_backend_pure_iscsi and
enable_cinder_backend_pure_fc options to etc/kolla/globals.yml
to enable use of the FlashArray backend.
Update the documentation to include a section on configuring
Cinder with the FlashArray.

Implements: blueprint pure-cinder-driver
Change-Id: I464733f1322237321ed1ffff8636cf30bd1cbb38

An FCD, also known as an Improved Virtual Disk (IVD) or
Managed Virtual Disk, is a named virtual disk independent of
a virtual machine. Using FCDs for Cinder volumes eliminates
the need for shadow virtual machines.
This patch adds Kolla support.

Change-Id: Ic0b66269e6d32762e786c95cf6da78cb201d2765

glance_api_version and os_region_name are removed from cinder.
see: https://docs.openstack.org/cinder/xena/configuration/block-storage/samples/cinder.conf.html

Closes-Bug: #1830997
Change-Id: I751bfe64d47935f183ff2ca891ec56f61e618009

Closes-Bug: #1933025

Change-Id: Ib67d715ddfa986a5b70a55fdda39e6d0e3333162

Following upstream which removed ZFSSA support in Ussuri [1].

[1] https://review.opendev.org/c/openstack/cinder/+/690137

Change-Id: Idb311e18b437fba696759ecb1cf2a6b4803aa5c5

In services which use the Apache HTTP server to service HTTP requests,
there exists a TimeOut directive [1] which defaults to 60 seconds. APIs
which come under heavy load, such as Cinder, can sometimes exceed this
which results in a HTTP 504 Gateway timeout, or similar. However, the
request can still be serviced without error. For example, if Nova calls
the Cinder API to detach a volume, and this operation takes longer
than the shortest of the two timeouts, Nova will emit a stack trace
with a 504 Gateway timeout. At some time later, the request to detach
the volume will succeed. The Nova and Cinder DBs then become
out-of-sync with each other, and frequently DB surgery is required.

Although strictly this category of bugs should be fixed in OpenStack
services, it is not realistic to expect this to happen in the short
term. Therefore, this change makes it easier to set the Apache HTTP
timeout via a new variable.

An example of a related bug is here:

https://bugs.launchpad.net/nova/+bug/1888665

Whilst this timeout can currently be set by overriding the WSGI
config for individual services, this change makes it much easier.

Change-Id: Ie452516655cbd40d63bdad3635fd66693e40ce34
Closes-Bug: #1917648

The Cinder API log is currently written to a file called
cinder-wsgi.log, and the WSGI logs to cinder-api.log. Fluentd
then tries to parse the WSGI log as an OpenStack log which
results in 'got incomplete line' errors and prevents proper
ingestion of these logs.

Co-Authored-By: yaoning <yaoning@unitedstack.com>
Closes-Bug: 1916752
Change-Id: I3296dcc4780160cbf88bd18285571276f58bb249

Change-Id: I52cee3679e4a733daa165062d64884577e9acc1a
Closes-Bug: #1900082

When the internal VIP is moved in the event of a failure of the active
controller, OpenStack services can become unresponsive as they try to
talk with MariaDB using connections from the SQLAlchemy pool.

It has been argued that OpenStack doesn't really need to use connection
pooling with MariaDB [1]. This commit reduces the use of connection
pooling via two configuration options:

- max_pool_size is set to 1 to allow only a single connection in the
  pool (it is not possible to disable connection pooling entirely via
  oslo.db, and max_pool_size = 0 means unlimited pool size)
- lower connection_recycle_time from the default of one hour to 10
  seconds, which means the single connection in the pool will be
  recreated regularly

These settings have shown better reactivity of the system in the event
of a failover.

[1] http://lists.openstack.org/pipermail/openstack-dev/2015-April/061808.html

Change-Id: Ib6a62d4428db9b95569314084090472870417f3d
Closes-Bug: #1896635

This change adds support for encryption of communication between
OpenStack services and RabbitMQ. Server certificates are supported, but
currently client certificates are not.

The kolla-ansible certificates command has been updated to support
generating certificates for RabbitMQ for development and testing.

RabbitMQ TLS is enabled in the all-in-one source CI jobs, or when
The Zuul 'tls_enabled' variable is true.

Change-Id: I4f1d04150fb2b5af085b762890092f87ae6076b5
Implements: blueprint message-queue-ssl-support

It was found to be useless in [1].

It is one of distro_python_version usages.

Note Freezer and Horizon still use python_path (and hence
distro_python_version) for different purposes.

[1] https://review.opendev.org/675822

Change-Id: I6d6d9fdf4c28cb2b686d548955108c994b685bb1
Partially-Implements: blueprint drop-distro-python-version

The goal for this push request is to normalize the construction and use
 of internal, external, and admin URLs. While extending Kolla-ansible
 to enable a more flexible method to manage external URLs, we noticed
 that the same URL was constructed multiple times in different parts
 of the code. This can make it difficult for people that want to work
 with these URLs and create inconsistencies in a large code base with
 time. Therefore, we are proposing here the use of
 "single Kolla-ansible variable" per endpoint URL, which facilitates
 for people that are interested in overriding/extending these URLs.

As an example, we extended Kolla-ansible to facilitate the "override"
of public (external) URLs with the following standard
"<component/serviceName>.<companyBaseUrl>".
Therefore, the "NAT/redirect" in the SSL termination system (HAproxy,
HTTPD or some other) is done via the service name, and not by the port.
This allows operators to easily and automatically create more friendly
 URL names. To develop this feature, we first applied this patch that
 we are sending now to the community. We did that to reduce the surface
  of changes in Kolla-ansible.

Another example is the integration of Kolla-ansible and Consul, which
we also implemented internally, and also requires URLs changes.
Therefore, this PR is essential to reduce code duplicity, and to
facility users/developers to work/customize the services URLs.

Change-Id: I73d483e01476e779a5155b2e18dd5ea25f514e93
Signed-off-by: Rafael Weingärtner <rafael@apache.org>

This patch introduces a global keep alive timeout value for services
that leverage httpd + wsgi to handle http/https requests. The default
value is one minute.

Change-Id: Icf7cb0baf86b428a60a7e9bbed642999711865cd
Partially-Implements: blueprint add-ssl-internal-network

The Castellan (Barbican client) has different parameters to control
the used CA file.
This patch uses them.
Moreover, this aligns Barbican with other services by defaulting
its client config to the internal endpoint.

See also [1].

[1] https://bugs.launchpad.net/castellan/+bug/1876102

Closes-Bug: #1886615

Change-Id: I6a174468bd91d214c08477b93c88032a45c137be

The etcd service protocol is currently configured with internal_protocol.
The etcd service is not load balanced by a HAProxy container, so
there is no proxy layer to do TLS termination when internal_protocol
is configured to be "https".

Until the etcd service is configured to deploy with native TLS
termination, the etcd uses should be independent of
internal_protocol, and "http" by default.

Change-Id: I730c02331514244e44004aa06e9399c01264c65d
Closes-Bug: 1884137

When installing kolla with external ceph, ceph_cinder_user
var has to be set per documentation instead of ceph_cinder_volume_user.
This value is also rendered in example etc/kolla/globals.yml file.

This patch is fixing this bug or, let's say typo.

Change-Id: Id82b07867f4bc0e5d5e56363f0122014df6892bc

Change-Id: I4673f436d8943e6fce7e579446c27ec8215b7346

The use of default(omit) is for module parameters, not templates. We
define a default value for openstack_cacert, so it should never be
undefined anyway.

Change-Id: Idfa73097ca168c76559dc4f3aa8bb30b7113ab28

Change-Id: I9395ae32378f4ff1fd57be78d7daec7745579e04
Closes-Bug: #1869133

Currently the WSGI configuration for binary images uses python2.7
site-packages in some places. This change uses distro_python_version to
select the correct python path.

Change-Id: Id5f3f0ede106498b9264942fa0399d7c7862c122
Partially-Implements: blueprint python-3

To make the configuration easier for the user, and to allow non-standard
ceph authentication ids - introduce ceph_*_user variables.

Change-Id: I24e01c43c826b62b6748d93a498f4b7d8ce9e309

Introduce user modifiable variables instead of fixed-names
of Ceph keyring files for external Ceph functionality.

Change-Id: I1a33b3f9d6eca5babf53b91187461e43aef865ce

Include a reference to the globally configured Certificate Authority to
all services. Services use the CA to verify HTTPs connections.

Change-Id: I38da931cdd7ff46cce1994763b5c713652b096cc
Partially-Implements: blueprint support-trusted-ca-certificate-file

Currently External Ceph Cinder config requires the user to create cinder
service custom configuration.

This change alters the if/else statements to template out cinder backends
configuration when cinder_backend_ceph is True.

Change-Id: I143c3b44d2839e56d1dbf28484c0eaae0a753dc9

To fix instability and availability issues:

etcd3 is not available in repos for binary kolla images.

etcd3 does not support eventlet-based services [1].

[1] https://review.opendev.org/466098

Change-Id: I430bab735da204fc81696130b17931a89214c876
Closes-bug: #1852086
Closes-bug: #1854932

Change-Id: I51144d92f34ed51c499a4119c059e6475d02eb46

Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689



Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Both ubuntu source and binary install type support python3 now,
python_path should be updated.

Depends-On: https://review.opendev.org/675581
Partially Implements: blueprint python3-support

Change-Id: I4bf721b44220bde2d25d4d985f5ca411699a5a72

This is to allow operator to prevent enabling redis and/or
etcd from magically configuring cinder coordinator.

Note this change is backwards-compatible.

Change-Id: Ie10be55968e43e3b9cc347b1b58771c1f7b1b910
Related-Bug: #1840070
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>

The Hitachi NAS Platform iSCSI driver was marked as not supported by
Cinder in the Ocata realease[1].

[1] https://review.opendev.org/#/c/444287/

Change-Id: I1a25789374fddaefc57bc59badec06f91ee6a52a
Closes-Bug: #1832821