OSprofile allows user/devs trace OpenStack requests.

Implements: blueprint enable-osprofiler
Co-Authored-By: Bertrand Lallau <bertrand.lallau@gmail.com>
Change-Id: I82ea85d726011ef6cbf99380f395452d6d7f8053

In case Kolla's users want to deploy with both of
binary and source image, we should have a variable
install type that define install type for each project.

We also add specific image tag for each Openstack project.

This commit is implemented for Ceilometer, Ceph
Cinder, Cloudkitty and Congress projects.

Change-Id: I560ab84545db2e034abe03613c4e7cb407e64ab2
Implements: blueprint mixing-binary-and-source-image

Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455

cinder-volume will convert the disk locally when boot image from volume
with create new volume. whereas the container only have 10G space in
default, cinder-volume will complain that no space if image is large.

This patch mount cinder volume for cinder-volume and cinder-backup all
the time.

Change-Id: I8c68ea49374dc67839699b81e345cfc390a01fe3
Close-Bug: #1692485

In someplace in the code admin user, project and domain name
are hardcoded.

This change use the correct variable defined at:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L278

Change-Id: I3a4c78d81733bf7a2f3c67d259ec5c5f5bb38d3f

This changes moves cinder-api to make use of apache to
launch its processes.
Ubuntu binary removed support in Ocata for cinder-api
system services [1]. At this moment cinder in Ocata branch
is not working at all due this issue.

This changes aims to fix cinder in master only for ubuntu
binary with the purpose of make a clean backport to stable/ocata
without do any other change on the backport process.
A follow up change will migrate other OS/install_type to make use
of Apache but only changing master branch.

[1] http://lists.openstack.org/pipermail/openstack-dev/2017-January/111069.html

Change-Id: I9885d6acadbac1da46844f583e7cd37785505ae0
Closes-Bug: #1676751

Cinder backup requires iSCSI volume and host mount for backing
any iSCSI volume, not only from LVM backend. 'enable_iscsid' is
a catch-all setting for all iSCSI backends that also runs
necessary iSCSI client on the same host.

Change-Id: Ic486a9f72820df2e0a48a18166f043e5ddb5bd0a
Closes-Bug: #1688519

wait_for module waits 300 seconds for the port started or stopped.  This
is meaningless and useless in precheck. This patch change timeout to 1
seconds.

Change-Id: I9b251ec4ba17ce446655917e8ef5e152ef947298
Closes-Bug: #1688152

If used external ceph for cinder,the ceph storage not enable cephx.
So ceph keyring file not does not exist.
Task throw warning of copy over ceph keyring files.

Change-Id: Ibb3cd536c982ad9dede3c2a0ecfc02cdf6e4c1b9
Closes-Bug: #1684461

When using cinder lvm backend storage driver, lvcreate will load dm_*
linux modules when need. So /lib/modules is needed by cinder_volume
container.

Change-Id: I80985794d6aaa6b4ddc71680a97e0d07085dafa7
Closes-Bug: #1684378

Currently Cinder configuration hard-codes tgtadm as the only
iSCSI management utility. tgtd and tgtadm are older tools to
manage iSCSI volumes. Newer versions of Linux distros use
targetcli and kernel based iSCSI facilities (LIO) and can be
a preferred way for some users.

Change-Id: I10c0a7d596994a9201218e58ca2ad184774a8ce2
Closes-Bug: #1684139

Currently if Ceph is used for Cinder volumes the only choice of
backup driver is Ceph. It should be possible to use any backup
driver for any volume type, e.g. iSCSI volumes can go to Ceph
based backup storage.

Change-Id: Ifc8a767b9030391580de6a24383bb6b87053835d
Closes-Bug: #1683715

Change-Id: Ib684e3f4bcd26e40c5bfde9bec8a1d83563d5460
Signed-off-by: Pavel Glushchak <pglushchak@virtuozzo.com>

cinder-volume will be hanged if rados_connect_timeout is disabled. And
no error message will be thrown. Disable timeout is not a good idea.

This patch sets the rados_connect_timeout to 5.

Change-Id: I05e7343d09d75399a7031013b4329a2cb81792fb
Closes-Bug: #1676267

more infomation, see http://paste.openstack.org/show/603369/

Change-Id: If803071bfa88b8e5275b3291c05dc3a9f6e08222
Closes-Bug: #1674215

Change-Id: I2658259d88a8df1c0d716ed6f360f9514fb957b5
Closes-Bug: #1673703

Option "enable_v1_api" from group "DEFAULT" is deprecated.
This patch remove it.

Change-Id: I234e5f417ca310b60ca5e4990223f9d13f583894

Booting from volume require cinder's ceph client secret now. Move cinder
before nova in site.yml, because nova depends on cinder ceph client key
now.

Change-Id: I01c9ed80843d98305b8963894c4917c21a35d3ac
Closes-Bug: #1670676

* Move the tasks to the role
* Skip the task when container is already running

Change-Id: I1990d4dd2a02efa2b3766329000aa23419e0ff17
Closes-Bug: #1670286

Change-Id: I2e050d811a26ac83f2336d25b02952a9e9835142
Closes-Bug: #1662625

Co-Authored-By: Jeffrey Zhang <jeffrey.zhang@99cloud.net>
Change-Id: I92c0757af6567cbb4ff338493df9931ebef3366c

Change-Id: I1577cc3afef4dadd3a188c8ba749c9cdfad313ae

We do not use *.j2 filename pattern in custom config folder. Add
nfs_shares without .j2 extension for consistency. And do not remove the
.j2 extension file for back compatibility.

Change-Id: Ic66c2f3188e818b9dff4d693102f0d433d2695ac

Usernames can be configured with variables in
configuration files, but user creation is hardcoded.

Change-Id: I057cfb921d776217db66f59226dcfa79f3eb7368
Closes-Bug: #1661587

Change-Id: I548cb6cbc718864cffb95b73376a3e138eef0a6d
Closes-Bug: #1660961

Co-Authored-By: Mauricio Lima <mauriciolimab@gmail.com>
Change-Id: I9a4a6b6523dee4b388513386b7d85d421f2b7b89

Co-Authored-By: Mauricio Lima <mauriciolimab@gmail.com>
Change-Id: I8718c4a63f1d645f3e947f8ce90d196f9cbfd91a
Partially-implements: blueprint better-reconfigure

Change-Id: I5910da875eba9f9b9a18d27fcb69bb38ad67bad0

At present, cinder/nova/glance/gnocchi relative containers's
ceph.conf aren't be merge from user custom's config.
In some condition, we should add extra parameter to custom's
ceph.conf, for example:rbd_default_features = 1.

So, it is necessary to use merge_configs instead of template.
Closes-Bug: #1656162

Change-Id: I824e0c68af270b85c52382ae35987213266fc6f6

Change-Id: I00d2dcb0895548ba169ab85764cf546c2214cbf5

Change-Id: I63197f8c5646e44a9a7287e644c904a1e227af23

Cinder can be configured to support either
lvm or ceph backend. Using both backends only enables lvm
as the ceph backend configuration is combined
with the lvm backend part via elif code.

Change-Id: I57e515cf7cdd6377d60bfbc45a889553ce87207a
Closes-Bug: #1654996

And remove the unnecessary blanks.

Change-Id: Id2b5f8119c446b36c91ec0e63246c90a60459e24
Partially-implements: blueprint condition-pre-check

Change-Id: I6ca1957c1518d8dfe6a15b22e76c0b9a52e2f1c2

Include custom policy.json files in service-api.json.j2 files

Change-Id: Ic55bfc6f61131aa72c3497ce8b2282056bcc7f92
Partially-Implements: blueprint custom-policies

Notifications for cinder is not disabled when there is no other consuming
project, it's better to disable that to reduce some pressure for the message
bus[1].

[1]:
http://docs.openstack.org/developer/oslo.messaging/FAQ.html#i-don-t-need-notifications-on-the-message-bus-how-do-i-disable-them

Change-Id: I41c16b4ef6437b0104d4f37daa8b8640db7fdd50
Closes-bug: #1642440

This commit brings necessary configuration and container start-up
options to use two popular backends for backup - nfs and swift.
Both drivers support incremental backups and should work with any
cinder volume backend.

Change-Id: Ieed29a895dc1b9faad196e160a88ea9d293f9c75

Currently, policy.json is put in
"{{ node_config_directory }}/{{ service_name }}"
in target nodes.

Relocation policy.json to "{{ node_config_directory }}/{{ item }}"
with item is corresponding service compoment config directory.

Currently, the policy.json is copied to all services, but it
should be reviewed and left only in neccesary service
(at many cases, only API service needs that).

Redundant files will be removed in follow up patchset.

Change-Id: I0e997dccf4ec438c9c0436db71ec2fd06650f50d
Closes-Bug: #1639686

Nova and Cinder used a list of glance api servers - this list
does not provide a proper high availability because the servers
are connected at random without recalling who is failed last
time. E.g. out of three controllers with a failed first one nova
and cinder can try connection to the first controller because of
random / shuffled choice of the glance server. A proper solution
is to use VIP that connects to alive server from the beginning.
Also as workaround one could significantly increase max number
of retries in configuration files in hope that a random function
will choose a healthy glance server sometime - not a good choice.

Change-Id: Ifaf8ffe3697ec88a6da4c2b43c83975b63dc2e8c
Closes-Bug: #1643509

Needs to enable notifications for services when enabling searchlight
[1][2][3][4]

[1]
http://docs.openstack.org/developer/searchlight/plugins/cinder.html#cinder-conf
[2]
http://docs.openstack.org/developer/searchlight/plugins/glance.html#glance-configuration
[3]
http://docs.openstack.org/developer/searchlight/plugins/neutron.html#neutron-configuration
[4]
http://docs.openstack.org/developer/searchlight/plugins/nova.html#nova-configuration

Change-Id: Id2167d901ac3d65599e54feb1b2d1818d2a633c9
Closes-bug: #1643393