Change-Id: Ic941a396b5cd9abfb5e9941218e91b784f8bba0a
Implements: bp senlin-container

Partially-Implements: blueprint barbican-ansible

Change-Id: Id6be35b1d0527d5c38d4ea8576b233ebcc404718

Manila endpoints (internal and external) should be created in
haproxy configuration just like other services.

Change-Id: I5dbc6ca94a118b9655e1c5a87b0a5163153ab5af
Closes-Bug: #1621556

Change-Id: I75b58248bfc4e86cace75faa82526d55a9ebbdbf
Partially-Implements: blueprint sahara-role

Co-Authored-By: zhubingbing <zhubingbing10@gmail.com>

Change-Id: Id83c852e32c3dd583e6128e888ac511634e8eabb
Partially-Implements: blueprint cloudkitty

Partially-Implements: blueprint ansible-gnocchi

Change-Id: I8dd0460bd21ac0a233fab0142ec7b6079459bdc2

This ensures that the same client IP address will always reach the same
server as long as no server goes down or up. [0]

Prevents a situation where during Murano package upload - we end up
having zip file on one control node but the import continues on another
and ends up failing.

[0] http://cbonte.github.io/haproxy-dconv/configuration-1.7.html#4-balance



TrivialFix
Co-Authored-By: Vladislav Belogrudov <vladislav.belogrudov@oracle.com>

Change-Id: I5f90d2757f31e8b24459a585153d5aa7fe6ad90a

This changed introduces 4 new parameters to be able to use an existing
elasticsearch service for central logging.

* elasticsearch_address - address of elasticsearch server
* elasticsearch_protocol - protocol (HTTP/HTTPS) used by elasticsearch server
* enable_elasticsearch - deploy elasticsearch container
* enable_kibana - deploy kibana container

Closes-bug: #1584861

Change-Id: Ia1ff9ae8b6d9929c3826da02693d1e2fc9ea2522

Previous work on Watcher added the Docker images, this
change adds the ansible configuration.

There is support for HA, via haproxy to balance across the
Watcher API hosts.

There is also a hook into nova.conf to conditionally add
Nova compute Host metrics via Ceilometer if Watcher is enabled.

This defaults to enabled false.

Change-Id: I8763528bb6ff12943b810212c71396d2d7cf6836
Partial-bug: #1598929
Partially-implements: bp watcher
Signed-off-by: Dave Walker (Daviey) <email@daviey.com>

Added pipeline.yaml, event_pipeline.yaml and event_definitions.yaml
based on sample files in OpenStack documentation

Edited haproxy.cfg for ceilometer support

Edited ceilometer-base dockerfile for missing dependency

Change-Id: I6ade05255e7e1aa7dbcffd026fad5869036d0d32
Closes-Bug: #1604004

The forwardfor option cannot be used in certain modes
such as TCP.  To resolve that create a special default
section for MariaDB

Change-Id: I743bbbfb732b04f115d1a878a0dfc22e29d2623d
Closes-Bug: #1549746

The Nova EC2 API is disabled by default, the default value
of the enabled_apis parameter in nova.conf is "osapi_compute, metadata"
The EC2 API is marked as deprecated and will be removed from Nova in
the future.

Change-Id: I6b9d66017e066cde5749be45b367194d2192ead3
Closes-bug: #1586605

It's impossible to drop root for the HAProxy container.
But HAProxy provides a possibility to use a chroot jail.

When attaching to the HAProxy container, we see that
the root directory is changed:

    $ sudo docker exec -ti haproxy bash
    (haproxy)[root@operator /]# ls -di /
    259 /

Co-Authored-By: Vikram Hosakote <vhosakot@cisco.com>

Closes-Bug: #1552289

Change-Id: I9d55e9b741b8560cac53dc8b837a24a3029a4dc0

Use HAProxy to terminate a TLS connection on port 5601 for the
Kibana dashboard when TLS is enabled for Kolla. x-forwarded-for
and x-forwarded-proto headers are set to give Kibana the info it
needs to write returned URLs.

Change-Id: I03a2dd3a8e2513d38281b30bf4bae6449fec0316
Closes-bug: #1566117

To be kolla deploy multiple clouds, we need to be able to configure
virtual_router_id other wise haproxy will fail setup the VIP for the
second cloud.

Partially-Implements: blueprint multiple-cloud
Closes-Bug: #1564547
Change-Id: I9eb27dd6fba61205841eadafc96601e235d2fe6d

When a node uses two physical interfaces for its two VIPs, these
physical interfaces should be tied together, so both VIPs will
be taken out of scheduling if either one fails.  Without this change,
if a request comes into one interface that needs access to the
second interface to process the request, the original request
unnecessarily fails.  Repeating this results in a black hole where
a failing server keeps getting new requests.

Change-Id: Ic51e6584c1fbda3eb7821cb47f759c77e562cc65
Closes-Bug: #1550455

Closes-Bug: #1564440
Change-Id: I8b9ff303d131fab18e51e9f99c64f4e4004f41b4

haproxy-system-wrapper is a solution for systemd from upstream. it can
handle the reload graceful.

Change-Id: I6a3d141af065e429bd1be1b7252f5c6df1fda3bb
Closes-Bug: #1559238

Change-Id: Ib501571dd34cb68924775ce738499d63df5718dd
Closes-Bug: 1556487

These values are optional only when the services are not enabled.

If the file does not exist we should not warn, but rather inform.

Ceph-mon is an exception here since its bootstrap process means
the files may or may not exist initially.

TrivialFix

Change-Id: Ic02bece76d480e99deecf612036f37abb5604135

Without this option the vip will always bounce to the highest priority
node that is up. So if you reboot the highest priority node the vip
will fail to the second highest. When the highest priority node
recovers it will claim the vip again leaving you will two fail overs
rather than one.

TrivialFix

Change-Id: I4a3c6c10eee391cdbdd80c44a71a9fafd1069944

haproxy 1.6+ does not allow the formatting that was used for stats
listener. We need to adjust it to the correct syntax

TrivialFix

Change-Id: I5f0111c756d40a0cf7385e6963ebbb57adb36b35

Kibana is a tool for operators. It should not be accessible though
the external VIP.

Closes-Bug: #1554977
Change-Id: I1dc101de18e4e01ebde9d317ab7e3193e307a14e

When configured with a separate external VIP, glance registry
should listen on only the internal VIP.

TrivialFix

Change-Id: Ie186f2ea391b53b9ea0cb230c573c9e09efd44b2

This patch includes changes relative to integrating Heka with
Elasticsearch and Kibana.

The main change is the addition of an Heka ElasticSearchOutput plugin
to make Heka send the logs it collects to Elasticsearch.

Since Logstash is not used the enable_elk deploy variable is renamed
to enable_central_logging.

If enable_central_logging is false then Elasticsearch and Kibana are
not started, and Heka won't attempt to send logs to Elasticsearch.

By default enable_central_logging is set to false. If
enable_central_logging is set to true after deployment then the Heka
container needs to be recreated (for Heka to get the new
configuration).

The Kibana configuration used property names that are deprecated in
Kibana 4.2. This is changed to use non-deprecated property names.

Previously logs read from files and from Syslog had a different Type
in Heka. This is changed to always use "log" for the Type. In this
way just one index instead of two is used in Elasticsearch, making
things easier to the user on the visualization side.

The HAProxy configuration is changed to add entries for Kibana.
Kibana server is now accessible via the internal VIP, and also via
the external VIP if there's one configured.

The HAProxy configuration is changed to add an entry for
Elasticsearch. So Elasticsearch is now accessible via the internal
VIP. Heka uses that channel for communicating with Elasticsearch.

Note that currently the Heka logs include "Plugin
elasticsearch_output" errors when Heka starts. This occurs when Heka
starts processing logs while Elasticsearch is not yet started. These
are transient errors that go away when Elasticsearch is ready. And
with buffering enabled on the ElasticSearchOuput plugin logs will be
buffered and then retransmitted when Elasticsearch is ready.

Change-Id: I6ff7a4f0ad04c4c666e174693a35ff49914280bb
Implements: blueprint central-logging-service

TrivialFix

Closes-Bug: #1551314
Change-Id: Id85859500aec283703b6b6714abf213a42286182

TLS can be used to encrypt and authenticate the connection with
OpenStack endpoints.  This patch provides the necessary
parameters and changes the resulting service configurations to
enable TLS for the Kolla deployed OpenStack cloud.

The new input parameters are:

kolla_enable_tls_external: "yes" or "no" (default is "no")
kolla_external_fqdn_cert: "/etc/kolla/certificates/haproxy.pem"
kolla_external_fqdn_cacert: "/etc/kolla/certificates/haproxy-ca.crt"

Implements: blueprint kolla-ssl

Change-Id: I48ef8a781c3035d58817f9bf6f36d59a488bab41

Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.

This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.

kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn

The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.

Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla

HAProxy: change to use option forwardfor to pass origin IP address
to backend via X-Forwarded-For header

Keystone: Apache does the audit logs for keystone.  Change the
LogFormat to display the passed address instead of the connection
address which is that of the load balancer.

Nova, Cinder, Glance: these services can make use of the address
passed in X-Forwarded-For.  With this setting the API logs for
these services include the client IP address.

Change-Id: Ia861ecc11a7c7d463d0366586926d1a842853f69
Closes-Bug: #1548935

To improve security, operators have asked for two VIPs for
their cloud.

VIP 1 is the internal VIP that can reach internal and admin endpoints.
In addition, the internal VIP can also reach other internal services,
such as the database and message services.
VIP 2 is the external VIP that can only reach public endpoints.

With one VIP only, all services are reached at the same address.

To add a second VIP, this patch adds two new configuration parameters.

kolla_external_vip_address: is an IPv4 address to use for created VIP
kolla_external_vip_interface: is the network interface to use for VIP
In this scenario, the first VIP (the internal VIP), is defined by
the original parameters (kolla_internal address and network_interface).

When using two VIPs, the existing kolla_external_address parameter
should be/point to/resolve to the kolla_external_vip_address.

Closes-bug: 1535333

Change-Id: I5bfcefaf7899298455cdade8209c34324aebfecb

Partially implements: blueprint heka
Change-Id: I7e2bf4e520fa14fd40e3b329f3b2998ae6ea47f4

Partial-Bug: #1544545
Change-Id: I292bcaeacb080ff4c5ab6b42b7d899039d6b19c4

Change-Id: Ia6ac371845d1f2b545406d096b35a6d5f68be5f6
Implements: blueprint ansible-mongodb

Implements blueprint radosgw-container

Change-Id: Idc88a67a0979be626d3eaa9b2b9a527010aa2006

In heterogeneous environment, api_interfaces are different each other.
So we should specify it from hostvars.

Implements: bp configure-network-interface
Change-Id: Id15d70bfb9ebb62a64a3847a6b77407efb171dbe

Change-Id: Idb25ac4d3148c9b9400cf675ac2e47d35cce6224
Implements: blueprint ansible-magnum

Due bad rebases there is a huge section of the spice patch missing
from the implementation unfortunately. This patch finishes the rest
of this patch out properly.

Change-Id: I693c6745e9594fd91eb6453f6de9dfcbd410e89c
Paritally-Implements: blueprint nova-proxies

The bootstrap must occur on the nova-api node due to binding in the
nova-api directory (same goes for all other services)

Closes-Bug: #1513439
Backport: Liberty
Change-Id: Iab88b49712828085e4d7e7f85e6d8f0b7999a9bf

Adjust all the configs to list all the rabbitmq hosts rather than
running rabbitmq through the VIP. This is made possible by clusterer
which has already merged.

Change-Id: I5db48f5f10ec68f4c8863a29bc13984f6845a4f9
Partially-Implements: blueprint rabbitmq-clusterer

This commit fixes configuration for murano-api HA

Change-Id: I6b6f2c266cbb4c8131a949e31094572dad91fc40
Closes-Bug: #1510921
Backport: Liberty