Rework horizon role to support local_settings.d

This patch implements horizon's preferred way how
to configure itself described in docs [1],

[1] https://docs.openstack.org/horizon/latest/configuration/settings.html

Depends-On: https://review.opendev.org/c/openstack/kolla/+/906339
Change-Id: I60ab4634bf4333c47d00b12fc4ec00570062bd18

ansible/roles/horizon/tasks/config.yml

@@ -77,7 +77,7 @@
   notify:
     - Restart horizon container
 
-- name: Copying over local_settings
+- name: Copying over kolla-settings.py
   become: true
   vars:
     horizon: "{{ horizon_services['horizon'] }}"
@@ -95,7 +95,7 @@
   notify:
     - Restart horizon container
 
-- name: Copying over custom_local_settings
+- name: Copying over custom-settings.py
   become: true
   vars:
     horizon: "{{ horizon_services['horizon'] }}"

ansible/roles/horizon/templates/horizon.json.j2

@@ -19,14 +19,14 @@
         },
 {% endfor %}
         {
-            "source": "{{ container_config_directory }}/local_settings",
-            "dest": "/etc/openstack-dashboard/local_settings",
+            "source": "{{ container_config_directory }}/_9998-kolla-settings.py",
+            "dest": "/etc/openstack-dashboard/local_settings.d/_9998-kolla-settings.py",
             "owner": "horizon",
             "perm": "0600"
         },
         {
-            "source": "{{ container_config_directory }}/custom_local_settings",
-            "dest": "/etc/openstack-dashboard/custom_local_settings",
+            "source": "{{ container_config_directory }}/_9999-custom-settings.py",
+            "dest": "/etc/openstack-dashboard/local_settings.d/_9999-custom-settings.py",
             "owner": "horizon",
             "perm": "0600"
         }{% if horizon_enable_tls_backend | bool %},

doc/source/reference/shared-services/horizon-guide.rst

@@ -18,7 +18,7 @@ Horizon by using a custom python settings file that will override
 the default options set on the local_settings file.
 
 As an example, for setting a different (material) theme as the default one,
-a file named custom_local_settings should be created under the directory
+a file named _9999-custom-settings.py should be created under the directory
 ``{{ node_custom_config }}/horizon/`` with the following contents:
 
 .. code-block:: python
@@ -45,7 +45,7 @@ This entry updates AVAILABLE_THEMES adding the new theme at the list end.
 
 Theme files have to be copied into:
 ``{{ node_custom_config }}/horizon/themes/my_custom_theme``.
-The new theme can be set as default in custom_local_settings:
+The new theme can be set as default in _9999-custom-settings.py:
 
 .. code-block:: python
 

releasenotes/notes/horizon-local-settings-d-32c9ad3d40ca23f4.yaml

+---
+upgrade:
+  - |
+    Horizon role was reworked to preffered local_settings.d
+    configuration model. Files ``local_settings`` and
+    ``custom_local_settings`` were renamed to ``_9998-kolla-settings.py``
+    and ``_9999-custom-settings.py`` Users who use horizon's custom
+    configuration have to change the names of those files in
+    ``/etc/kolla/config/horizon`` also.