Skip to content
Snippets Groups Projects
  1. Mar 23, 2022
  2. Nov 15, 2021
  3. Mar 04, 2021
    • Doug Szumski's avatar
      Support disabling Monasca alerting pipeline · 44409784
      Doug Szumski authored
      The Monasca alerting pipeline provides multi-tenancy alerts and
      notifications. It runs as an Apache Storm topology and generally
      places a significant memory and CPU burden on monitoring hosts,
      particularly when there are lot of metrics. This is fine if the
      alerting service is in use, but sometimes it is not. For example
      you may use Prometheus for monitoring the control plane, and
      wish to offer tenants a monitoring service via Monasca without
      alerting and notification functionality. In this case it makes
      sense to disable this part of the Monasca pipeline and this patch
      adds support for that.
      
      If the service is ever re-enabled, all alerts and notifications
      should spawn back automatically since they are persisted in the
      central mysql database cluster.
      
      Change-Id: I84aa04125c621712f805f41c8efbc92c8e156db9
      44409784
  4. Mar 03, 2021
    • Doug Szumski's avatar
      Remove Monasca Log Transformer · 0743a9bf
      Doug Szumski authored
      Historically Monasca Log Transformer has been for log
      standardisation and processing. For example, logs from different
      sources may use slightly different error levels such as WARN, 5,
      or WARNING. Monasca Log Transformer is a place where these could
      be 'squashed' into a single error level to simplify log searches
      based on labels such as these.
      
      However, in Kolla Ansible, we do this processing in Fluentd so
      that the simpler Fluentd -> Elastic -> Kibana pipeline also
      benefits. This helps to avoid spreading out log parsing
      configuration over many services, with the Fluentd Monasca output
      plugin being yet another potential place for processing (which
      should be avoided). It therefore makes sense to remove this
      service entirely, and squash any existing configuration which
      can't be moved to Fluentd into the Log Perister service. I.e.
      by removing this pipeline, we don't loose any functionality,
      we encourage log processing to take place in Fluentd, or at least
      outside of Monasca, and we make significant gains in efficiency
      by removing a topic from Kafka which contains a copy of all logs
      in transit.
      
      Finally, users forwarding logs from outside the control plane,
      eg. from tenant instances, should be encouraged to process the
      logs at the point of sending using whichever framework they are
      forwarding them with. This makes sense, because all Logstash
      configuration in Monasca is only accessible by control plane
      admins. A user can't typically do any processing inside Monasca,
      with or without this change.
      
      Change-Id: I65c76d0d1cd488725e4233b7e75a11d03866095c
      0743a9bf
Loading