Skip to content
Snippets Groups Projects
  1. Dec 30, 2024
  2. Aug 09, 2024
    • Christian Berendt's avatar
      Add fluentd_enable_watch_timer · 110686e6
      Christian Berendt authored
      By default, the watch timer in Fluentd is set to True.
      To save CPU and I/O consumption this can be set to False, which
      kolla-ansible has been hardcoding so far.
      
      When the watch timer is disabled, in_tail relies entirely
      on inotify. In certain constellations, this may not work
      reliably. In these cases, the watch timer needs to be activated, so this
      change adds a variable to make the setting configurable.
      
      Change-Id: Ic8ce6fbc3ed8f31d5d090e114b35703532679729
      110686e6
  3. Jul 25, 2024
  4. Nov 28, 2023
  5. May 24, 2023
  6. Nov 11, 2022
    • Doug Szumski's avatar
      Remove support for deploying OpenStack Monasca · adb8f89a
      Doug Szumski authored
      Kolla Ansible is switching to OpenSearch and is dropping support for
      deploying ElasticSearch. This is because the final OSS release of
      ElasticSearch has exceeded its end of life.
      
      Monasca is affected because it uses both Logstash and ElasticSearch.
      Whilst it may continue to work with OpenSearch, Logstash remains an
      issue.
      
      In the absence of any renewed interest in the project, we remove
      support for deploying it. This helps to reduce the complexity
      of log processing configuration in Kolla Ansible, freeing up
      development time.
      
      Change-Id: I6fc7842bcda18e417a3fd21c11e28979a470f1cf
      adb8f89a
  7. Apr 16, 2022
  8. Mar 23, 2022
  9. Dec 28, 2021
    • Radosław Piliszek's avatar
      Fluentd cleanup · df2a29e4
      Radosław Piliszek authored
      We no longer have to support multiple fluentd versions and rely
      on the workaround.
      
      Change-Id: Id4083a9f07317fc04467e52176850f3909703169
      df2a29e4
  10. Oct 12, 2021
    • Maksim Malchuk's avatar
      Update dnsmasq logging · d559e2c3
      Maksim Malchuk authored
      
      This change adds the dnsmasq.log for the ironic-dnsmasq container and
      also enables more verbose logging when debug logging enabled.
      This can be triggered globbaly via 'openstack_logging_debug' or per
      service via 'ironic_logging_debug' or 'neutron_logging_debug'.
      
      Change-Id: I0e6b089beb88827effbcc365625eb2df902f5470
      Signed-off-by: default avatarMaksim Malchuk <maksim.malchuk@gmail.com>
      d559e2c3
  11. May 23, 2020
    • Doug Szumski's avatar
      Switch to Monasca API for logs · b39a0f80
      Doug Szumski authored
      The Monasca Log API has been removed and in this change we switch
      to using the unified API. If dedicated log APIs are required then
      this can be supported through configuration. Out of the box the
      Monasca API is used for both logs and metrics which is envisaged to
      work for most use cases.
      
      In order to use the unified API for logs, we need to disable the
      legacy Kafka client. We also rename the Monasca API config file
      to remove a warning about using the old style name.
      
      Depends-On: https://review.opendev.org/#/c/728638
      Change-Id: I9b6bf5b6690f4b4b3445e7d15a40e45dd42d2e84
      b39a0f80
  12. Mar 18, 2020
  13. Dec 09, 2019
    • Mark Goddard's avatar
      Fix fluentd parsing of WSGI logs · a32cddca
      Mark Goddard authored
      WSGI log files use a different input configuration than OpenStack log
      files. Currently this depends on log files matching either *-access.log
      or *-error.log. Some services use *_access.log or *_error.log, so are
      not parsed correctly.
      
      This change modifies the fluentd configuration to accept an underscore
      or hyphen for WSGI log file names.
      
      Change-Id: I566d6cac0b6749054fd5422ec8f36f99dacb1db7
      Closes-Bug: #1720371
      a32cddca
  14. Aug 22, 2019
  15. Aug 15, 2019
  16. Jun 18, 2019
    • Doug Szumski's avatar
      Increase log coverage for Monasca · cb404743
      Doug Szumski authored
      Monasca Python service logs prior to this change were being dropped
      due to missing entries in the Fluent record_transformer config file.
      This change adds support for ingesting those logs, and explicitly
      removes support for ingesting Monasca Log API logs to reduce the risk
      of feedback, for example if debug logging is turned on in the Monasca
      Log API.
      
      Change-Id: I9e3436a8f946873867900eed5ff0643d84584358
      cb404743
    • Doug Szumski's avatar
      Ingest non-standard Monasca logs · 4b31fdcf
      Doug Szumski authored
      Presently, errors can appear in Fluentd and Monasca Log API logs due
      to log output from some Monasca services, which do not use Oslo log,
      being processed alongside other OpenStack logs which do.
      
      This change parses these log files separately to prevent these errors.
      
      Change-Id: Ie3cbb51424989b01727b5ebaaeba032767073462
      4b31fdcf
  17. Jun 05, 2019
  18. May 31, 2019
    • Gaetan Trellu's avatar
      Adds Qinling Ansible role · edb34898
      Gaetan Trellu authored
      Qinling is an OpenStack project to provide "Function as a Service".
      This project aims to provide a platform to support serverless functions.
      
      Change-Id: I239a0130f8c8b061b531dab530d65172b0914d7c
      Implements: blueprint ansible-qinling-support
      Story: 2005760
      Task: 33468
      edb34898
  19. Mar 08, 2019
    • Bai Yongjun's avatar
      Add cyborg to kolla-ansible · ed2fd243
      Bai Yongjun authored
      Because kolla-ansible not have cyborg so should add it.
      
      Implements: blueprint add-cyborg-to-kolla-ansible
      
      Depend-On: I497e67e3a754fccfd2ef5a82f13ccfaf890a6fcd
      
      Change-Id: I6f7ae86f855c5c64697607356d0ff3161f91b239
      ed2fd243
  20. Feb 22, 2019
  21. Nov 20, 2018
    • Eduardo Gonzalez's avatar
      Fix fluentd input log parser for OpenDaylight · 76482c9a
      Eduardo Gonzalez authored
      OpenDaylight logs have different format than openstack,
      is a karaf log with java error traces.
      This PS add required config to make fluentd parse properly
      ODL logs.
      
      Change-Id: I34fb96c8a424679b3b618f2ff6a840b8dc165bec
      76482c9a
  22. Jul 10, 2018
    • Stanislaw Bogatkin's avatar
      Keep timestamp field · 90b886e5
      Stanislaw Bogatkin authored
      There are cases when we can lost original timestamp field given from
      logs, like when we send our logs to the next fluentd forwarder in chain
      of forwarders, it will rewrite our timestamp by default. Save
      `Timestamp` field explicitly to avoid such situation and be able to
      reconstruct messages date and time.
      
      Closes-Bug: #1781046
      Change-Id: I2b4486aedacbe16dc4c0fb2e4e4984bd80e59f2d
      90b886e5
  23. May 21, 2018
  24. Dec 29, 2017
    • Mark Goddard's avatar
      Add ironic inspector to fluentd log collection · 68b2d48f
      Mark Goddard authored
      Previously ironic inspector logs were not being collected, as they are
      stored in /var/log/kolla/ironic-inspector/ rather than
      /var/log/kolla/ironic/. This change modifies the fluentd input
      configuration to use the correct directory.
      
      Also adds the ironic-inspector service to the fluentd programname
      rewrite rule.
      
      Change-Id: Ie5d67f395e4c32fef992e674f2d53dbd6a17f238
      68b2d48f
  25. Sep 18, 2017
    • Bertrand Lallau's avatar
      Fluentd: reduce CPU and I/O consumption · 498dab24
      Bertrand Lallau authored
      Disable the additional watch timer (enable_watch_timer).
      Setting this parameter to false will significantly reduce CPU and I/O
      consumption when tailing a large number of files on systems with inotify
      support.
      
      Change-Id: Ib432e6cdece01dbb94b3d2f3d7fbdd04b81d25fa
      498dab24
  26. Sep 04, 2017
    • Bertrand Lallau's avatar
      Fluentd: manage multiline log pattern · 38a6d304
      Bertrand Lallau authored
      Actual Fluentd config doesn't manage multiline log pattern like:
      
      2017-08-16 21:41:03.097 7 WARNING oslo_config.cfg [-] Option "firewall_driver" from group "DEFAULT" is deprecated for removal (
      nova-network is deprecated, as are any related configuration options.
      ). Its value may be silently ignored in the future.
      
      This log will generate 1 Fluentd event for each line (in previous
      example 3 events).
      This is really a wrong behaviour and made Kibana interface not friendly.
      This patch uses a multiline parser.
      
      Change-Id: Iee337645d1dc42876ec79be08ee3715f11a53d87
      Closes-Bug: #1711399
      38a6d304
  27. Aug 29, 2017
    • Bertrand Lallau's avatar
      Fluentd: rework and fix tail source config · eab91d42
      Bertrand Lallau authored
      Actually a unique 'tail' source is used for all logs files in
      kolla/*/*.log.
      
      So log files from services like: chrony, elasticsearch, etcd, grafana,
      influxdb, kibana, mongodb... are actually processed by our Fluentd
      config, but should not cause all following config will failed to parse
      them. It just uses CPU for nothing cause they will
      never match the following filters:
      * record_transformer => add Hostname, Logger, programname field
      * rewrite_tag_filter => rewrite 'programname' TAG if needed
      
      Hence these filters should only be applied to Openstack services logs to
      avoid CPU usage.
      
      Furthermore WGI and Apache logs files must be treated differently.
      
      Change-Id: I425979160291e1123e3775e39243bca06ed22c2f
      Closes-Bug: #1711390
      eab91d42
  28. Aug 21, 2017
    • Eduardo Gonzalez's avatar
      Update fluentd config for rpm distros · 1d5bd219
      Eduardo Gonzalez authored
      Fluentd repo/packages are changed in depends changes below.
      Rpm distros need changes in their config for properly
      start and deploy fluentd containers.
      
      Change-Id: I474c4d20c407717c6da973383bbbd5539be480f1
      Depends-On: I4802618373cfa53d67d1dfe65be7abe1950b3d04
      1d5bd219
  29. Apr 05, 2017
  30. Apr 02, 2017
    • Bertrand Lallau's avatar
      Fix Fluentd warn on dnsmasq.log file parsing · 4f6470bb
      Bertrand Lallau authored
      Fluentd actually parse all files in /var/log/kolla/*/*.log as Openstack
      services files.
      Dnsmasq file /var/log/kolla/neutron/dnsmasq.log, must be add to
      exclude_path to avoid this warning message:
      "/var/log/kolla/neutron/dnsmasq.log unreadable. It is excluded and would
      be examined next time."
      
      Change-Id: I8dc5320b9ed299200c4f1e6ee52d34a918e2f110
      Closes-Bug: #1678530
      4f6470bb
  31. Jan 24, 2017
    • zhubingbing's avatar
      Add fluentd role · d3d4a933
      zhubingbing authored
      
      * add fluentd role
      * remove heka configure
      
      Co-Authored-By: default avataryangzb09 <yangzb09@qq.com>
      
      Partially-Implements: blueprint add-fluentd-role
      
      Change-Id: Ica804a99f5bb8b157f406299c5982b7b6283b3e3
      d3d4a933
Loading