Skip to content
Snippets Groups Projects
  • Doug Szumski's avatar
    bd54b991
    Constrain the size of Docker logs · bd54b991
    Doug Szumski authored
    Even though Kolla services are configured to log output to file rather than
    stdout, some stdout still occurs when for example the container re(starts).
    Since the Docker logs are not constrained in size, they can fill up the
    docker volumes drive and bring down the host. One example of when this is
    particularly problematic is when Fluentd cannot parse a log message. The
    warning output is written to the Docker log and in production we have seen
    it eat 100GB of disk space in less than a day. We could configure Fluentd
    not to do this, but the problem may still occur via another mechanism.
    
    Change-Id: Ia6d3935263a5909c71750b34eb69e72e6e558b7a
    Closes-Bug: #1794249
    bd54b991
    History
    Constrain the size of Docker logs
    Doug Szumski authored
    Even though Kolla services are configured to log output to file rather than
    stdout, some stdout still occurs when for example the container re(starts).
    Since the Docker logs are not constrained in size, they can fill up the
    docker volumes drive and bring down the host. One example of when this is
    particularly problematic is when Fluentd cannot parse a log message. The
    warning output is written to the Docker log and in production we have seen
    it eat 100GB of disk space in less than a day. We could configure Fluentd
    not to do this, but the problem may still occur via another mechanism.
    
    Change-Id: Ia6d3935263a5909c71750b34eb69e72e6e558b7a
    Closes-Bug: #1794249