Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40762

Logs can grow without bound when using timestamper plugin

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: timestamper-plugin
    • Labels:
      None
    • Environment:
      Jenkins 2.19.1
      Timestamper 1.8.7 (but observed in older versions as well)
    • Similar Issues:

      Description

      When a single pipeline step generates a lot of logging all at once, the log on disk will repeat itself without end, eventually running the host out of disk space.

      This persists between Jenkins restarts and can only be stopped by stopping Jenkins, deleting the $JENKINS_HOME/jobs/[job]/build/[build id] directory, and starting Jenkins again.

      STR:
      1. Create a new pipeline job with the following pipeline script definition:
      stage ('Boom') {
      node {
      timestamps

      { sh "seq 1 100000" }

      }
      }

      2. Run the job.

      Expected result:
      Job terminates eventually.

      Actual result:
      Job does not terminate, and fills the log continuously.

        Attachments

          Activity

          Hide
          eflores Edwin Flores added a comment -

          This bug is caused by:

          1. A timeout in the Pipeline Nodes and Processes plugin[1] which, if writing logs is taking more than 10 seconds, will interrupt the thread and attempt to write the log again from the start; and
          2. Poor performance in TimestampNotesOutputStream. TimestampNote.encode() takes more than 100us on my machine and totally dominates run time.

          The pull request at [2] improves throughput significantly by caching the encoded TimestampNote.

          [1] https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/1c0f5ff191c37fcf339d637834d3cc900171f71e/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep.java#L306
          [2] https://github.com/jenkinsci/timestamper-plugin/pull/20

          Show
          eflores Edwin Flores added a comment - This bug is caused by: A timeout in the Pipeline Nodes and Processes plugin [1] which, if writing logs is taking more than 10 seconds, will interrupt the thread and attempt to write the log again from the start; and Poor performance in TimestampNotesOutputStream. TimestampNote.encode() takes more than 100us on my machine and totally dominates run time. The pull request at [2] improves throughput significantly by caching the encoded TimestampNote. [1] https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/1c0f5ff191c37fcf339d637834d3cc900171f71e/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep.java#L306 [2] https://github.com/jenkinsci/timestamper-plugin/pull/20
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Edwin Flores
          Path:
          src/main/java/hudson/plugins/timestamper/TimestampNotesOutputStream.java
          http://jenkins-ci.org/commit/timestamper-plugin/d0c6e9700b9c76735d5740d205dcda9f6c14dd2b
          Log:
          JENKINS-40762 Cache encoded timestamp note in TimestampNotesOutputStream

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Edwin Flores Path: src/main/java/hudson/plugins/timestamper/TimestampNotesOutputStream.java http://jenkins-ci.org/commit/timestamper-plugin/d0c6e9700b9c76735d5740d205dcda9f6c14dd2b Log: JENKINS-40762 Cache encoded timestamp note in TimestampNotesOutputStream
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: StevenGBrown
          Path:
          src/main/java/hudson/plugins/timestamper/TimestampNotesOutputStream.java
          http://jenkins-ci.org/commit/timestamper-plugin/1744efd94ad87cf61060af6da2aa1f01c79a7e3c
          Log:
          Merge pull request #20 from edf825/master

          JENKINS-40762 Cache encoded timestamp note in TimestampNotesOutputStream

          Compare: https://github.com/jenkinsci/timestamper-plugin/compare/041b6f5f9b8d...1744efd94ad8

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: StevenGBrown Path: src/main/java/hudson/plugins/timestamper/TimestampNotesOutputStream.java http://jenkins-ci.org/commit/timestamper-plugin/1744efd94ad87cf61060af6da2aa1f01c79a7e3c Log: Merge pull request #20 from edf825/master JENKINS-40762 Cache encoded timestamp note in TimestampNotesOutputStream Compare: https://github.com/jenkinsci/timestamper-plugin/compare/041b6f5f9b8d...1744efd94ad8
          Hide
          stevengbrown Steven G Brown added a comment -

          Fix included in Timestamper 1.8.8.

          Show
          stevengbrown Steven G Brown added a comment - Fix included in Timestamper 1.8.8.

            People

            • Assignee:
              stevengbrown Steven G Brown
              Reporter:
              eflores Edwin Flores
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: