Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22607

LogRotator inefficient when number of builds to kept is very large

    Details

    • Similar Issues:

      Description

      If you have configured LogRotator to delete all builds older than 30 days, yet you do around 1000 builds a day, then at the end of each build LogRotator.perform will scan backwards from the previous build all the way to the first build (or nearby), loading each build along the way, just to find the (on average) one build which is now older than 30 days so it can be deleted. This forces thousands of builds to be kept in memory, or if they cannot all be held in memory, forces each build to reload thousands of older build records that it does not really need.

      Would be better to do a binary search through the build history to find the newest build which is suitable for deletion.

        Attachments

          Activity

          Hide
          jglick Jesse Glick added a comment -

          Fixing this when numToKeep is set looks impractical, since AbstractLazyLoadRunMap.idOnDisk is not exposed (not even its size), and so there is no way of knowing which build is the 30000th oldest—you cannot rely on this having a particular build number. Fixing it for daysToKeep looks easier since you can simply browse forward from the oldest build and keep on deleting them so long as they are sufficiently old.

          Show
          jglick Jesse Glick added a comment - Fixing this when numToKeep is set looks impractical, since AbstractLazyLoadRunMap.idOnDisk is not exposed (not even its size), and so there is no way of knowing which build is the 30000th oldest—you cannot rely on this having a particular build number. Fixing it for daysToKeep looks easier since you can simply browse forward from the oldest build and keep on deleting them so long as they are sufficiently old.
          Hide
          jglick Jesse Glick added a comment -

          Not sure if the fix should be lts-candidate, given that it does not work for numToKeep.

          Show
          jglick Jesse Glick added a comment - Not sure if the fix should be lts-candidate , given that it does not work for numToKeep .
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/LogRotator.java
          http://jenkins-ci.org/commit/jenkins/d9188ce2ad2070eab5ea27cb7f96257b351392a7
          Log:
          [FIXED JENKINS-22607] Avoid loading recent builds when deletion of builds older than some date has been requested.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/tasks/LogRotator.java http://jenkins-ci.org/commit/jenkins/d9188ce2ad2070eab5ea27cb7f96257b351392a7 Log: [FIXED JENKINS-22607] Avoid loading recent builds when deletion of builds older than some date has been requested.
          Hide
          dogfood dogfood added a comment -

          Integrated in jenkins_main_trunk #3307
          [FIXED JENKINS-22607] Avoid loading recent builds when deletion of builds older than some date has been requested. (Revision d9188ce2ad2070eab5ea27cb7f96257b351392a7)

          Result = SUCCESS
          Jesse Glick : d9188ce2ad2070eab5ea27cb7f96257b351392a7
          Files :

          • core/src/main/java/hudson/tasks/LogRotator.java
          • changelog.html
          Show
          dogfood dogfood added a comment - Integrated in jenkins_main_trunk #3307 [FIXED JENKINS-22607] Avoid loading recent builds when deletion of builds older than some date has been requested. (Revision d9188ce2ad2070eab5ea27cb7f96257b351392a7) Result = SUCCESS Jesse Glick : d9188ce2ad2070eab5ea27cb7f96257b351392a7 Files : core/src/main/java/hudson/tasks/LogRotator.java changelog.html

            People

            • Assignee:
              jglick Jesse Glick
              Reporter:
              jglick Jesse Glick
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: