Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22822

Queue.maintain does disk I/O via PeepholePermalink.resolve

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      https://gist.githubusercontent.com/jglick/de0cf09312d8fce00ab1/raw/gistfile1.txt

      Every time Queue.maintain is called, which is often, for every buildable job in the queue, which could be a lot, Job.getEstimatedDurationCandidates is called. During startup this could do some work loading the last few builds, but we expect those to be held in memory thereafter. However simply finding the last successful/any build involves PeepholePermalink.resolve, which hits the disk every time, with no caching. (It claims to "cache" but this is caching via symlink, not in memory.)

      While we are waiting for the disk I/O operation to complete, the Queue is locked, and lots of other threads will typically be waiting for it.

        Attachments

          Activity

          Hide
          jglick Jesse Glick added a comment -

          Yes, please do.

          Show
          jglick Jesse Glick added a comment - Yes, please do.
          Hide
          abayer Andrew Bayer added a comment -

          Looking very good so far - queue is only up to 16, so it could get worse later, but no page load slowdown at all currently.

          Show
          abayer Andrew Bayer added a comment - Looking very good so far - queue is only up to 16, so it could get worse later, but no page load slowdown at all currently.
          Hide
          abayer Andrew Bayer added a comment -

          So yeah, big performance improvement on the UI - great fix, jglick! Thanks!

          Show
          abayer Andrew Bayer added a comment - So yeah, big performance improvement on the UI - great fix, jglick! Thanks!
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/jenkins/model/PeepholePermalink.java
          core/src/test/java/jenkins/model/PeepholePermalinkTest.java
          http://jenkins-ci.org/commit/jenkins/c74dc66f4e9d954b43b64ff5be83b74ebb0f3a54
          Log:
          [FIXED JENKINS-22822] PeepholePermalink should maintain an in-memory cache of symlinks to avoid disk I/O.

          (cherry picked from commit 2ff7a905ac866aac5aea7aae0cb97ee8906b5829)

          Conflicts:
          changelog.html

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/model/PeepholePermalink.java core/src/test/java/jenkins/model/PeepholePermalinkTest.java http://jenkins-ci.org/commit/jenkins/c74dc66f4e9d954b43b64ff5be83b74ebb0f3a54 Log: [FIXED JENKINS-22822] PeepholePermalink should maintain an in-memory cache of symlinks to avoid disk I/O. (cherry picked from commit 2ff7a905ac866aac5aea7aae0cb97ee8906b5829) Conflicts: changelog.html
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/jenkins/model/PeepholePermalink.java
          core/src/test/java/jenkins/model/PeepholePermalinkTest.java
          http://jenkins-ci.org/commit/jenkins/2853564b2387a19893975ed6f042c3c25bf5e062
          Log:
          [FIXED JENKINS-22822] PeepholePermalink should maintain an in-memory cache of symlinks to avoid disk I/O.

          (cherry picked from commit 2ff7a905ac866aac5aea7aae0cb97ee8906b5829)

          Conflicts:
          changelog.html

          Compare: https://github.com/jenkinsci/jenkins/compare/f3848d2d4865...2853564b2387

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/model/PeepholePermalink.java core/src/test/java/jenkins/model/PeepholePermalinkTest.java http://jenkins-ci.org/commit/jenkins/2853564b2387a19893975ed6f042c3c25bf5e062 Log: [FIXED JENKINS-22822] PeepholePermalink should maintain an in-memory cache of symlinks to avoid disk I/O. (cherry picked from commit 2ff7a905ac866aac5aea7aae0cb97ee8906b5829) Conflicts: changelog.html Compare: https://github.com/jenkinsci/jenkins/compare/f3848d2d4865...2853564b2387

            People

            • Assignee:
              jglick Jesse Glick
              Reporter:
              jglick Jesse Glick
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: