Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26879

Job Statistics portlet slow to load with many jobs and builds

    Details

    • Similar Issues:

      Description

      The Job Statistics portlet takes a long time to initially load with 100+ jobs and 1000+ builds each. Thread dump shows the stack trace is mostly spent in RunList.size() which is being called in StatJobs.HealthStatus.getHealthStatus(). This forces every build to be loaded from disk when, looking at the code, all it needs to do is check if the list is empty.

      "Handling GET /view/ServersDashboard/ from 192.168.70.40 : RequestHandlerThread10 View/index.jelly Dashboard/main.jelly StatJobs/portlet.jelly StatJobs/statjobs.jelly" Id=59 Group=main RUNNABLE
      at java.io.FileInputStream.readBytes(Native Method)
      at java.io.FileInputStream.read(FileInputStream.java:272)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:254)

      • locked java.io.BufferedInputStream@7bb58e96
        at java.io.FilterInputStream.read(FilterInputStream.java:83)
        at java.io.PushbackInputStream.read(PushbackInputStream.java:139)
        at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.getHeader(XmlHeaderAwareReader.java:79)
        at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.<init>(XmlHeaderAwareReader.java:61)
        at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65)
        at hudson.XmlFile.unmarshal(XmlFile.java:163)
        at hudson.model.Run.reload(Run.java:321)
        at hudson.model.Run.<init>(Run.java:309)
        at hudson.model.AbstractBuild.<init>(AbstractBuild.java:178)
        at hudson.model.Build.<init>(Build.java:103)
        at hudson.model.FreeStyleBuild.<init>(FreeStyleBuild.java:38)
        at sun.reflect.GeneratedConstructorAccessor70.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at jenkins.model.lazy.LazyBuildMixIn.loadBuild(LazyBuildMixIn.java:158)
        at jenkins.model.lazy.LazyBuildMixIn$1.create(LazyBuildMixIn.java:135)
        at hudson.model.RunMap.retrieve(RunMap.java:220)
        at hudson.model.RunMap.retrieve(RunMap.java:57)
        at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:465)
      • locked hudson.model.RunMap@4ca4594d
        at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:448)
        at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:356)
        at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:332)
        at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:74)
        at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:63)
        at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
        at hudson.util.RunList.size(RunList.java:108)
        at hudson.plugins.view.dashboard.stats.StatJobs$HealthStatus.getHealthStatus(StatJobs.java:54)
        at hudson.plugins.view.dashboard.stats.StatJobs.getJobStat(StatJobs.java:104)

      Even after the initial load, subsequent requests still take several seconds due to complete traversal of every build for every job.

      "Handling GET / from 192.168.70.53 : RequestHandlerThread20 View/index.jelly Dashboard/main.jelly StatJobs/portlet.jelly StatJobs/statjobs.jelly" Id=69 Group=main RUNNABLE
      at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:332)
      at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:74)
      at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:63)
      at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
      at hudson.util.RunList.size(RunList.java:108)
      at hudson.plugins.view.dashboard.stats.StatJobs$HealthStatus.getHealthStatus(StatJobs.java:54)
      at hudson.plugins.view.dashboard.stats.StatJobs.getJobStat(StatJobs.java:104)

        Attachments

          Activity

          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: cburroughs
          Path:
          src/main/java/hudson/plugins/view/dashboard/stats/StatJobs.java
          http://jenkins-ci.org/commit/dashboard-view-plugin/4eff12fc24bb09ab9c95bc0bd60eb28f3e75cd8d
          Log:
          optimize Job Statistics portlet

          Loading every build for every job can be expensive, especially in
          pathologically cases with > 10k builds per job. Fortunately the
          portlet only needs to know that a non-zero number of builds exist and
          not any of their details.

          Problem diagnosed and fix approach proposed by Matthew Sawin

          [FIXED JENKINS-26879]

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: cburroughs Path: src/main/java/hudson/plugins/view/dashboard/stats/StatJobs.java http://jenkins-ci.org/commit/dashboard-view-plugin/4eff12fc24bb09ab9c95bc0bd60eb28f3e75cd8d Log: optimize Job Statistics portlet Loading every build for every job can be expensive, especially in pathologically cases with > 10k builds per job. Fortunately the portlet only needs to know that a non-zero number of builds exist and not any of their details. Problem diagnosed and fix approach proposed by Matthew Sawin [FIXED JENKINS-26879]
          Hide
          dalvizu Dan Alvizu added a comment -

          Fixed in 2.9.7 - thanks Chris Burroughs

          Show
          dalvizu Dan Alvizu added a comment - Fixed in 2.9.7 - thanks Chris Burroughs

            People

            • Assignee:
              cburroughs Chris Burroughs
              Reporter:
              msawin Matthew Sawin
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: