Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7370

Analysis collector causes Hudson to take a long time to startup

    Details

    • Similar Issues:

      Description

      A clean start of Hudson takes currently approx 45-65 minutes. The initial loading page "Please wait while Hudson is getting ready to work...." is only visible page during this time. The start time was much lower as before, but dramatically increased after updating the analysis plugins some months ago and has been steadily increasing as we get more builds and more projects in our Hudson installation.

      During the startup time, we see the following line, repeating for every build in our system:


      Sep 2, 2010 4:17:49 PM hudson.plugins.analysis.core.BuildResult loadResult
      INFO: Loaded data file /home/hudson/hudson/server_installation/jobs/projectname/builds/2010-08-06_09-31-30/analysis.xml for build 0


      We have a total of 1114 analysis.xml (for all jobs and builds) totaling 6.7GB.

      => We assume that the analysis-collector plugin in scanning all of these files on every startup.

      When we do "kill -3" repeatedly on the Hudson java process during the startup time, we find that the following stack trace is representative of what is going on:
      ----------------------------------
      "Loading job projectname" daemon prio=10 tid=0x0854fc00 nid=0x7bd runnable [0x2f3a0000..0x2f3a2020]
      java.lang.Thread.State: RUNNABLE
      at com.thoughtworks.xstream.core.util.ClassLoaderReference.loadClass(ClassLoaderReference.java:31)
      at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:61)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:71)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.PackageAliasingMapper.realClass(PackageAliasingMapper.java:88)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.ClassAliasingMapper.realClass(ClassAliasingMapper.java:86)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.ArrayMapper.realClass(ArrayMapper.java:96)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at hudson.util.XStream2$CompatibilityMapper.realClass(XStream2.java:138)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:56)
      at com.thoughtworks.xstream.core.util.HierarchicalStreams.readClassType(HierarchicalStreams.java:29)
      at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:70)
      at com.thoughtworks.xstream.converters.collections.ArrayConverter.unmarshal(ArrayConverter.java:55)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
      at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:137)
      at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:33)
      at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:926)
      at hudson.util.XStream2.unmarshal(XStream2.java:79)
      at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:912)
      at com.thoughtworks.xstream.XStream.fromXML(XStream.java:856)
      at hudson.XmlFile.read(XmlFile.java:126)
      at hudson.plugins.analysis.core.BuildResult.loadResult(BuildResult.java:759)
      at hudson.plugins.analysis.core.BuildResult.getProject(BuildResult.java:739)

      • locked <0xa75e52c0> (a java.lang.Object)
        at hudson.plugins.analysis.core.BuildResult.getContainer(BuildResult.java:883)
        at hudson.plugins.analysis.core.BuildResult.getAnnotations(BuildResult.java:484)
        at hudson.plugins.analysis.collector.AnalysisResult.countAnnotations(AnalysisResult.java:80)
        at hudson.plugins.analysis.collector.AnalysisResult.readResolve(AnalysisResult.java:70)
        at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.thoughtworks.xstream.converters.reflection.SerializationMethodInvoker.callReadResolve(SerializationMethodInvoker.java:46)
        at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:181)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
        at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:290)
        at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:233)
        at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:180)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
        at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:71)
        at hudson.util.RobustCollectionConverter.populateCollection(RobustCollectionConverter.java:85)
        at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:61)
        at hudson.util.RobustCollectionConverter.unmarshal(RobustCollectionConverter.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
        at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:290)
        at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:233)
        at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:180)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:137)
        at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:33)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:926)
        at hudson.util.XStream2.unmarshal(XStream2.java:79)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:912)
        at hudson.XmlFile.unmarshal(XmlFile.java:148)
        at hudson.model.Run.<init>(Run.java:257)
        at hudson.model.AbstractBuild.<init>(AbstractBuild.java:151)
        at hudson.model.Build.<init>(Build.java:99)
        at hudson.model.FreeStyleBuild.<init>(FreeStyleBuild.java:41)
        at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at hudson.model.AbstractProject.loadBuild(AbstractProject.java:870)
        at hudson.model.AbstractProject$1.create(AbstractProject.java:238)
        at hudson.model.AbstractProject$1.create(AbstractProject.java:236)
        at hudson.model.RunMap.load(RunMap.java:221)
      • locked <0x368da870> (a hudson.model.RunMap)
        at hudson.model.AbstractProject.onLoad(AbstractProject.java:236)
        at hudson.model.Project.onLoad(Project.java:87)
        at hudson.model.Items.load(Items.java:106)
        at hudson.model.Hudson$12.run(Hudson.java:2167)
        at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:146)
        at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:259)
        at hudson.model.Hudson$3.runTask(Hudson.java:679)
        at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:187)
        at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:94)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
        ----------------------------------

      By looking at the code of the analysis collector and other plugin, we believe the problem lies in the method "readResolve" in "plugins/analysis-collector/src/main/java/hudson/plugins/analysis/collector/AnalysisResult.java"

        Attachments

          Activity

          Hide
          drulli Ulli Hafner added a comment -

          Thanks for reporting. I'll change the readResolve method and will initialize the mapping on demand.

          Show
          drulli Ulli Hafner added a comment - Thanks for reporting. I'll change the readResolve method and will initialize the mapping on demand.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : drulli
          Path:
          trunk/hudson/plugins/analysis-collector/src/main/java/hudson/plugins/analysis/collector/AnalysisResult.java
          http://jenkins-ci.org/commit/34909
          Log:
          [FIXED JENKINS-7370] Create mapping and load XML data only on demand.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : drulli Path: trunk/hudson/plugins/analysis-collector/src/main/java/hudson/plugins/analysis/collector/AnalysisResult.java http://jenkins-ci.org/commit/34909 Log: [FIXED JENKINS-7370] Create mapping and load XML data only on demand.
          Hide
          drulli Ulli Hafner added a comment -

          Integrated in Hudson Plug-ins (Compile) #55
          [FIXED JENKINS-7370] Create mapping and load XML data only on demand.

          Show
          drulli Ulli Hafner added a comment - Integrated in Hudson Plug-ins (Compile) #55 [FIXED JENKINS-7370] Create mapping and load XML data only on demand.

            People

            • Assignee:
              drulli Ulli Hafner
              Reporter:
              terjestrand terjestrand
            • Votes:
              2 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: