Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7370

Analysis collector causes Hudson to take a long time to startup

    Details

    • Similar Issues:

      Description

      A clean start of Hudson takes currently approx 45-65 minutes. The initial loading page "Please wait while Hudson is getting ready to work...." is only visible page during this time. The start time was much lower as before, but dramatically increased after updating the analysis plugins some months ago and has been steadily increasing as we get more builds and more projects in our Hudson installation.

      During the startup time, we see the following line, repeating for every build in our system:


      Sep 2, 2010 4:17:49 PM hudson.plugins.analysis.core.BuildResult loadResult
      INFO: Loaded data file /home/hudson/hudson/server_installation/jobs/projectname/builds/2010-08-06_09-31-30/analysis.xml for build 0


      We have a total of 1114 analysis.xml (for all jobs and builds) totaling 6.7GB.

      => We assume that the analysis-collector plugin in scanning all of these files on every startup.

      When we do "kill -3" repeatedly on the Hudson java process during the startup time, we find that the following stack trace is representative of what is going on:
      ----------------------------------
      "Loading job projectname" daemon prio=10 tid=0x0854fc00 nid=0x7bd runnable [0x2f3a0000..0x2f3a2020]
      java.lang.Thread.State: RUNNABLE
      at com.thoughtworks.xstream.core.util.ClassLoaderReference.loadClass(ClassLoaderReference.java:31)
      at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:61)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:71)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.PackageAliasingMapper.realClass(PackageAliasingMapper.java:88)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.ClassAliasingMapper.realClass(ClassAliasingMapper.java:86)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.ArrayMapper.realClass(ArrayMapper.java:96)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at hudson.util.XStream2$CompatibilityMapper.realClass(XStream2.java:138)
      at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38)
      at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:56)
      at com.thoughtworks.xstream.core.util.HierarchicalStreams.readClassType(HierarchicalStreams.java:29)
      at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:70)
      at com.thoughtworks.xstream.converters.collections.ArrayConverter.unmarshal(ArrayConverter.java:55)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
      at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:137)
      at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:33)
      at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:926)
      at hudson.util.XStream2.unmarshal(XStream2.java:79)
      at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:912)
      at com.thoughtworks.xstream.XStream.fromXML(XStream.java:856)
      at hudson.XmlFile.read(XmlFile.java:126)
      at hudson.plugins.analysis.core.BuildResult.loadResult(BuildResult.java:759)
      at hudson.plugins.analysis.core.BuildResult.getProject(BuildResult.java:739)

      • locked <0xa75e52c0> (a java.lang.Object)
        at hudson.plugins.analysis.core.BuildResult.getContainer(BuildResult.java:883)
        at hudson.plugins.analysis.core.BuildResult.getAnnotations(BuildResult.java:484)
        at hudson.plugins.analysis.collector.AnalysisResult.countAnnotations(AnalysisResult.java:80)
        at hudson.plugins.analysis.collector.AnalysisResult.readResolve(AnalysisResult.java:70)
        at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.thoughtworks.xstream.converters.reflection.SerializationMethodInvoker.callReadResolve(SerializationMethodInvoker.java:46)
        at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:181)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
        at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:290)
        at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:233)
        at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:180)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
        at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:71)
        at hudson.util.RobustCollectionConverter.populateCollection(RobustCollectionConverter.java:85)
        at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:61)
        at hudson.util.RobustCollectionConverter.unmarshal(RobustCollectionConverter.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
        at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:290)
        at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:233)
        at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:180)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:137)
        at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:33)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:926)
        at hudson.util.XStream2.unmarshal(XStream2.java:79)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:912)
        at hudson.XmlFile.unmarshal(XmlFile.java:148)
        at hudson.model.Run.<init>(Run.java:257)
        at hudson.model.AbstractBuild.<init>(AbstractBuild.java:151)
        at hudson.model.Build.<init>(Build.java:99)
        at hudson.model.FreeStyleBuild.<init>(FreeStyleBuild.java:41)
        at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at hudson.model.AbstractProject.loadBuild(AbstractProject.java:870)
        at hudson.model.AbstractProject$1.create(AbstractProject.java:238)
        at hudson.model.AbstractProject$1.create(AbstractProject.java:236)
        at hudson.model.RunMap.load(RunMap.java:221)
      • locked <0x368da870> (a hudson.model.RunMap)
        at hudson.model.AbstractProject.onLoad(AbstractProject.java:236)
        at hudson.model.Project.onLoad(Project.java:87)
        at hudson.model.Items.load(Items.java:106)
        at hudson.model.Hudson$12.run(Hudson.java:2167)
        at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:146)
        at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:259)
        at hudson.model.Hudson$3.runTask(Hudson.java:679)
        at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:187)
        at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:94)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
        ----------------------------------

      By looking at the code of the analysis collector and other plugin, we believe the problem lies in the method "readResolve" in "plugins/analysis-collector/src/main/java/hudson/plugins/analysis/collector/AnalysisResult.java"

        Attachments

          Activity

            People

            • Assignee:
              drulli Ulli Hafner
              Reporter:
              terjestrand terjestrand
            • Votes:
              2 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: