Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-16023

Lazy loading causes massive delays after a period of inactivity when loading dashboard

    Details

    • Similar Issues:

      Description

      After a period of inactivity or when the JVM decides to run GC loading the dashboard can easily take longer than 1-2 minutes. Please see attached stacktrace.

        Attachments

          Issue Links

            Activity

            johno Johno Crawford created issue -
            johno Johno Crawford made changes -
            Field Original Value New Value
            Description Handling GET /hudson-rc/ : http-10080-21
            java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
            java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:228)
            java.io.File.exists(File.java:733)
            hudson.model.RunMap.retrieve(RunMap.java:217)
            hudson.model.RunMap.retrieve(RunMap.java:59)
            jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638)
            jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:601)
            jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:344)
            hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
            hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103)
            hudson.model.Job.getLastFailedBuild(Job.java:780)
            sun.reflect.GeneratedMethodAccessor139.invoke(Unknown Source)
            sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            java.lang.reflect.Method.invoke(Method.java:597)
            org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
            org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
            org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
            org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
            org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
            org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
            org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
            org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
            hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72)
            org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
            org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
            org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
            org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
            org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            org.kohsuke.stapler.jelly.groovy.JellyBuilder.doInvokeMethod(JellyBuilder.java:283)
            org.kohsuke.stapler.jelly.groovy.Namespace$ProxyImpl.invoke(Namespace.java:92)
            $Proxy58.projectView(Unknown Source)
            lib.JenkinsTagLib$projectView.call(Unknown Source)
            org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
            org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
            org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
            hudson.model.View.main.run(main.groovy:14)
            org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:66)
            org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:59)
            org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
            org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44)
            org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
            org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
            org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
            org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:107)
            org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127)
            org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
            org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
            org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:625)
            org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
            org.kohsuke.stapler.Stapler.invoke(Stapler.java:488)
            org.kohsuke.stapler.Stapler.service(Stapler.java:162)
            javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
            org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
            org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
            net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
            net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
            net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
            org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
            hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
            hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
            org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
            org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:63)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
            hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
            org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
            org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
            org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
            org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
            org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
            org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
            org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
            org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
            org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
            org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
            org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
            org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
            java.lang.Thread.run(Thread.java:619)
             
            "Handling GET /hudson-rc/ : http-10080-21" daemon prio=5 tid=1094 RUNNABLE
            at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
            at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:228)
            Local Variable: java.io.File#1653
            at java.io.File.exists(File.java:733)
            at hudson.model.RunMap.retrieve(RunMap.java:217)
            at hudson.model.RunMap.retrieve(RunMap.java:59)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638)
            Local Variable: java.io.File#1654
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:601)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:344)
            Local Variable: hudson.model.RunMap#107
            Local Variable: jenkins.model.lazy.AbstractLazyLoadRunMap$Direction#1
            Local Variable: java.util.TreeMap$Entry#76935
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
            Local Variable: hudson.model.FreeStyleBuild#215
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103)
            at hudson.model.Job.getLastFailedBuild(Job.java:780)
            at sun.reflect.GeneratedMethodAccessor139.invoke(<unknown string>)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
            at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
            at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
            at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
            at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
            Local Variable: org.apache.commons.jexl.parser.ASTReference#2352
            at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
            at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
            at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
            Local Variable: org.apache.commons.jexl.ExpressionImpl#1862
            Local Variable: hudson.ExpressionFactory2$JellyJexlContext#1
            at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72)
            Local Variable: hudson.ExpressionFactory2$JexlExpression#1862
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
            Local Variable: org.apache.commons.jelly.tags.core.CoreTagLibrary$3#164
            Local Variable: java.lang.String#347180
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#6
            Local Variable: org.apache.commons.jelly.JellyContext#1
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
            at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: java.net.URL#11160
            Local Variable: org.apache.commons.jelly.impl.TagScript#941
            at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
            Local Variable: hudson.util.CopyOnWriteList$1#1
            Local Variable: org.apache.commons.jelly.tags.core.ForEachTag#2
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: java.net.URL#11163
            Local Variable: org.apache.commons.jelly.impl.TagScript#940
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#7
            Local Variable: org.apache.commons.jelly.JellyContext#2
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1253
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
            Local Variable: java.util.AbstractList$Itr#8
            Local Variable: org.apache.commons.jelly.tags.core.ForEachTag#1
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: org.apache.commons.jelly.impl.TagScript#938
            Local Variable: java.net.URL#11111
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#9
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1252
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#10
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#11
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1235
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#12
            Local Variable: org.apache.commons.jelly.JellyContext#3
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.kohsuke.stapler.jelly.groovy.JellyBuilder.doInvokeMethod(JellyBuilder.java:283)
            Local Variable: groovy.xml.QName#1
            Local Variable: java.util.Arrays$ArrayList#167
            Local Variable: org.kohsuke.stapler.jelly.CustomTagLibrary$1#276
            Local Variable: org.kohsuke.stapler.jelly.groovy.JellyBuilder$1#1
            at org.kohsuke.stapler.jelly.groovy.Namespace$ProxyImpl.invoke(Namespace.java:92)
            Local Variable: class lib.JenkinsTagLib
            Local Variable: java.lang.reflect.Method#14650
            Local Variable: java.lang.Object[]#74735
            Local Variable: org.kohsuke.stapler.jelly.groovy.Namespace$ProxyImpl#1
            Local Variable: java.lang.String#372793
            at $Proxy58.projectView(<unknown string>)
            at lib.JenkinsTagLib$projectView.call(<unknown string>)
            Local Variable: lib.JenkinsTagLib$projectView#1
            at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
            at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
            Local Variable: java.lang.Object[]#74736
            at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
            Local Variable: org.codehaus.groovy.runtime.callsite.AbstractCallSite#35
            Local Variable: java.util.LinkedHashMap#5843
            Local Variable: hudson.model.View.main$_run_closure1#1
            at hudson.model.View.main.run(main.groovy:14)
            Local Variable: $Proxy58#1
            Local Variable: org.codehaus.groovy.runtime.callsite.CallSite[]#2
            Local Variable: org.kohsuke.stapler.jelly.groovy.Namespace#1
            at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:66)
            Local Variable: hudson.model.View.main#1
            Local Variable: org.kohsuke.stapler.jelly.groovy.JellyBuilder#1
            at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:59)
            Local Variable: org.kohsuke.stapler.jelly.groovy.GroovierJellyScript#2
            Local Variable: org.apache.commons.jelly.JellyContext#4
            at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
            Local Variable: org.apache.catalina.loader.WebappClassLoader#1
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: java.net.URL#11099
            Local Variable: org.apache.commons.jelly.tags.define.InvokeBodyTag#2
            Local Variable: org.apache.commons.jelly.impl.TagScript#925
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#13
            at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            Local Variable: java.util.HashMap#32240
            Local Variable: java.util.HashMap#32060
            at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            Local Variable: org.kohsuke.stapler.jelly.CallTagLibScript$1#3
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: java.net.URL#10985
            Local Variable: org.apache.commons.jelly.impl.TagScript#923
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#14
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#15
            Local Variable: org.apache.commons.jelly.JellyContext#5
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#16
            at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            Local Variable: java.util.HashMap#32241
            Local Variable: java.util.HashMap#32063
            at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            Local Variable: org.kohsuke.stapler.jelly.CallTagLibScript$1#4
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: org.apache.commons.jelly.impl.TagScript#895
            Local Variable: java.net.URL#11206
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#17
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1177
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#18
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1176
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1175
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#19
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1167
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#20
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1166
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#21
            Local Variable: org.apache.commons.jelly.JellyContext#6
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44)
            Local Variable: org.kohsuke.stapler.jelly.CompressTag#1
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: org.apache.commons.jelly.impl.TagScript#864
            at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
            at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
            Local Variable: org.kohsuke.stapler.jelly.CustomJellyContext#24
            at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
            Local Variable: org.kohsuke.stapler.jelly.HTMLWriterOutput#1
            Local Variable: org.kohsuke.stapler.jelly.DefaultScriptInvoker#1
            at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:107)
            Local Variable: org.kohsuke.stapler.jelly.JellyViewScript#47
            Local Variable: org.kohsuke.stapler.jelly.JellyClassTearOff#62
            at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127)
            Local Variable: org.kohsuke.stapler.jelly.JellyFacet#1
            at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
            Local Variable: java.util.AbstractList$Itr#22
            Local Variable: org.kohsuke.stapler.MetaClass#62
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
            Local Variable: hudson.model.ListView#11
            at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:625)
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
            Local Variable: org.kohsuke.stapler.RequestImpl#1
            Local Variable: hudson.model.Hudson#1
            Local Variable: org.kohsuke.stapler.ResponseImpl#1
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:488)
            at org.kohsuke.stapler.Stapler.service(Stapler.java:162)
            Local Variable: java.lang.String#350622
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
            Local Variable: net.bull.javamelody.JspWrapper$HttpRequestWrapper#1
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
            at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
            Local Variable: java.lang.String#349353
            Local Variable: net.bull.javamelody.CounterServletResponseWrapper#1
            at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
            at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
            at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
            at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
            at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
            Local Variable: hudson.security.ChainedServletFilter$1#1
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:63)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            Local Variable: org.acegisecurity.context.HttpSessionContextIntegrationFilter$OnRedirectUpdateSessionResponseWrapper#1
            at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
            at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
            Local Variable: org.kohsuke.stapler.compression.CompressionServletResponse#1
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            Local Variable: org.apache.catalina.connector.RequestFacade#31
            Local Variable: org.apache.catalina.connector.ResponseFacade#31
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
            Local Variable: org.apache.catalina.core.ApplicationFilterChain#31
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
            at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
            Local Variable: org.apache.catalina.connector.Response#31
            Local Variable: org.apache.catalina.connector.Request#31
            Local Variable: org.apache.coyote.Response#31
            Local Variable: org.apache.coyote.Request#31
            at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
            Local Variable: org.apache.coyote.RequestInfo#31
            at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
            Local Variable: org.apache.coyote.http11.Http11Processor#31
            at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
            Local Variable: java.net.Socket#73
            Local Variable: org.apache.tomcat.util.net.JIoEndpoint$Worker#16
            at java.lang.Thread.run(Thread.java:619)
            johno Johno Crawford made changes -
            Attachment stacktrace.txt [ 22916 ]
            Description  
            "Handling GET /hudson-rc/ : http-10080-21" daemon prio=5 tid=1094 RUNNABLE
            at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
            at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:228)
            Local Variable: java.io.File#1653
            at java.io.File.exists(File.java:733)
            at hudson.model.RunMap.retrieve(RunMap.java:217)
            at hudson.model.RunMap.retrieve(RunMap.java:59)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638)
            Local Variable: java.io.File#1654
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:601)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:344)
            Local Variable: hudson.model.RunMap#107
            Local Variable: jenkins.model.lazy.AbstractLazyLoadRunMap$Direction#1
            Local Variable: java.util.TreeMap$Entry#76935
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
            Local Variable: hudson.model.FreeStyleBuild#215
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103)
            at hudson.model.Job.getLastFailedBuild(Job.java:780)
            at sun.reflect.GeneratedMethodAccessor139.invoke(<unknown string>)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
            at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
            at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
            at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
            at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
            Local Variable: org.apache.commons.jexl.parser.ASTReference#2352
            at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
            at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
            at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
            Local Variable: org.apache.commons.jexl.ExpressionImpl#1862
            Local Variable: hudson.ExpressionFactory2$JellyJexlContext#1
            at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72)
            Local Variable: hudson.ExpressionFactory2$JexlExpression#1862
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
            Local Variable: org.apache.commons.jelly.tags.core.CoreTagLibrary$3#164
            Local Variable: java.lang.String#347180
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#6
            Local Variable: org.apache.commons.jelly.JellyContext#1
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
            at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: java.net.URL#11160
            Local Variable: org.apache.commons.jelly.impl.TagScript#941
            at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
            Local Variable: hudson.util.CopyOnWriteList$1#1
            Local Variable: org.apache.commons.jelly.tags.core.ForEachTag#2
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: java.net.URL#11163
            Local Variable: org.apache.commons.jelly.impl.TagScript#940
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#7
            Local Variable: org.apache.commons.jelly.JellyContext#2
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1253
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
            Local Variable: java.util.AbstractList$Itr#8
            Local Variable: org.apache.commons.jelly.tags.core.ForEachTag#1
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: org.apache.commons.jelly.impl.TagScript#938
            Local Variable: java.net.URL#11111
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#9
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1252
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#10
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#11
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1235
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#12
            Local Variable: org.apache.commons.jelly.JellyContext#3
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.kohsuke.stapler.jelly.groovy.JellyBuilder.doInvokeMethod(JellyBuilder.java:283)
            Local Variable: groovy.xml.QName#1
            Local Variable: java.util.Arrays$ArrayList#167
            Local Variable: org.kohsuke.stapler.jelly.CustomTagLibrary$1#276
            Local Variable: org.kohsuke.stapler.jelly.groovy.JellyBuilder$1#1
            at org.kohsuke.stapler.jelly.groovy.Namespace$ProxyImpl.invoke(Namespace.java:92)
            Local Variable: class lib.JenkinsTagLib
            Local Variable: java.lang.reflect.Method#14650
            Local Variable: java.lang.Object[]#74735
            Local Variable: org.kohsuke.stapler.jelly.groovy.Namespace$ProxyImpl#1
            Local Variable: java.lang.String#372793
            at $Proxy58.projectView(<unknown string>)
            at lib.JenkinsTagLib$projectView.call(<unknown string>)
            Local Variable: lib.JenkinsTagLib$projectView#1
            at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
            at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
            Local Variable: java.lang.Object[]#74736
            at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
            Local Variable: org.codehaus.groovy.runtime.callsite.AbstractCallSite#35
            Local Variable: java.util.LinkedHashMap#5843
            Local Variable: hudson.model.View.main$_run_closure1#1
            at hudson.model.View.main.run(main.groovy:14)
            Local Variable: $Proxy58#1
            Local Variable: org.codehaus.groovy.runtime.callsite.CallSite[]#2
            Local Variable: org.kohsuke.stapler.jelly.groovy.Namespace#1
            at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:66)
            Local Variable: hudson.model.View.main#1
            Local Variable: org.kohsuke.stapler.jelly.groovy.JellyBuilder#1
            at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:59)
            Local Variable: org.kohsuke.stapler.jelly.groovy.GroovierJellyScript#2
            Local Variable: org.apache.commons.jelly.JellyContext#4
            at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
            Local Variable: org.apache.catalina.loader.WebappClassLoader#1
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: java.net.URL#11099
            Local Variable: org.apache.commons.jelly.tags.define.InvokeBodyTag#2
            Local Variable: org.apache.commons.jelly.impl.TagScript#925
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#13
            at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            Local Variable: java.util.HashMap#32240
            Local Variable: java.util.HashMap#32060
            at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            Local Variable: org.kohsuke.stapler.jelly.CallTagLibScript$1#3
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: java.net.URL#10985
            Local Variable: org.apache.commons.jelly.impl.TagScript#923
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#14
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#15
            Local Variable: org.apache.commons.jelly.JellyContext#5
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#16
            at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            Local Variable: java.util.HashMap#32241
            Local Variable: java.util.HashMap#32063
            at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            Local Variable: org.kohsuke.stapler.jelly.CallTagLibScript$1#4
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: org.apache.commons.jelly.impl.TagScript#895
            Local Variable: java.net.URL#11206
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#17
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1177
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#18
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1176
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1175
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#19
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1167
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#20
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            Local Variable: org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1#1166
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            Local Variable: java.util.AbstractList$Itr#21
            Local Variable: org.apache.commons.jelly.JellyContext#6
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44)
            Local Variable: org.kohsuke.stapler.jelly.CompressTag#1
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            Local Variable: org.apache.commons.jelly.impl.TagScript#864
            at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
            at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
            Local Variable: org.kohsuke.stapler.jelly.CustomJellyContext#24
            at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
            Local Variable: org.kohsuke.stapler.jelly.HTMLWriterOutput#1
            Local Variable: org.kohsuke.stapler.jelly.DefaultScriptInvoker#1
            at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:107)
            Local Variable: org.kohsuke.stapler.jelly.JellyViewScript#47
            Local Variable: org.kohsuke.stapler.jelly.JellyClassTearOff#62
            at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127)
            Local Variable: org.kohsuke.stapler.jelly.JellyFacet#1
            at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
            Local Variable: java.util.AbstractList$Itr#22
            Local Variable: org.kohsuke.stapler.MetaClass#62
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
            Local Variable: hudson.model.ListView#11
            at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:625)
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
            Local Variable: org.kohsuke.stapler.RequestImpl#1
            Local Variable: hudson.model.Hudson#1
            Local Variable: org.kohsuke.stapler.ResponseImpl#1
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:488)
            at org.kohsuke.stapler.Stapler.service(Stapler.java:162)
            Local Variable: java.lang.String#350622
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
            Local Variable: net.bull.javamelody.JspWrapper$HttpRequestWrapper#1
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
            at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
            Local Variable: java.lang.String#349353
            Local Variable: net.bull.javamelody.CounterServletResponseWrapper#1
            at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
            at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
            at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
            at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
            at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
            Local Variable: hudson.security.ChainedServletFilter$1#1
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:63)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            Local Variable: org.acegisecurity.context.HttpSessionContextIntegrationFilter$OnRedirectUpdateSessionResponseWrapper#1
            at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
            at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
            Local Variable: org.kohsuke.stapler.compression.CompressionServletResponse#1
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            Local Variable: org.apache.catalina.connector.RequestFacade#31
            Local Variable: org.apache.catalina.connector.ResponseFacade#31
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
            Local Variable: org.apache.catalina.core.ApplicationFilterChain#31
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
            at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
            Local Variable: org.apache.catalina.connector.Response#31
            Local Variable: org.apache.catalina.connector.Request#31
            Local Variable: org.apache.coyote.Response#31
            Local Variable: org.apache.coyote.Request#31
            at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
            Local Variable: org.apache.coyote.RequestInfo#31
            at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
            Local Variable: org.apache.coyote.http11.Http11Processor#31
            at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
            Local Variable: java.net.Socket#73
            Local Variable: org.apache.tomcat.util.net.JIoEndpoint$Worker#16
            at java.lang.Thread.run(Thread.java:619)
            See attached stacktrace.
            johno Johno Crawford made changes -
            Description See attached stacktrace. Please see attached stacktrace.
            johno Johno Crawford made changes -
            Description Please see attached stacktrace. After a period of inactivity loading the index page with the job listing hangs in between 30 - 60 seconds. Please see attached stacktrace.
            johno Johno Crawford made changes -
            Summary Lazy loading causes massive delays after a period of inactivity when loading job tabs Lazy loading causes massive delays after a period of inactivity when loading index page with job listing
            johno Johno Crawford made changes -
            Link This issue is related to JENKINS-15858 [ JENKINS-15858 ]
            Hide
            johno Johno Crawford added a comment -

            I suspect this is actually caused by the massive amount of build history we had for some jobs. After enabling fine logging on hudson.model.RunMap it was obvious which jobs were causing Jenkins to hang. I have now configured discard old builds for the jobs, will report back if it helped or not.

            Show
            johno Johno Crawford added a comment - I suspect this is actually caused by the massive amount of build history we had for some jobs. After enabling fine logging on hudson.model.RunMap it was obvious which jobs were causing Jenkins to hang. I have now configured discard old builds for the jobs, will report back if it helped or not.
            johno Johno Crawford made changes -
            Link This issue is related to JENKINS-15858 [ JENKINS-15858 ]
            Hide
            johno Johno Crawford added a comment - - edited

            Purging the build history has helped a little, however when the JVM runs GC Jenkins is forced to reload all the build history again.

            Show
            johno Johno Crawford added a comment - - edited Purging the build history has helped a little, however when the JVM runs GC Jenkins is forced to reload all the build history again.
            Hide
            johno Johno Crawford added a comment - - edited

            Looks like hudson.model.Job.getLastFailedBuild(Job.java:780) is the expensive operation here if the job in question has a long build history with no failures as it will load all history or until it finds a failed job, unstable job etc..

            Show
            johno Johno Crawford added a comment - - edited Looks like hudson.model.Job.getLastFailedBuild(Job.java:780) is the expensive operation here if the job in question has a long build history with no failures as it will load all history or until it finds a failed job, unstable job etc..
            johno Johno Crawford made changes -
            Summary Lazy loading causes massive delays after a period of inactivity when loading index page with job listing Lazy loading causes massive delays after a period of inactivity when loading dashboard
            Hide
            vlatombe Vincent Latombe added a comment -

            <vlatombe> probably now that builds are fetched lazily, it may be worth to store some information at job level like lastSuccessFul/lastUnstableBuild/lastFailedBuild build numbers or timestamps
            [14:26] <Johnou> yes I agree, doesn't make sense to load all jobs until it finds that information
            [14:26] <Johnou> defeats the point of lazy init

            Show
            vlatombe Vincent Latombe added a comment - <vlatombe> probably now that builds are fetched lazily, it may be worth to store some information at job level like lastSuccessFul/lastUnstableBuild/lastFailedBuild build numbers or timestamps [14:26] <Johnou> yes I agree, doesn't make sense to load all jobs until it finds that information [14:26] <Johnou> defeats the point of lazy init
            johno Johno Crawford made changes -
            Assignee Kohsuke Kawaguchi [ kohsuke ]
            johno Johno Crawford made changes -
            Description After a period of inactivity loading the index page with the job listing hangs in between 30 - 60 seconds. Please see attached stacktrace. After a period of inactivity or when the JVM decides to run GC loading the dashboard can easily take longer than 1-2 minutes. Please see attached stacktrace.
            Hide
            hanabishi Marcus Jacobsson added a comment -

            We are also having some problem with long load times after upgrading to a version with lazy loading. We are using getPreviousSuccessfulBuild quite frequently and we will disable the code that uses this operation and see if it helps.

            We have are running a Jenkins server with close to 1000 different jobs and noticed that we have a lot of hits on the root, that I think will list all builds. After looking at some statistics from JavaMelody we see that we have quite a lot of hits to the root and to cc.xml in the root. Could this force Jenkins to load unnecessary build data to the memory? And if so, is there a way to prevent this?
            Request, % of cumulative time, Hits, Mean time (ms)
            /cc.xml GET 11 79,903 21
            / GET 6 127 7,746 101,459

            Show
            hanabishi Marcus Jacobsson added a comment - We are also having some problem with long load times after upgrading to a version with lazy loading. We are using getPreviousSuccessfulBuild quite frequently and we will disable the code that uses this operation and see if it helps. We have are running a Jenkins server with close to 1000 different jobs and noticed that we have a lot of hits on the root, that I think will list all builds. After looking at some statistics from JavaMelody we see that we have quite a lot of hits to the root and to cc.xml in the root. Could this force Jenkins to load unnecessary build data to the memory? And if so, is there a way to prevent this? Request, % of cumulative time, Hits, Mean time (ms) /cc.xml GET 11 79,903 21 / GET 6 127 7,746 101,459
            jglick Jesse Glick made changes -
            Link This issue depends on JENKINS-16089 [ JENKINS-16089 ]
            Hide
            jglick Jesse Glick added a comment -

            The implementation of JENKINS-16089 should fix this for the lastSuccessfulBuild permalink (and corresponding view column). I am perplexed by why other Job.getLast*Build methods were not also converted to use Permalink.

            Show
            jglick Jesse Glick added a comment - The implementation of JENKINS-16089 should fix this for the lastSuccessfulBuild permalink (and corresponding view column). I am perplexed by why other Job.getLast*Build methods were not also converted to use Permalink .
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-8754 [ JENKINS-8754 ]
            jglick Jesse Glick made changes -
            Labels performance
            Hide
            jglick Jesse Glick added a comment -

            If my hypothesis is correct, the fix should be easy, but the tricky part is coming up with a proper functional test. Run.onLoad does not call any listeners. I guess you could attach a RunAction to a build when created then call purgeCache on the run map and check that Job.getLastFailedBuild does not load any builds other than the last failed one.

            Show
            jglick Jesse Glick added a comment - If my hypothesis is correct, the fix should be easy, but the tricky part is coming up with a proper functional test. Run.onLoad does not call any listeners. I guess you could attach a RunAction to a build when created then call purgeCache on the run map and check that Job.getLastFailedBuild does not load any builds other than the last failed one.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/hudson/model/Job.java
            test/src/test/java/hudson/model/JobTest.java
            http://jenkins-ci.org/commit/jenkins/be5cd4fd169005f50eea8ccefac4b91a9b4c503c
            Log:
            [FIXED JENKINS-16023] Use permalinks for as many Job.getLast*Build methods as possible.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/model/Job.java test/src/test/java/hudson/model/JobTest.java http://jenkins-ci.org/commit/jenkins/be5cd4fd169005f50eea8ccefac4b91a9b4c503c Log: [FIXED JENKINS-16023] Use permalinks for as many Job.getLast*Build methods as possible.
            scm_issue_link SCM/JIRA link daemon made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            jglick Jesse Glick made changes -
            Labels performance lts-candidate performance
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #2563
            [FIXED JENKINS-16023] Use permalinks for as many Job.getLast*Build methods as possible. (Revision be5cd4fd169005f50eea8ccefac4b91a9b4c503c)

            Result = SUCCESS
            Jesse Glick : be5cd4fd169005f50eea8ccefac4b91a9b4c503c
            Files :

            • test/src/test/java/hudson/model/JobTest.java
            • core/src/main/java/hudson/model/Job.java
            • changelog.html
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2563 [FIXED JENKINS-16023] Use permalinks for as many Job.getLast*Build methods as possible. (Revision be5cd4fd169005f50eea8ccefac4b91a9b4c503c) Result = SUCCESS Jesse Glick : be5cd4fd169005f50eea8ccefac4b91a9b4c503c Files : test/src/test/java/hudson/model/JobTest.java core/src/main/java/hudson/model/Job.java changelog.html
            kutzi kutzi made changes -
            Link This issue is related to JENKINS-18196 [ JENKINS-18196 ]
            olivergondza Oliver Gondža made changes -
            Labels lts-candidate performance lts-1.509.3-fixed performance
            olivergondza Oliver Gondža made changes -
            Labels lts-1.509.3-fixed performance 1.509.3-fixed performance
            jglick Jesse Glick made changes -
            Link This issue depends on JENKINS-18846 [ JENKINS-18846 ]
            jglick Jesse Glick made changes -
            Labels 1.509.3-fixed performance 1.509.3-fixed lazy-loading performance
            Hide
            pancake pancake added a comment -

            Johno, could you please share information on how to get "Local Variable"s included in a stack trace?

            Show
            pancake pancake added a comment - Johno , could you please share information on how to get "Local Variable"s included in a stack trace?
            Hide
            johno Johno Crawford added a comment -

            @pancake VisualVM 1.3+ shows local variables in thread dumps.

            Show
            johno Johno Crawford added a comment - @pancake VisualVM 1.3+ shows local variables in thread dumps.
            Hide
            pancake pancake added a comment -

            @johno oh, indeed "Local Variable"s are shown in "Threads at the heap dump". Thanks!

            Show
            pancake pancake added a comment - @johno oh, indeed "Local Variable"s are shown in "Threads at the heap dump". Thanks!
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-20989 [ JENKINS-20989 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 146825 ] JNJira + In-Review [ 192101 ]

              People

              • Assignee:
                kohsuke Kohsuke Kawaguchi
                Reporter:
                johno Johno Crawford
              • Votes:
                3 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: