Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15652

ArrayIndexOutOfBoundsException from AbstractLazyLoadRunMap.search

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Hello,

      Our Jenkins instance (1.487 - latest at the time of writing) runs as a Windows service on Windows XP and has 2 slaves running on similar boxes. I came in Monday morning to find all executors on all 3 boxes "Dead". When I clicked to see why they were dead I was greeted by this message:

      Caught exception evaluating: item.isStuck(). Reason: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #749 DESC: lo=39,hi=38,size=39,size2=39

      When I looked in the logs I found the following sequence of stack traces repeated countless times, for the past few hours.

      Any ideas?

      Yannis

      29-Oct-2012 09:16:16 hudson.ExpressionFactory2$JexlExpression evaluate
      WARNING: Caught exception evaluating: item.isStuck(). Reason: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #749 DESC: lo=39,hi=38,size=39,size2=39
      java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #749 DESC: lo=39,hi=38,size=39,size2=39
      at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
      at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
      at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103)
      at hudson.model.Job.getLastBuildsOverThreshold(Job.java:812)
      at hudson.model.Job.getEstimatedDuration(Job.java:819)
      at hudson.model.Queue$BuildableItem.isStuck(Queue.java:1651)
      at sun.reflect.GeneratedMethodAccessor253.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
      at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
      at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
      at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
      at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
      at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
      at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
      at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
      at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
      at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
      at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:625)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:488)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:162)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
      at winstone.ServletConfiguration.execute(ServletConfiguration.java:248)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:376)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
      at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
      at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:331)
      at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:215)
      at winstone.RequestHandlerThread.run(RequestHandlerThread.java:138)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:662)
      29-Oct-2012 09:16:16 jenkins.model.lazy.AbstractLazyLoadRunMap search
      WARNING: Assertion error: failing to load #749 DESC: lo=39,hi=38,size=39,size2=39
      java.lang.Exception
      at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:417)
      at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
      at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103)
      at hudson.model.Job.getLastBuildsOverThreshold(Job.java:812)
      at hudson.model.Job.getEstimatedDuration(Job.java:819)
      at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320)
      at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303)
      at hudson.model.Queue.maintain(Queue.java:1033)
      at hudson.model.Queue$MaintainTask.doRun(Queue.java:1759)
      at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      29-Oct-2012 09:16:16 hudson.triggers.SafeTimerTask run
      SEVERE: Timer task hudson.model.Queue$MaintainTask@1c2e53c failed
      java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #749 DESC: lo=39,hi=38,size=39,size2=39
      at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
      at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
      at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103)
      at hudson.model.Job.getLastBuildsOverThreshold(Job.java:812)
      at hudson.model.Job.getEstimatedDuration(Job.java:819)
      at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320)
      at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303)
      at hudson.model.Queue.maintain(Queue.java:1033)
      at hudson.model.Queue$MaintainTask.doRun(Queue.java:1759)
      at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      29-Oct-2012 09:16:16 jenkins.model.lazy.AbstractLazyLoadRunMap search
      WARNING: Assertion error: failing to load #749 DESC: lo=39,hi=38,size=39,size2=39
      java.lang.Exception
      at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:417)
      at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
      at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103)
      at hudson.model.Job.getLastBuildsOverThreshold(Job.java:812)
      at hudson.model.Job.getEstimatedDuration(Job.java:819)
      at hudson.model.Queue$BuildableItem.isStuck(Queue.java:1651)
      at sun.reflect.GeneratedMethodAccessor253.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
      at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
      at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
      at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
      at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
      at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
      at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
      at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
      at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
      at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
      at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
      at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
      at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:107)
      at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:625)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:488)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:162)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
      at winstone.ServletConfiguration.execute(ServletConfiguration.java:248)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:376)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
      at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
      at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:331)
      at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:215)
      at winstone.RequestHandlerThread.run(RequestHandlerThread.java:138)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:662)

        Attachments

          Issue Links

            Activity

            yannislionis Yannis Lionis created issue -
            Hide
            mglauche Michael Glauche added a comment - - edited

            I'm having exactly the same issue with 1.487
            (running jenkins with jdk 1.6.0_27-b07, on 64bit Windows)

            Show
            mglauche Michael Glauche added a comment - - edited I'm having exactly the same issue with 1.487 (running jenkins with jdk 1.6.0_27-b07, on 64bit Windows)
            azschorn Andreas Zschorn made changes -
            Field Original Value New Value
            Priority Major [ 3 ] Critical [ 2 ]
            azschorn Andreas Zschorn made changes -
            Priority Critical [ 2 ] Blocker [ 1 ]
            Hide
            benken_parasoft Joseph Benken added a comment -

            In 1.488, all my executors were dead. Stack trace from executors:

            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #1449 EXACT: lo=9,hi=7,size=10,size2=10
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:502)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:355)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103)
            at hudson.model.Job.getLastBuildsOverThreshold(Job.java:812)
            at hudson.model.Job.getEstimatedDuration(Job.java:819)
            at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320)
            at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303)
            at hudson.model.Queue.maintain(Queue.java:1033)
            at hudson.model.Queue.pop(Queue.java:861)
            at hudson.model.Executor.grabJob(Executor.java:284)
            at hudson.model.Executor.run(Executor.java:205)

            Show
            benken_parasoft Joseph Benken added a comment - In 1.488, all my executors were dead. Stack trace from executors: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #1449 EXACT: lo=9,hi=7,size=10,size2=10 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418) at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:502) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:355) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:103) at hudson.model.Job.getLastBuildsOverThreshold(Job.java:812) at hudson.model.Job.getEstimatedDuration(Job.java:819) at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320) at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303) at hudson.model.Queue.maintain(Queue.java:1033) at hudson.model.Queue.pop(Queue.java:861) at hudson.model.Executor.grabJob(Executor.java:284) at hudson.model.Executor.run(Executor.java:205)
            Hide
            rogers Roger Scriven added a comment -

            I'm also having a similar issue which is really blocking progress on out CI setup. I'm using Windows Server 2008 R2 with Jenkins running as a service. The problems appeared to start when I upgraded to 1.488 and created new jobs to use the 'View map from file' feature of the Perforce plugin. I subsequently upgraded to 1.489 but have now rolled back to 1.488.

            Any suggestions/workarounds would be appreciated because our CI work is currently at a halt because of this.

            Roger

            Nov 9, 2012 9:10:27 AM hudson.triggers.SafeTimerTask run
            SEVERE: Timer task hudson.model.Queue$MaintainTask@1de2b1 failed
            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #256 DESC: lo=11,hi=10,size=11,size2=11
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:100)
            at hudson.model.Job.getLastBuildsOverThreshold(Job.java:813)
            at hudson.model.Job.getEstimatedDuration(Job.java:820)
            at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320)
            at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303)
            at hudson.model.Queue.maintain(Queue.java:1033)
            at hudson.model.Queue$MaintainTask.doRun(Queue.java:1759)
            at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
            at java.util.TimerThread.mainLoop(Unknown Source)
            at java.util.TimerThread.run(Unknown Source)

            Nov 9, 2012 9:10:27 AM jenkins.model.lazy.AbstractLazyLoadRunMap search
            WARNING: Assertion error: failing to load #256 DESC: lo=11,hi=10,size=11,size2=11
            java.lang.Exception
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:417)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:100)
            at hudson.model.Job.getLastBuildsOverThreshold(Job.java:813)
            at hudson.model.Job.getEstimatedDuration(Job.java:820)

            Show
            rogers Roger Scriven added a comment - I'm also having a similar issue which is really blocking progress on out CI setup. I'm using Windows Server 2008 R2 with Jenkins running as a service. The problems appeared to start when I upgraded to 1.488 and created new jobs to use the 'View map from file' feature of the Perforce plugin. I subsequently upgraded to 1.489 but have now rolled back to 1.488. Any suggestions/workarounds would be appreciated because our CI work is currently at a halt because of this. Roger Nov 9, 2012 9:10:27 AM hudson.triggers.SafeTimerTask run SEVERE: Timer task hudson.model.Queue$MaintainTask@1de2b1 failed java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #256 DESC: lo=11,hi=10,size=11,size2=11 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:100) at hudson.model.Job.getLastBuildsOverThreshold(Job.java:813) at hudson.model.Job.getEstimatedDuration(Job.java:820) at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320) at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303) at hudson.model.Queue.maintain(Queue.java:1033) at hudson.model.Queue$MaintainTask.doRun(Queue.java:1759) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.TimerThread.mainLoop(Unknown Source) at java.util.TimerThread.run(Unknown Source) Nov 9, 2012 9:10:27 AM jenkins.model.lazy.AbstractLazyLoadRunMap search WARNING: Assertion error: failing to load #256 DESC: lo=11,hi=10,size=11,size2=11 java.lang.Exception at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:417) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:100) at hudson.model.Job.getLastBuildsOverThreshold(Job.java:813) at hudson.model.Job.getEstimatedDuration(Job.java:820)
            Hide
            adoolittle Alex Doolittle added a comment -

            Nearly identical issue, all executors dead on all slaves at item.isStuck(); however, instead of ArrayIndexOutOfBounds we're getting a NullPointerException. Jenkins 1.487 hosted on RHEL, with mixed OSes for the executors.

            Show
            adoolittle Alex Doolittle added a comment - Nearly identical issue, all executors dead on all slaves at item.isStuck(); however, instead of ArrayIndexOutOfBounds we're getting a NullPointerException. Jenkins 1.487 hosted on RHEL, with mixed OSes for the executors.
            Hide
            lkeagle Loren Keagle added a comment -

            I'm seeing a similar stack trace randomly. I can generally restart Jenkins and everything will be find for another day or so. Running 1.489 on Windows Server 2008, all 64 bit.

            Here's a snippet of the failure from the build log:

            Time Elapsed 00:04:10.80
            Build step 'Build a Visual Studio project or solution using MSBuild' marked build as failure
            [locks-and-latches] Releasing all the locks
            [locks-and-latches] All the locks released
            Archiving artifacts
            Recording test results
            Processing tests results in file(s) BinaryFiles/Exe/*/_results.trx
            BinaryFiles\Exe\Win32\Debug\UnitTests\gui\test_results.trx
            BinaryFiles\Exe\Win32\Release\UnitTests\gui\test_results.trx
            Description set:
            ERROR: Publisher hudson.tasks.Mailer aborted due to exception
            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #94 EXACT: lo=8,hi=0,size=9,size2=9
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:502)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:355)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:297)
            at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:1015)
            at hudson.model.AbstractProject.hasParticipant(AbstractProject.java:1514)
            at hudson.model.User.getProjects(User.java:444)
            at hudson.scm.MailAddressResolverImpl.findMailAddressFor(MailAddressResolverImpl.java:21)
            at hudson.tasks.MailAddressResolver.resolve(MailAddressResolver.java:100)
            at hudson.tasks.Mailer$UserProperty.getAddress(Mailer.java:530)
            at hudson.tasks.MailSender.buildCulpritList(MailSender.java:407)
            at hudson.tasks.MailSender.createEmptyMail(MailSender.java:367)
            at hudson.tasks.MailSender.createFailureMail(MailSender.java:226)
            at hudson.tasks.MailSender.getMail(MailSender.java:153)
            at hudson.tasks.MailSender.execute(MailSender.java:99)
            at hudson.tasks.Mailer.perform(Mailer.java:115)
            at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
            at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
            at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:779)
            at hudson.model.Build$BuildExecution.post2(Build.java:183)
            at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:726)
            at hudson.model.Run.execute(Run.java:1541)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:236)
            Finished: FAILURE

            Show
            lkeagle Loren Keagle added a comment - I'm seeing a similar stack trace randomly. I can generally restart Jenkins and everything will be find for another day or so. Running 1.489 on Windows Server 2008, all 64 bit. Here's a snippet of the failure from the build log: Time Elapsed 00:04:10.80 Build step 'Build a Visual Studio project or solution using MSBuild' marked build as failure [locks-and-latches] Releasing all the locks [locks-and-latches] All the locks released Archiving artifacts Recording test results Processing tests results in file(s) BinaryFiles/Exe/* / _results.trx BinaryFiles\Exe\Win32\Debug\UnitTests\gui\test_results.trx BinaryFiles\Exe\Win32\Release\UnitTests\gui\test_results.trx Description set: ERROR: Publisher hudson.tasks.Mailer aborted due to exception java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #94 EXACT: lo=8,hi=0,size=9,size2=9 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418) at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:502) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:355) at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:297) at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:1015) at hudson.model.AbstractProject.hasParticipant(AbstractProject.java:1514) at hudson.model.User.getProjects(User.java:444) at hudson.scm.MailAddressResolverImpl.findMailAddressFor(MailAddressResolverImpl.java:21) at hudson.tasks.MailAddressResolver.resolve(MailAddressResolver.java:100) at hudson.tasks.Mailer$UserProperty.getAddress(Mailer.java:530) at hudson.tasks.MailSender.buildCulpritList(MailSender.java:407) at hudson.tasks.MailSender.createEmptyMail(MailSender.java:367) at hudson.tasks.MailSender.createFailureMail(MailSender.java:226) at hudson.tasks.MailSender.getMail(MailSender.java:153) at hudson.tasks.MailSender.execute(MailSender.java:99) at hudson.tasks.Mailer.perform(Mailer.java:115) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:779) at hudson.model.Build$BuildExecution.post2(Build.java:183) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:726) at hudson.model.Run.execute(Run.java:1541) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236) Finished: FAILURE
            Hide
            lkeagle Loren Keagle added a comment -

            Interestingly, first build failed after a restart. Problem seems to be getting worse. This is a brand new build server for us, and I copied the JENKINS_HOME directory directly from our old build server (same OS, just running 1.486 instead). There seems to be many problems related to this new lazy-loading of projects. I wish there was a way to turn it off. I'd rather have a slower startup than these crashes.

            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #88 EXACT: lo=6,hi=0,size=7,size2=7
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:502)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:355)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:297)
            at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:1015)
            at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:139)
            at hudson.model.Job.isLogUpdated(Job.java:264)
            at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1205)
            at hudson.model.AbstractProject.isBuildBlocked(AbstractProject.java:1142)
            at hudson.model.Queue.isBuildBlocked(Queue.java:937)
            at hudson.model.Queue.maintain(Queue.java:1000)
            at hudson.model.Queue.pop(Queue.java:861)
            at hudson.model.Executor.grabJob(Executor.java:284)
            at hudson.model.Executor.run(Executor.java:205)

            Show
            lkeagle Loren Keagle added a comment - Interestingly, first build failed after a restart. Problem seems to be getting worse. This is a brand new build server for us, and I copied the JENKINS_HOME directory directly from our old build server (same OS, just running 1.486 instead). There seems to be many problems related to this new lazy-loading of projects. I wish there was a way to turn it off. I'd rather have a slower startup than these crashes. java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #88 EXACT: lo=6,hi=0,size=7,size2=7 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418) at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:502) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:355) at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:297) at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:1015) at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:139) at hudson.model.Job.isLogUpdated(Job.java:264) at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1205) at hudson.model.AbstractProject.isBuildBlocked(AbstractProject.java:1142) at hudson.model.Queue.isBuildBlocked(Queue.java:937) at hudson.model.Queue.maintain(Queue.java:1000) at hudson.model.Queue.pop(Queue.java:861) at hudson.model.Executor.grabJob(Executor.java:284) at hudson.model.Executor.run(Executor.java:205)
            Hide
            azschorn Andreas Zschorn added a comment - - edited

            Same for us, on windows 2008 RC2 and linux slaves. But the crashes are here all time. All slaves are dead after a short time. If such an unstable mechanism is instruduced, i would have at least the chance to turn it off. Startup time is really a no issue in comparision to not working at all. AbstractLazyLoadRunMap is quite good obfuscated by the programmer. So debugging it is not a suitable option.

            Show
            azschorn Andreas Zschorn added a comment - - edited Same for us, on windows 2008 RC2 and linux slaves. But the crashes are here all time. All slaves are dead after a short time. If such an unstable mechanism is instruduced, i would have at least the chance to turn it off. Startup time is really a no issue in comparision to not working at all. AbstractLazyLoadRunMap is quite good obfuscated by the programmer. So debugging it is not a suitable option.
            Hide
            jeremyamoore Jeremy Moore added a comment - - edited

            Getting a similar error here with similar stack trace. It is always the same "failing to load #195 EXACT: lo=1466,hi=68,size=1466,size2=1466" for us. Could it be an issue with a specific job's data? It seems to be calling "getPreviousBuild" or "getLastBuild". I assume "#195" is the build number? We are on build 1.489. We had finally updated from the last "hudson" build before the fork.

            Show
            jeremyamoore Jeremy Moore added a comment - - edited Getting a similar error here with similar stack trace. It is always the same "failing to load #195 EXACT: lo=1466,hi=68,size=1466,size2=1466" for us. Could it be an issue with a specific job's data? It seems to be calling "getPreviousBuild" or "getLastBuild". I assume "#195" is the build number? We are on build 1.489. We had finally updated from the last "hudson" build before the fork.
            Hide
            marc Marc Reder added a comment - - edited

            We have the same problem with some of our jobs. Jenkins 1.487, Windows x64 Server 2008 R2 SP1, jdk 1.6.0_27
            Interesting thing is, it only happened on nightly jobs.

            23.11.2012 08:15:07 jenkins.InitReactorRunner$1 onTaskFailed
            SCHWERWIEGEND: Failed Loading job userServices-NIGHTLY
            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #21 DESC: lo=15,hi=0,size=18,size2=18
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210)
            at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349)
            at hudson.model.Run.onLoad(Run.java:303)
            at hudson.maven.MavenModuleSetBuild.onLoad(MavenModuleSetBuild.java:140)
            at hudson.model.RunMap.retrieve(RunMap.java:221)
            at hudson.model.RunMap.retrieve(RunMap.java:59)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:297)
            at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:998)
            at hudson.maven.AbstractMavenProject.createTransientActions(AbstractMavenProject.java:184)
            at hudson.maven.MavenModuleSet.createTransientActions(MavenModuleSet.java:365)
            at hudson.model.AbstractProject.updateTransientActions(AbstractProject.java:665)
            at hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:361)
            at hudson.model.AbstractProject.onLoad(AbstractProject.java:299)
            at hudson.maven.MavenModuleSet.onLoad(MavenModuleSet.java:667)
            at hudson.model.Items.load(Items.java:221)
            at jenkins.model.Jenkins$17.run(Jenkins.java:2507)
            at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:146)
            at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:259)
            at jenkins.model.Jenkins$7.runTask(Jenkins.java:883)
            at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:187)
            at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:94)
            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
            at java.lang.Thread.run(Thread.java:662)

            After deleting the build history, we run that project again and got the following error:

            mavenExecutionResult exceptions not empty
            message : Internal error: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #24 ASC: lo=7,hi=0,size=7,size2=7
            cause : Assertion error: failing to load #24 ASC: lo=7,hi=0,size=7,size2=7
            Stack trace :
            org.apache.maven.InternalErrorException: Internal error: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #24 ASC: lo=7,hi=0,size=7,size2=7
            at org.apache.maven.lifecycle.internal.BuilderCommon.handleBuildError(BuilderCommon.java:128)
            at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:95)
            at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
            at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
            at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
            at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319)
            at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
            at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
            at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
            at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
            at hudson.maven.Maven3Builder.call(Maven3Builder.java:98)
            at hudson.maven.Maven3Builder.call(Maven3Builder.java:64)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:326)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
            at java.util.concurrent.FutureTask.run(FutureTask.java:138)
            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
            at java.lang.Thread.run(Thread.java:662)
            Caused by: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #24 ASC: lo=7,hi=0,size=7,size2=7
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
            at hudson.model.AbstractProject.getNearestBuild(AbstractProject.java:1003)
            at hudson.maven.MavenModuleSetBuild.getModuleBuilds(MavenModuleSetBuild.java:272)
            at hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:500)
            at hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:491)
            at sun.reflect.GeneratedMethodAccessor556.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at hudson.model.Executor$1.call(Executor.java:531)
            at hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23)
            at $Proxy75.end(Unknown Source)
            at sun.reflect.GeneratedMethodAccessor1377.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:275)
            at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:256)
            at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:215)
            ... 9 more

            without making a change, the second try worked.

            Show
            marc Marc Reder added a comment - - edited We have the same problem with some of our jobs. Jenkins 1.487, Windows x64 Server 2008 R2 SP1, jdk 1.6.0_27 Interesting thing is, it only happened on nightly jobs. 23.11.2012 08:15:07 jenkins.InitReactorRunner$1 onTaskFailed SCHWERWIEGEND: Failed Loading job userServices-NIGHTLY java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #21 DESC: lo=15,hi=0,size=18,size2=18 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:210) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:303) at hudson.maven.MavenModuleSetBuild.onLoad(MavenModuleSetBuild.java:140) at hudson.model.RunMap.retrieve(RunMap.java:221) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432) at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:297) at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:998) at hudson.maven.AbstractMavenProject.createTransientActions(AbstractMavenProject.java:184) at hudson.maven.MavenModuleSet.createTransientActions(MavenModuleSet.java:365) at hudson.model.AbstractProject.updateTransientActions(AbstractProject.java:665) at hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:361) at hudson.model.AbstractProject.onLoad(AbstractProject.java:299) at hudson.maven.MavenModuleSet.onLoad(MavenModuleSet.java:667) at hudson.model.Items.load(Items.java:221) at jenkins.model.Jenkins$17.run(Jenkins.java:2507) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:146) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:259) at jenkins.model.Jenkins$7.runTask(Jenkins.java:883) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:187) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:94) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) After deleting the build history, we run that project again and got the following error: mavenExecutionResult exceptions not empty message : Internal error: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #24 ASC: lo=7,hi=0,size=7,size2=7 cause : Assertion error: failing to load #24 ASC: lo=7,hi=0,size=7,size2=7 Stack trace : org.apache.maven.InternalErrorException: Internal error: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #24 ASC: lo=7,hi=0,size=7,size2=7 at org.apache.maven.lifecycle.internal.BuilderCommon.handleBuildError(BuilderCommon.java:128) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:95) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59) at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239) at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158) at hudson.maven.Maven3Builder.call(Maven3Builder.java:98) at hudson.maven.Maven3Builder.call(Maven3Builder.java:64) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #24 ASC: lo=7,hi=0,size=7,size2=7 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418) at hudson.model.AbstractProject.getNearestBuild(AbstractProject.java:1003) at hudson.maven.MavenModuleSetBuild.getModuleBuilds(MavenModuleSetBuild.java:272) at hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:500) at hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:491) at sun.reflect.GeneratedMethodAccessor556.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at hudson.model.Executor$1.call(Executor.java:531) at hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23) at $Proxy75.end(Unknown Source) at sun.reflect.GeneratedMethodAccessor1377.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:275) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:256) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:215) ... 9 more without making a change, the second try worked.
            Hide
            aschmacks Alexander Schmacks added a comment - - edited

            I'm not sure that this problem is directly related to a recent Jenkins update.
            In our environment (Jenkins 1.489 on Windows 2003 server) this exception started to appear last Friday. An update to Jenkins 1.491 didn't help. Neither did a downgrade back to 1.489.

            But after thinking about it: we had troubles with our time server last Friday so that suddenly the server clock changed from Friday noon to Thursday evening.
            This lead to some "past" builds referencing previous builds from the "future" - at least this is what Jenkins seems to see now.

            I've noted the problematic build numbers in the log and searched in the build.xml files in our jobs directory. Some single build directories had the wrong timestamp in their name (in the past) - which I corrected to a date that's more appropriate (not knowing the real build timestamp anymore I had to make an educated guess ).
            I.e., it seems that Jenkins builds the job history by the directory names.

            Once these timestamped directory names were corrected the exception didn't appear again, the executor's not dying anymore.

            Show
            aschmacks Alexander Schmacks added a comment - - edited I'm not sure that this problem is directly related to a recent Jenkins update. In our environment (Jenkins 1.489 on Windows 2003 server) this exception started to appear last Friday. An update to Jenkins 1.491 didn't help. Neither did a downgrade back to 1.489. But after thinking about it: we had troubles with our time server last Friday so that suddenly the server clock changed from Friday noon to Thursday evening. This lead to some "past" builds referencing previous builds from the "future" - at least this is what Jenkins seems to see now. I've noted the problematic build numbers in the log and searched in the build.xml files in our jobs directory. Some single build directories had the wrong timestamp in their name (in the past) - which I corrected to a date that's more appropriate (not knowing the real build timestamp anymore I had to make an educated guess ). I.e., it seems that Jenkins builds the job history by the directory names. Once these timestamped directory names were corrected the exception didn't appear again, the executor's not dying anymore.
            Hide
            rogers Roger Scriven added a comment -

            I found Alexander's observation most helpful. We started to experience Dead Executor problems in early November and our clocks 'went back' at the end of October. I did not try to correct build files because at this stage the need to preserve the history is not great, so instead I removed the build directories and I have not experienced any Dead Executor problems since.

            Many thanks Alexander!

            Roger

            Show
            rogers Roger Scriven added a comment - I found Alexander's observation most helpful. We started to experience Dead Executor problems in early November and our clocks 'went back' at the end of October. I did not try to correct build files because at this stage the need to preserve the history is not great, so instead I removed the build directories and I have not experienced any Dead Executor problems since. Many thanks Alexander! Roger
            Hide
            aschmacks Alexander Schmacks added a comment -

            Thanks!

            And yes, it seems that it's safer to delete the problematic builds than "mess" with the build history manually.
            After some days silence the problem appeared again. I deleted the builds then and now it seems that the problem has really vanished.

            Show
            aschmacks Alexander Schmacks added a comment - Thanks! And yes, it seems that it's safer to delete the problematic builds than "mess" with the build history manually. After some days silence the problem appeared again. I deleted the builds then and now it seems that the problem has really vanished.
            Hide
            lkeagle Loren Keagle added a comment -

            I can confirm that removing all build history seems to stabilize the system. I noticed that I had an assortment of items in my build history. Some jobs had each build in a directory based on its build number. Some had build directories that were timestamped. Some were timestamped, but also had a symbolic link named after the build number, but pointing to a timestamp directory. I also manually deleted all of the lastBuilt and lastFailed links in the job directories. Nothing has failed since, but it sucks that I have to rebuild every job in order to turn all those little circles from gray to green again.

            So it seems as though there's some assumptions being made regarding the layout and naming conventions of build output directories. I don't know if it's due to a combination of plugins or the base Jenkins code not cleaning up after itself. I guess it could be related to clocks and DST, but I'm pretty sure my problems started before we jumped forward.

            Show
            lkeagle Loren Keagle added a comment - I can confirm that removing all build history seems to stabilize the system. I noticed that I had an assortment of items in my build history. Some jobs had each build in a directory based on its build number. Some had build directories that were timestamped. Some were timestamped, but also had a symbolic link named after the build number, but pointing to a timestamp directory. I also manually deleted all of the lastBuilt and lastFailed links in the job directories. Nothing has failed since, but it sucks that I have to rebuild every job in order to turn all those little circles from gray to green again. So it seems as though there's some assumptions being made regarding the layout and naming conventions of build output directories. I don't know if it's due to a combination of plugins or the base Jenkins code not cleaning up after itself. I guess it could be related to clocks and DST, but I'm pretty sure my problems started before we jumped forward.
            Hide
            jwernerny John Werner added a comment -

            More info to help track this down:
            The issue did not start happening until we upgraded from 1.472 to 1.492. This was done on 27-Nov-2012. [We've been in a we-need-to-get-this-released mode so we have held off on Jenkins and plug-in offsets for a while.]

            When DST expired, we were still running 1.472. I searched the old err.log files and did not find the issue.

            The machine is a Windows Server 2003, Standard Edition, and Jenkins is running as a service.

            Show
            jwernerny John Werner added a comment - More info to help track this down: The issue did not start happening until we upgraded from 1.472 to 1.492. This was done on 27-Nov-2012. [We've been in a we-need-to-get-this-released mode so we have held off on Jenkins and plug-in offsets for a while.] When DST expired, we were still running 1.472. I searched the old err.log files and did not find the issue. The machine is a Windows Server 2003, Standard Edition, and Jenkins is running as a service.
            Hide
            candrews candrews added a comment -

            This may be the same as JENKINS-15957

            Show
            candrews candrews added a comment - This may be the same as JENKINS-15957
            Hide
            mwpowellhtx Michael Powell added a comment -

            Roger all that: I move the build history aside and it seems to be back on track, no Dead Threads. it does seem like some what of a pickle to have multiple executors, multiple workspaces (workspace, workspace@2, etc), and have a discrepancy in the build logs that fails to track accordingly.

            Show
            mwpowellhtx Michael Powell added a comment - Roger all that: I move the build history aside and it seems to be back on track, no Dead Threads. it does seem like some what of a pickle to have multiple executors, multiple workspaces (workspace, workspace@2, etc), and have a discrepancy in the build logs that fails to track accordingly.
            Hide
            lefrog Pascal Lambert added a comment - - edited

            Kinda same error for us too after we upgrade from 1.486 to 1.493.

            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #249 DESC: lo=247,hi=244,size=414,size2=414

            Find out the cause was we had our projects "build trigger" define both in "post build" task and "Build after other projects are built". For example if Project A depend on Project B. In project B configuration we state as a Post Build task to trigger build of project A. And in Project A we also state as build trigger to build after Project B success build. Although it's a bad configuration we never noticed it because it was fine until version 1.493 (or earlier since we jump straight from 1.486 to 1.493).

            Show
            lefrog Pascal Lambert added a comment - - edited Kinda same error for us too after we upgrade from 1.486 to 1.493. java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #249 DESC: lo=247,hi=244,size=414,size2=414 Find out the cause was we had our projects "build trigger" define both in "post build" task and "Build after other projects are built". For example if Project A depend on Project B. In project B configuration we state as a Post Build task to trigger build of project A. And in Project A we also state as build trigger to build after Project B success build. Although it's a bad configuration we never noticed it because it was fine until version 1.493 (or earlier since we jump straight from 1.486 to 1.493).
            Hide
            lkeagle Loren Keagle added a comment -

            Pascal, from what I recall, those two settings are supposed to be mirrored by design. I've never tried it, but the documentation led me to believe that changing that setting in an upstream project automatically changed the corresponding setting in the downstream project. i.e. you can't disable it in one without disabling it in the other. Is there anything else you did to your configuration while you were resolving your issue?

            Show
            lkeagle Loren Keagle added a comment - Pascal, from what I recall, those two settings are supposed to be mirrored by design. I've never tried it, but the documentation led me to believe that changing that setting in an upstream project automatically changed the corresponding setting in the downstream project. i.e. you can't disable it in one without disabling it in the other. Is there anything else you did to your configuration while you were resolving your issue?
            Hide
            lefrog Pascal Lambert added a comment -

            Indeed I just notice it this morning.
            Downstream project gets schedule right but the Executor still failed all the times
            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #246 DESC: lo=20,hi=18,size=20,size2=20
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207)
            at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349)
            at hudson.model.Run.onLoad(Run.java:305)
            at hudson.model.RunMap.retrieve(RunMap.java:221)
            at hudson.model.RunMap.retrieve(RunMap.java:59)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207)
            at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349)
            at hudson.model.Run.onLoad(Run.java:305)
            at hudson.model.RunMap.retrieve(RunMap.java:221)
            at hudson.model.RunMap.retrieve(RunMap.java:59)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432)
            at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:297)
            at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:1014)
            at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:137)
            at hudson.model.Job.isBuilding(Job.java:256)
            at hudson.model.AbstractProject.getBuildingUpstream(AbstractProject.java:1247)
            at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1212)
            at hudson.model.AbstractProject.isBuildBlocked(AbstractProject.java:1141)
            at hudson.model.Queue.isBuildBlocked(Queue.java:937)
            at hudson.model.Queue.maintain(Queue.java:1000)
            at hudson.model.Queue.pop(Queue.java:861)
            at hudson.model.Executor.grabJob(Executor.java:284)
            at hudson.model.Executor.run(Executor.java:205)

            Show
            lefrog Pascal Lambert added a comment - Indeed I just notice it this morning. Downstream project gets schedule right but the Executor still failed all the times java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #246 DESC: lo=20,hi=18,size=20,size2=20 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:305) at hudson.model.RunMap.retrieve(RunMap.java:221) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:305) at hudson.model.RunMap.retrieve(RunMap.java:221) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432) at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:297) at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:1014) at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:137) at hudson.model.Job.isBuilding(Job.java:256) at hudson.model.AbstractProject.getBuildingUpstream(AbstractProject.java:1247) at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1212) at hudson.model.AbstractProject.isBuildBlocked(AbstractProject.java:1141) at hudson.model.Queue.isBuildBlocked(Queue.java:937) at hudson.model.Queue.maintain(Queue.java:1000) at hudson.model.Queue.pop(Queue.java:861) at hudson.model.Executor.grabJob(Executor.java:284) at hudson.model.Executor.run(Executor.java:205)
            Hide
            mkinzie mkinzie added a comment -

            I think the ArrayIndexOutOfBoundsException on AbstractLazyLoadRunMap is being displayed prior to, and in place of the real error in some instances. I just updated to Jenkins version 1.501, and it had been probably 6 months since we had previously updated, and as soon as I did, I got the aforementioned error - all builds failed immediately because of this. It turned out the real reason for the failure was that we're using the Git Plugin, and had a bad path specified to the git executable. Fixing that fixed everything - but you couldn't tell that that's what the problem was, because instead of getting an error that it couldn't find git, it just spit out the array out of bounds stack trace.

            Show
            mkinzie mkinzie added a comment - I think the ArrayIndexOutOfBoundsException on AbstractLazyLoadRunMap is being displayed prior to, and in place of the real error in some instances. I just updated to Jenkins version 1.501, and it had been probably 6 months since we had previously updated, and as soon as I did, I got the aforementioned error - all builds failed immediately because of this. It turned out the real reason for the failure was that we're using the Git Plugin, and had a bad path specified to the git executable. Fixing that fixed everything - but you couldn't tell that that's what the problem was, because instead of getting an error that it couldn't find git, it just spit out the array out of bounds stack trace.
            Hide
            taksan taksan added a comment -

            It seems we are having a similar problem. All of our executors were dead today with the following stacktrace:

            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #12608 DESC: lo=97,hi=95,size=98,size2=98
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418)
            at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1025)
            at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434)
            at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189)
            at hudson.model.Job.getLastBuildsOverThreshold(Job.java:853)
            at hudson.model.Job.getEstimatedDuration(Job.java:864)
            at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320)
            at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303)
            at hudson.model.Queue.maintain(Queue.java:1033)
            at hudson.model.Queue.pop(Queue.java:861)
            at hudson.model.Executor.grabJob(Executor.java:284)
            at hudson.model.Executor.run(Executor.java:205)

            This weekend, the daylight saving just ended and the clocks went back one hour, so it seems the same problem Alexander described.

            Besides removing the builds, are there other workarounds?

            Show
            taksan taksan added a comment - It seems we are having a similar problem. All of our executors were dead today with the following stacktrace: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #12608 DESC: lo=97,hi=95,size=98,size2=98 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:418) at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1025) at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434) at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189) at hudson.model.Job.getLastBuildsOverThreshold(Job.java:853) at hudson.model.Job.getEstimatedDuration(Job.java:864) at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320) at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303) at hudson.model.Queue.maintain(Queue.java:1033) at hudson.model.Queue.pop(Queue.java:861) at hudson.model.Executor.grabJob(Executor.java:284) at hudson.model.Executor.run(Executor.java:205) This weekend, the daylight saving just ended and the clocks went back one hour, so it seems the same problem Alexander described. Besides removing the builds, are there other workarounds?
            Hide
            christianapel Christian Apel added a comment -

            We have the same issue with Jenkins 1.505 (running on Windows Server 2008 R2 Standard 64bit, deployed in a Tomcat 6 which is configured as a Windows service). In the last few days, I have found all executors dead every morning when I came into the office. This has started suddenly without any configuration changes or complex software updates, but after we have added approximately 400 further builds.

            Wouldn't it be an option to make the lazy loading configurable, so that this could be at least temporarily fixed?

            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #6 DESC: lo=6,hi=4,size=6,size2=6
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432)
            at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1053)
            at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434)
            at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189)
            at hudson.model.Job.getLastBuildsOverThreshold(Job.java:877)
            at hudson.model.Job.getEstimatedDuration(Job.java:888)
            at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320)
            at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303)
            at hudson.model.Queue.maintain(Queue.java:1035)
            at hudson.model.Queue.pop(Queue.java:863)
            at hudson.model.Executor.grabJob(Executor.java:285)
            at hudson.model.Executor.run(Executor.java:206)

            08.04.2013 08:41:45 hudson.ExpressionFactory2$JexlExpression evaluate
            WARNING: Caught exception evaluating: item.isStuck() in /jenkins/. Reason: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #6 DESC: lo=6,hi=4,size=6,size2=6
            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #6 DESC: lo=6,hi=4,size=6,size2=6
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432)
            at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1053)
            at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434)
            at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189)
            at hudson.model.Job.getLastBuildsOverThreshold(Job.java:877)
            at hudson.model.Job.getEstimatedDuration(Job.java:888)
            at hudson.model.Queue$BuildableItem.isStuck(Queue.java:1654)
            at sun.reflect.GeneratedMethodAccessor358.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
            at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
            at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
            at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
            at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
            at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
            at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
            at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
            at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
            at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
            at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
            at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
            at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
            at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44)
            at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
            at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
            at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
            at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
            at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:107)
            at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127)
            at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:562)
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658)
            at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:624)
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658)
            at org.kohsuke.stapler.Stapler.invoke(Stapler.java:487)
            at org.kohsuke.stapler.Stapler.service(Stapler.java:164)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
            at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
            at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
            at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66)
            at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
            at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:64)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
            at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
            at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
            at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:861)
            at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:579)
            at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1584)
            at java.lang.Thread.run(Thread.java:662)

            Show
            christianapel Christian Apel added a comment - We have the same issue with Jenkins 1.505 (running on Windows Server 2008 R2 Standard 64bit, deployed in a Tomcat 6 which is configured as a Windows service). In the last few days, I have found all executors dead every morning when I came into the office. This has started suddenly without any configuration changes or complex software updates, but after we have added approximately 400 further builds. Wouldn't it be an option to make the lazy loading configurable, so that this could be at least temporarily fixed? java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #6 DESC: lo=6,hi=4,size=6,size2=6 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432) at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1053) at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434) at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189) at hudson.model.Job.getLastBuildsOverThreshold(Job.java:877) at hudson.model.Job.getEstimatedDuration(Job.java:888) at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:320) at hudson.model.queue.MappingWorksheet.<init>(MappingWorksheet.java:303) at hudson.model.Queue.maintain(Queue.java:1035) at hudson.model.Queue.pop(Queue.java:863) at hudson.model.Executor.grabJob(Executor.java:285) at hudson.model.Executor.run(Executor.java:206) 08.04.2013 08:41:45 hudson.ExpressionFactory2$JexlExpression evaluate WARNING: Caught exception evaluating: item.isStuck() in /jenkins/. Reason: java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #6 DESC: lo=6,hi=4,size=6,size2=6 java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #6 DESC: lo=6,hi=4,size=6,size2=6 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432) at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1053) at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434) at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189) at hudson.model.Job.getLastBuildsOverThreshold(Job.java:877) at hudson.model.Job.getEstimatedDuration(Job.java:888) at hudson.model.Queue$BuildableItem.isStuck(Queue.java:1654) at sun.reflect.GeneratedMethodAccessor358.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258) at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104) at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83) at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57) at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51) at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80) at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74) at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53) at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:107) at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:562) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:624) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:487) at org.kohsuke.stapler.Stapler.service(Stapler.java:164) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:64) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:861) at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:579) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1584) at java.lang.Thread.run(Thread.java:662)
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-8754 [ JENKINS-8754 ]
            Hide
            kutzi kutzi added a comment -

            There have been several changes/fixes in that area in newer versions (also since 1.505), so I would be good if you could check, if your issues have been resolved in the newest version.

            Show
            kutzi kutzi added a comment - There have been several changes/fixes in that area in newer versions (also since 1.505), so I would be good if you could check, if your issues have been resolved in the newest version.
            Hide
            kutzi kutzi added a comment -

            Related to JENKINS-15642, if not the same

            Show
            kutzi kutzi added a comment - Related to JENKINS-15642 , if not the same
            kutzi kutzi made changes -
            Link This issue is related to JENKINS-15642 [ JENKINS-15642 ]
            Hide
            christianapel Christian Apel added a comment -

            Thanks for the hint. We are still on version 1.505 because of the issue JENKINS-17264, which has been introduced in version 1.506. But according to the latest comment of yesterday, the current snapshot of the m2release-plugin should solve this issue. I will try to install the latest Jenkins version in the next few days.

            Show
            christianapel Christian Apel added a comment - Thanks for the hint. We are still on version 1.505 because of the issue JENKINS-17264 , which has been introduced in version 1.506. But according to the latest comment of yesterday, the current snapshot of the m2release-plugin should solve this issue. I will try to install the latest Jenkins version in the next few days.
            Hide
            christianapel Christian Apel added a comment -

            I have tested the latest Jenkins release 1.511, but had to downgrade again to version 1.505 due to JENKINS-17627.

            Show
            christianapel Christian Apel added a comment - I have tested the latest Jenkins release 1.511, but had to downgrade again to version 1.505 due to JENKINS-17627 .
            Hide
            hostalp Petr H added a comment -

            after trying to upgrade from 1.458 to 1.511 I was getting tons of these exceptions right on startup. Reverted back to 1.458

            Apr 18, 2013 2:47:44 AM jenkins.model.lazy.AbstractLazyLoadRunMap search
            WARNING: Assertion error: failing to load #2255 DESC: lo=2,hi=0,size=3,size2=3
            java.lang.Exception
                    at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:441)
                    at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:220)
                    at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:354)
                    at hudson.model.Run.onLoad(Run.java:319)
                    at hudson.model.RunMap.retrieve(RunMap.java:226)
                    at hudson.model.RunMap.retrieve(RunMap.java:59)
                    at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:667)
                    at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:650)
                    at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:456)
                    at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:220)
                    at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:354)
            ...
            Apr 18, 2013 2:47:44 AM jenkins.InitReactorRunner$1 onTaskFailed
            SEVERE: Failed Loading job client
            java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #2255 DESC: lo=2,hi=0,size=3,size2=3
                    at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:442)
                    at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:220)
                    at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:354)
                    at hudson.model.Run.onLoad(Run.java:319)
                    at hudson.model.RunMap.retrieve(RunMap.java:226)
                    at hudson.model.RunMap.retrieve(RunMap.java:59)
                    at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:667)
                    at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:650)
                    at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:456)
                    at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:220)
                    at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:354)
            Show
            hostalp Petr H added a comment - after trying to upgrade from 1.458 to 1.511 I was getting tons of these exceptions right on startup. Reverted back to 1.458 Apr 18, 2013 2:47:44 AM jenkins.model.lazy.AbstractLazyLoadRunMap search WARNING: Assertion error: failing to load #2255 DESC: lo=2,hi=0,size=3,size2=3 java.lang.Exception at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:441) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:220) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:354) at hudson.model.Run.onLoad(Run.java:319) at hudson.model.RunMap.retrieve(RunMap.java:226) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:667) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:650) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:456) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:220) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:354) ... Apr 18, 2013 2:47:44 AM jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading job client java.lang.ArrayIndexOutOfBoundsException: Assertion error: failing to load #2255 DESC: lo=2,hi=0,size=3,size2=3 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:442) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:220) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:354) at hudson.model.Run.onLoad(Run.java:319) at hudson.model.RunMap.retrieve(RunMap.java:226) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:667) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:650) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:456) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:220) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:354)
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            Just went through all the comments up to this point.

            I think Alexander's comment is spot on. I'm pretty sure the binary search does not tolerate timestamp inconsistencies well (where later builds have younger timestamps.)

            Looking into a failure mode when that happens and see what it takes to gracefully recover from it.

            Show
            kohsuke Kohsuke Kawaguchi added a comment - Just went through all the comments up to this point. I think Alexander's comment is spot on. I'm pretty sure the binary search does not tolerate timestamp inconsistencies well (where later builds have younger timestamps.) Looking into a failure mode when that happens and see what it takes to gracefully recover from it.
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-15642 [ JENKINS-15642 ]
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-15642 [ JENKINS-15642 ]
            jglick Jesse Glick made changes -
            Summary All executors dead with item.isStuck(): ArrayIndexOutOfBoundsException and more in logs ArrayIndexOutOfBoundsException from AbstractLazyLoadRunMap.search
            Labels exception
            Affects Version/s current [ 10162 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-15957 [ JENKINS-15957 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            http://jenkins-ci.org/commit/jenkins/ab2774362f1d5a5510b99a4b8f02236e8ca23d6c
            Log:
            JENKINS-15652 Include directory in assertion messages so it is possible to tell which project is broken.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java http://jenkins-ci.org/commit/jenkins/ab2774362f1d5a5510b99a4b8f02236e8ca23d6c Log: JENKINS-15652 Include directory in assertion messages so it is possible to tell which project is broken.
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #2528
            JENKINS-15652 Include directory in assertion messages so it is possible to tell which project is broken. (Revision ab2774362f1d5a5510b99a4b8f02236e8ca23d6c)

            Result = UNSTABLE
            Jesse Glick : ab2774362f1d5a5510b99a4b8f02236e8ca23d6c
            Files :

            • core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2528 JENKINS-15652 Include directory in assertion messages so it is possible to tell which project is broken. (Revision ab2774362f1d5a5510b99a4b8f02236e8ca23d6c) Result = UNSTABLE Jesse Glick : ab2774362f1d5a5510b99a4b8f02236e8ca23d6c Files : core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Labels exception exception lts-candidate
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            http://jenkins-ci.org/commit/jenkins/6a70f6937ba34add2558d3f4e150d4f10a45fdc1
            Log:
            JENKINS-15652 More clearly distinguishing the three asserts.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java http://jenkins-ci.org/commit/jenkins/6a70f6937ba34add2558d3f4e150d4f10a45fdc1 Log: JENKINS-15652 More clearly distinguishing the three asserts.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
            http://jenkins-ci.org/commit/jenkins/787424881f3530b29c0c74e939f684e2e80af48e
            Log:
            JENKINS-15652 Reproduced at least one assertion failure in a unit test.

            Compare: https://github.com/jenkinsci/jenkins/compare/f1b513cf9166...787424881f35

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java http://jenkins-ci.org/commit/jenkins/787424881f3530b29c0c74e939f684e2e80af48e Log: JENKINS-15652 Reproduced at least one assertion failure in a unit test. Compare: https://github.com/jenkinsci/jenkins/compare/f1b513cf9166...787424881f35
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
            http://jenkins-ci.org/commit/jenkins/972d8bad229a4be6d98c8fcaad6880e1351af3ee
            Log:
            [FIXED JENKINS-15652] “Solving” errors by just return null from ALLRM.search and reducing to one-line warnings.
            @kohsuke confirms that the intent of throwing AIOOBE was to track down definite code errors;
            but in at least some known occurrences of this bug, the problem is invalid data:
            build records on disk with <number>s that are duplicated and/or out of order.
            Since that violates the precondition of the class, we do not pretend to successfully search for such records.
            At some point it would make sense to have the code detect anomalous conditions and initiate a builds folder cleanup.
            Take only the last build of a given number, and discard any earlier with higher number.
            I.e. just set <number> to 0 or something and ignore such builds, or rename build.xml to build-broken.xml.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java http://jenkins-ci.org/commit/jenkins/972d8bad229a4be6d98c8fcaad6880e1351af3ee Log: [FIXED JENKINS-15652] “Solving” errors by just return null from ALLRM.search and reducing to one-line warnings. @kohsuke confirms that the intent of throwing AIOOBE was to track down definite code errors; but in at least some known occurrences of this bug, the problem is invalid data: build records on disk with <number>s that are duplicated and/or out of order. Since that violates the precondition of the class, we do not pretend to successfully search for such records. At some point it would make sense to have the code detect anomalous conditions and initiate a builds folder cleanup. Take only the last build of a given number, and discard any earlier with higher number. I.e. just set <number> to 0 or something and ignore such builds, or rename build.xml to build-broken.xml.
            scm_issue_link SCM/JIRA link daemon made changes -
            Status In Progress [ 3 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #2557
            JENKINS-15652 More clearly distinguishing the three asserts. (Revision 6a70f6937ba34add2558d3f4e150d4f10a45fdc1)
            JENKINS-15652 Reproduced at least one assertion failure in a unit test. (Revision 787424881f3530b29c0c74e939f684e2e80af48e)

            Result = SUCCESS
            Jesse Glick : 6a70f6937ba34add2558d3f4e150d4f10a45fdc1
            Files :

            • core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java

            Jesse Glick : 787424881f3530b29c0c74e939f684e2e80af48e
            Files :

            • core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2557 JENKINS-15652 More clearly distinguishing the three asserts. (Revision 6a70f6937ba34add2558d3f4e150d4f10a45fdc1) JENKINS-15652 Reproduced at least one assertion failure in a unit test. (Revision 787424881f3530b29c0c74e939f684e2e80af48e) Result = SUCCESS Jesse Glick : 6a70f6937ba34add2558d3f4e150d4f10a45fdc1 Files : core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java Jesse Glick : 787424881f3530b29c0c74e939f684e2e80af48e Files : core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #2558
            [FIXED JENKINS-15652] “Solving” errors by just return null from ALLRM.search and reducing to one-line warnings. (Revision 972d8bad229a4be6d98c8fcaad6880e1351af3ee)

            Result = SUCCESS
            Jesse Glick : 972d8bad229a4be6d98c8fcaad6880e1351af3ee
            Files :

            • core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
            • changelog.html
            • core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2558 [FIXED JENKINS-15652] “Solving” errors by just return null from ALLRM.search and reducing to one-line warnings. (Revision 972d8bad229a4be6d98c8fcaad6880e1351af3ee) Result = SUCCESS Jesse Glick : 972d8bad229a4be6d98c8fcaad6880e1351af3ee Files : core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java changelog.html core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-18289 [ JENKINS-18289 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            http://jenkins-ci.org/commit/jenkins/e1f617cd25f37210d4902a24c6e63771cfb0c63d
            Log:
            JENKINS-15652 Include directory in assertion messages so it is possible to tell which project is broken.(cherry picked from commit ab2774362f1d5a5510b99a4b8f02236e8ca23d6c)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java http://jenkins-ci.org/commit/jenkins/e1f617cd25f37210d4902a24c6e63771cfb0c63d Log: JENKINS-15652 Include directory in assertion messages so it is possible to tell which project is broken.(cherry picked from commit ab2774362f1d5a5510b99a4b8f02236e8ca23d6c)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            http://jenkins-ci.org/commit/jenkins/65e0f9c974f9316b6ba6b2c9fa6a73b2a8013d67
            Log:
            JENKINS-15652 More clearly distinguishing the three asserts.(cherry picked from commit 6a70f6937ba34add2558d3f4e150d4f10a45fdc1)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java http://jenkins-ci.org/commit/jenkins/65e0f9c974f9316b6ba6b2c9fa6a73b2a8013d67 Log: JENKINS-15652 More clearly distinguishing the three asserts.(cherry picked from commit 6a70f6937ba34add2558d3f4e150d4f10a45fdc1)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
            http://jenkins-ci.org/commit/jenkins/b6b760b03fde987e452cdda75c27ca051208196f
            Log:
            JENKINS-15652 Reproduced at least one assertion failure in a unit test.(cherry picked from commit 787424881f3530b29c0c74e939f684e2e80af48e)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java http://jenkins-ci.org/commit/jenkins/b6b760b03fde987e452cdda75c27ca051208196f Log: JENKINS-15652 Reproduced at least one assertion failure in a unit test.(cherry picked from commit 787424881f3530b29c0c74e939f684e2e80af48e)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
            core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
            http://jenkins-ci.org/commit/jenkins/2af3d15442547bb0d3ddd13fd2af10c33beef258
            Log:
            [FIXED JENKINS-15652] “Solving” errors by just return null from ALLRM.search and reducing to one-line warnings. @kohsuke confirms that the intent of throwing AIOOBE was to track down definite code errors; but in at least some known occurrences of this bug, the problem is invalid data: build records on disk with <number>s that are duplicated and/or out of order. Since that violates the precondition of the class, we do not pretend to successfully search for such records. At some point it would make sense to have the code detect anomalous conditions and initiate a builds folder cleanup. Take only the last build of a given number, and discard any earlier with higher number. I.e. just set <number> to 0 or something and ignore such builds, or rename build.xml to build-broken.xml.

            Cherry-picked-from: 972d8bad229a4be6d98c8fcaad6880e1351af3ee

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java http://jenkins-ci.org/commit/jenkins/2af3d15442547bb0d3ddd13fd2af10c33beef258 Log: [FIXED JENKINS-15652] “Solving” errors by just return null from ALLRM.search and reducing to one-line warnings. @kohsuke confirms that the intent of throwing AIOOBE was to track down definite code errors; but in at least some known occurrences of this bug, the problem is invalid data: build records on disk with <number>s that are duplicated and/or out of order. Since that violates the precondition of the class, we do not pretend to successfully search for such records. At some point it would make sense to have the code detect anomalous conditions and initiate a builds folder cleanup. Take only the last build of a given number, and discard any earlier with higher number. I.e. just set <number> to 0 or something and ignore such builds, or rename build.xml to build-broken.xml. Cherry-picked-from: 972d8bad229a4be6d98c8fcaad6880e1351af3ee
            jglick Jesse Glick made changes -
            Labels exception lts-candidate 1.509.2-fixed exception
            jglick Jesse Glick made changes -
            Labels 1.509.2-fixed exception 1.509.2-fixed exception lazy-loading
            Hide
            hx_unbanned Linards L added a comment - - edited

            So ... is it fixed in non-LTS? v1.529.? Or simply resolved ... making the real fix postponed ... ?

            I am still having this error on 1.529 ..

            Sep 12, 2013 8:41:31 AM WARNING jenkins.model.lazy.AbstractLazyLoadRunMap search
            JENKINS-15652 Assertion error #1: failing to load E:\Hudson\jobs\<job_name>\builds #1 DESC: lo=3,hi=1,size=175,size2=175

            I am interested in this because currently our production builserver is pretty much bug-free ... except this one ...

            Show
            hx_unbanned Linards L added a comment - - edited So ... is it fixed in non-LTS? v1.529.? Or simply resolved ... making the real fix postponed ... ? I am still having this error on 1.529 .. Sep 12, 2013 8:41:31 AM WARNING jenkins.model.lazy.AbstractLazyLoadRunMap search JENKINS-15652 Assertion error #1: failing to load E:\Hudson\jobs\<job_name>\builds #1 DESC: lo=3,hi=1,size=175,size2=175 I am interested in this because currently our production builserver is pretty much bug-free ... except this one ...
            Hide
            jglick Jesse Glick added a comment -

            As in my commit comment:

            …and reducing to one-line warnings.

            JENKINS-18289 (in 1.526) is the real fix.

            Show
            jglick Jesse Glick added a comment - As in my commit comment: …and reducing to one-line warnings. JENKINS-18289 (in 1.526) is the real fix.
            evernat evernat made changes -
            Link This issue is related to JENKINS-16060 [ JENKINS-16060 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-17043 [ JENKINS-17043 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-18020 [ JENKINS-18020 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 146408 ] JNJira + In-Review [ 191919 ]

              People

              • Assignee:
                Unassigned
                Reporter:
                yannislionis Yannis Lionis
              • Votes:
                25 Vote for this issue
                Watchers:
                40 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: