Details

    • Similar Issues:

      Description

      A user encountered a hung startup after around a minute of activity. Successive thread dumps showed that initialization was paused here:

      "ModelList.init" daemon prio=10 … in Object.wait() …
         java.lang.Thread.State: RUNNABLE
      	at org.codehaus.groovy.util.ReferenceBundle.<clinit>(ReferenceBundle.java:37)
      	at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.<init>(MetaClassRegistryImpl.java:53)
      	at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.<init>(MetaClassRegistryImpl.java:61)
      	at groovy.lang.GroovySystem.<clinit>(GroovySystem.java:29)
      	at org.codehaus.groovy.runtime.InvokerHelper.<clinit>(InvokerHelper.java:49)
      	at groovy.lang.GroovyObjectSupport.<init>(GroovyObjectSupport.java:32)
      	at groovy.lang.Binding.<init>(Binding.java:34)
      	at groovy.lang.GroovyShell.<init>(GroovyShell.java:82)
      	at com.cloudbees.hudson.plugins.modeling.ModelList.init(ModelList.java:71)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:616)
      	at hudson.init.InitializerFinder.invoke(InitializerFinder.java:120)
      	at hudson.init.InitializerFinder$TaskImpl.run(InitializerFinder.java:184)
      	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:259)
      	at jenkins.model.Jenkins$6.runTask(Jenkins.java:840)
      	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:187)
      	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:94)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      	at java.lang.Thread.run(Thread.java:636)
      

      The line of code in question (assuming Groovy 1.8.5):

      ReferenceManager callBack = ReferenceManager.createCallBackedManager(queue);
      

      There were tons of request threads, mostly hung in something like:

      "Handling GET /…some page… : http-8080-2" daemon prio=10 … in Object.wait() …
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00002aaabb3400a8> (a com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference)
      	at java.lang.Object.wait(Object.java:502)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.waitForValue(ComputingConcurrentHashMap.java:328)
      	- locked <0x00002aaabb3400a8> (a com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.getOrCompute(ComputingConcurrentHashMap.java:160)
      	at com.google.common.collect.ComputingConcurrentHashMap.getOrCompute(ComputingConcurrentHashMap.java:69)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingMapAdapter.get(ComputingConcurrentHashMap.java:393)
      	at org.kohsuke.stapler.CachingScriptLoader.findScript(CachingScriptLoader.java:61)
      	at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:112)
      	…
      

      But one was different:

      "Handling GET / : http-8080-1" daemon prio=10 … in Object.wait() …
         java.lang.Thread.State: RUNNABLE
      	at org.codehaus.groovy.util.ReferenceManager.<clinit>(ReferenceManager.java:153)
      	at org.codehaus.groovy.util.ManagedReference.<clinit>(ManagedReference.java:25)
      	at org.codehaus.groovy.reflection.ReflectionCache.getCachedClass(ReflectionCache.java:107)
      	at org.codehaus.groovy.reflection.ReflectionCache.<clinit>(ReflectionCache.java:52)
      	at org.codehaus.groovy.classgen.asm.BytecodeHelper.box(BytecodeHelper.java:585)
      	at org.codehaus.groovy.classgen.asm.BytecodeHelper.box(BytecodeHelper.java:577)
      	at org.codehaus.groovy.classgen.asm.OperandStack.box(OperandStack.java:183)
      	at org.codehaus.groovy.classgen.asm.InvocationWriter.makeCall(InvocationWriter.java:220)
      	at org.codehaus.groovy.classgen.asm.InvocationWriter.makeInvokeMethodCall(InvocationWriter.java:66)
      	at org.codehaus.groovy.classgen.asm.InvocationWriter.writeInvokeMethod(InvocationWriter.java:292)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitMethodCallExpression(AsmClassGenerator.java:655)
      	at org.codehaus.groovy.ast.expr.MethodCallExpression.visit(MethodCallExpression.java:75)
      	at org.codehaus.groovy.classgen.asm.StatementWriter.writeExpressionStatement(StatementWriter.java:599)
      	at org.codehaus.groovy.classgen.asm.OptimizingStatementWriter.writeExpressionStatement(OptimizingStatementWriter.java:354)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitExpressionStatement(AsmClassGenerator.java:501)
      	at org.codehaus.groovy.ast.stmt.ExpressionStatement.visit(ExpressionStatement.java:40)
      	at org.codehaus.groovy.classgen.asm.StatementWriter.writeBlockStatement(StatementWriter.java:80)
      	at org.codehaus.groovy.classgen.asm.OptimizingStatementWriter.writeBlockStatement(OptimizingStatementWriter.java:155)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitBlockStatement(AsmClassGenerator.java:447)
      	at org.codehaus.groovy.ast.stmt.BlockStatement.visit(BlockStatement.java:69)
      	at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClassCodeContainer(ClassCodeVisitorSupport.java:101)
      	at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitConstructorOrMethod(ClassCodeVisitorSupport.java:112)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitStdMethod(AsmClassGenerator.java:312)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitConstructorOrMethod(AsmClassGenerator.java:269)
      	at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitConstructor(ClassCodeVisitorSupport.java:119)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitConstructor(AsmClassGenerator.java:384)
      	at org.codehaus.groovy.ast.ClassNode.visitContents(ClassNode.java:1052)
      	at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClass(ClassCodeVisitorSupport.java:50)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitClass(AsmClassGenerator.java:173)
      	at org.codehaus.groovy.control.CompilationUnit$14.call(CompilationUnit.java:767)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:967)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:546)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:524)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:501)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:306)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:287)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:267)
      	at org.kohsuke.stapler.jelly.groovy.GroovyClassLoaderTearOff.parse(GroovyClassLoaderTearOff.java:82)
      	at org.kohsuke.stapler.jelly.groovy.GroovyClassTearOff.parseScript(GroovyClassTearOff.java:57)
      	at org.kohsuke.stapler.jelly.groovy.GroovyClassTearOff.parseScript(GroovyClassTearOff.java:46)
      	at org.kohsuke.stapler.AbstractTearOff.resolveScript(AbstractTearOff.java:72)
      	at org.kohsuke.stapler.jelly.JellyClassTearOff.resolveScript(JellyClassTearOff.java:80)
      	at org.kohsuke.stapler.jelly.JellyClassTearOff.resolveScript(JellyClassTearOff.java:47)
      	at org.kohsuke.stapler.AbstractTearOff.loadScript(AbstractTearOff.java:78)
      	at org.kohsuke.stapler.CachingScriptLoader$1.apply(CachingScriptLoader.java:30)
      	at org.kohsuke.stapler.CachingScriptLoader$1.apply(CachingScriptLoader.java:27)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute(ComputingConcurrentHashMap.java:355)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.compute(ComputingConcurrentHashMap.java:184)
      	- locked <0x00002aaabb340170> (a com.google.common.collect.MapMakerInternalMap$StrongEntry)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.getOrCompute(ComputingConcurrentHashMap.java:153)
      	at com.google.common.collect.ComputingConcurrentHashMap.getOrCompute(ComputingConcurrentHashMap.java:69)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingMapAdapter.get(ComputingConcurrentHashMap.java:393)
      	at org.kohsuke.stapler.CachingScriptLoader.findScript(CachingScriptLoader.java:61)
      	at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:112)
      	…
      

      which is in:

      SOFT_BUNDLE = new ReferenceBundle(manager, ReferenceType.SOFT);
      

      Seems like a class loading deadlock: two static initializers referring to one another, and two threads each initially referring to one of these classes.

      Groovy 2.0.5 has the same code. Cannot find any existing reports of this, so it is probably a rare race condition.

        Attachments

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              jglick Jesse Glick
              Reporter:
              jglick Jesse Glick
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: