Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30271

Jenkins job hangs and cannot be killed after aborting DSL job

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Running job-dsl-plugin off the master branch (78ae0c67c3d0b3ce630cca0987c5fba869c8dfb1) plus 3 of my outstanding pull requests folded in. I don't think that this issue is related to my changes (which are about adding new DSL configuration options for plugins)

      Steps to reproduce:

      1. Run a job with a Process DSL Script task (ideally long-running)
      2. Abort job while it's executing. Job will not stop, but will finish normally.
      3. Run the job again. Job will hang and cannot be aborted.

      Interestingly, Thread.interrupt() from the script console will not kill it. Using Monitoring plugin's kill feature does kill the job, but the next run will still hang. Only a restart of the Jenkins master fixes the problem.

      UPDATE: also happens without aborting jobs, making this more than minor. BTW, I am running this on two masters and it happened only once on one and a lot on the other.

      Stacktrace for the stuck job:

      Executor #1 for master : executing DSL Job Builder #108
      sun.misc.Unsafe.park(Native Method)
      java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
      org.codehaus.groovy.util.LockableObject.lock(LockableObject.java:34)
      org.codehaus.groovy.reflection.ClassInfo.lock(ClassInfo.java:268)
      org.codehaus.groovy.reflection.ClassInfo.getMetaClass(ClassInfo.java:193)
      org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.getMetaClass(MetaClassRegistryImpl.java:231)
      org.codehaus.groovy.runtime.InvokerHelper.getMetaClass(InvokerHelper.java:747)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.createPojoSite(CallSiteArray.java:109)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.createCallSite(CallSiteArray.java:150)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
      javaposse.jobdsl.dsl.helpers.BuildParametersContext.checkParameterName(BuildParametersContext.groovy:270)
      javaposse.jobdsl.dsl.helpers.BuildParametersContext.simpleParam(BuildParametersContext.groovy:187)
      javaposse.jobdsl.dsl.helpers.BuildParametersContext.this$3$simpleParam(BuildParametersContext.groovy)
      javaposse.jobdsl.dsl.helpers.BuildParametersContext$this$3$simpleParam.callCurrent(Unknown Source)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
      javaposse.jobdsl.dsl.helpers.BuildParametersContext$this$3$simpleParam.callCurrent(Unknown Source)
      javaposse.jobdsl.dsl.helpers.BuildParametersContext.stringParam(BuildParametersContext.groovy:179)
      javaposse.jobdsl.dsl.helpers.BuildParametersContext.stringParam(BuildParametersContext.groovy)
      sun.reflect.GeneratedMethodAccessor459.invoke(Unknown Source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:361)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
      PipelineJobsBuilder$_run_closure1_closure9_closure10_closure11_closure20.doCall(PipelineJobsBuilder.groovy:634)
      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      groovy.lang.Closure.call(Closure.java:415)
      groovy.lang.Closure.call(Closure.java:428)
      org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1379)
      org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1351)
      org.codehaus.groovy.runtime.dgm$170.invoke(Unknown Source)
      org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:271)
      org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.callSafe(AbstractCallSite.java:82)
      PipelineJobsBuilder$_run_closure1_closure9_closure10_closure11.doCall(PipelineJobsBuilder.groovy:633)
      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
      PipelineJobsBuilder$_run_closure1_closure9_closure10_closure11.doCall(PipelineJobsBuilder.groovy)
      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:39)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:54)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
      javaposse.jobdsl.dsl.ContextHelper.executeInContext(ContextHelper.groovy:14)
      javaposse.jobdsl.dsl.ContextHelper$executeInContext.call(Unknown Source)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      javaposse.jobdsl.dsl.ContextHelper$executeInContext.call(Unknown Source)
      javaposse.jobdsl.dsl.Job.parameters(Job.groovy:468)
      sun.reflect.GeneratedMethodAccessor626.invoke(Unknown Source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:361)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
      PipelineJobsBuilder$_run_closure1_closure9_closure10.doCall(PipelineJobsBuilder.groovy:632)
      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      groovy.lang.Closure.call(Closure.java:415)
      groovy.lang.Closure.call(Closure.java:428)
      org.codehaus.groovy.runtime.DefaultGroovyMethods.with(DefaultGroovyMethods.java:196)
      org.codehaus.groovy.runtime.dgm$926.invoke(Unknown Source)
      org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoMetaMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:313)
      org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:64)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:69)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
      javaposse.jobdsl.dsl.JobParent.processJob(JobParent.groovy:108)
      sun.reflect.GeneratedMethodAccessor559.invoke(Unknown Source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:272)
      org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:52)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
      org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:57)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:149)
      javaposse.jobdsl.dsl.JobParent.freeStyleJob(JobParent.groovy:42)
      sun.reflect.GeneratedMethodAccessor637.invoke(Unknown Source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:361)
      org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnCurrentN(ScriptBytecodeAdapter.java:78)
      PipelineJobsBuilder$_run_closure1_closure9.doCall(PipelineJobsBuilder.groovy:615)
      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      groovy.lang.Closure.call(Closure.java:415)
      groovy.lang.Closure.call(Closure.java:428)
      org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1379)
      org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1351)
      org.codehaus.groovy.runtime.dgm$170.doMethodInvoke(Unknown Source)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1079)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      groovy.lang.DelegatingMetaClass.invokeMethod(DelegatingMetaClass.java:149)
      org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:39)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
      PipelineJobsBuilder$_run_closure1.doCall(PipelineJobsBuilder.groovy:589)
      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:497)
      org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
      groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
      org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      groovy.lang.Closure.call(Closure.java:415)
      groovy.lang.Closure.call(Closure.java:428)
      org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1379)
      org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1351)
      org.codehaus.groovy.runtime.dgm$170.doMethodInvoke(Unknown Source)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1079)
      groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:903)
      groovy.lang.DelegatingMetaClass.invokeMethod(DelegatingMetaClass.java:149)
      org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:39)
      org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
      org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
      PipelineJobsBuilder.run(PipelineJobsBuilder.groovy:586)
      javaposse.jobdsl.dsl.DslScriptLoader.runDslEngineForParent(DslScriptLoader.java:80)
      javaposse.jobdsl.dsl.DslScriptLoader.runDslEngine(DslScriptLoader.java:123)
      javaposse.jobdsl.plugin.ExecuteDslScripts.perform(ExecuteDslScripts.java:216)
      hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
      hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:761)
      hudson.model.Build$BuildExecution.build(Build.java:203)
      hudson.model.Build$BuildExecution.doRun(Build.java:160)
      hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536)
      hudson.model.Run.execute(Run.java:1741)
      hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
      hudson.model.ResourceController.execute(ResourceController.java:98)
      hudson.model.Executor.run(Executor.java:374)

        Attachments

        1. config.xml
          5 kB
        2. job-dsl.hpi
          2.33 MB
        3. Stuck_seedjob_stacktrace.txt
          20 kB
        4. Thread Dump.txt
          51 kB

          Issue Links

            Activity

              People

              • Assignee:
                daspilker Daniel Spilker
                Reporter:
                akom Alexander Komarov
              • Votes:
                6 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated: