Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49365

Resuming of pipelines vs. docker.image(...).inside(...)?

XMLWordPrintable

      (Alternative way of asking for help; cf. https://groups.google.com/forum/#!msg/jenkinsci-users/xMxrpRJHIoA/KQoWVbnAAwAJ for my initial attempt)

      When playing around with resuming of pipelines after restarting the Jenkins master (note - not sure if of relevance though: it is a single jenkins master without any slaves setup) during pipeline execution, I stumbled over the the following problem when the restart happens in the course of executing a docker.image(...).inside(...) closure:

      java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131)
    at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:948)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1255)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
    at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
    at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
    at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
    at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
    at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
    at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
    at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
    at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
    at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149)
    at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135)
    at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53)
    at org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
    at java.util.HashMap.readObject(HashMap.java:1409)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:309)
    at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1644)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
    at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
    at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
    at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
    at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:742)
    at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:735)
    at org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
    at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
    at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
    at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
    at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
    at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
    at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.setOneValue(ListFuture.java:158)
    at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.access$000(ListFuture.java:40)
    at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture$2.run(ListFuture.java:107)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
    at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
    at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
    at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
    at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
    at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly.access$500(TryRepeatedly.java:48)
    at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:112)
    at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
Caused: java.io.IOException: Failed to load build state
    at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:814)
    at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:812)
    at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:861)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

      Here is the build log right before the problem and including the resume logs:

       [Pipeline] sh
 [ACME-Pipeline (Branch)-WS] Running shell script
 + docker inspect -f . ACME/phpunit:5
 .
 [Pipeline] withDockerContainer
 Jenkins does not seem to be running inside a container
 $ docker run -t -d -u 10112:10005 --entrypoint -w "/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS" -v "/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS:/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS:rw,z" -v "/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS@tmp:/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS@tmp:rw,z" -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** --entrypoint cat ACME/phpunit:5
 [Pipeline] \{
 [Pipeline] sh
 [ACME-Pipeline (Branch)-WS] Running shell script
 ...
 [Pipeline] sh
 [ACME-Pipeline (Branch)-WS] Running shell script
 + ./gradlew --no-daemon -PphpUnitXml=phpunit-result.xml executePlainPHPUnitTests
 :deleteDependencyCache UP-TO-DATE
 :executePlainPHPUnitTests
 PHPUnit 6.5.5 by Sebastian Bergmann and contributors.

............................................................... 63 / 484 ( 13%)
 ............................................................... 126 / 484 ( 26%)
 ............................................................... 189 / 484 ( 39%)
 ............................................................... 252 / 484 ( 52%)
 ............................................................... 315 / 484 ( 65%)
 ............................................................... 378 / 484 ( 78%)
 ............................................................... 441 / 484 ( 91%)
 ........................................... 484 / 484 (100%)

Time: 10.73 seconds, Memory: 78.00MB

OK (484 tests, 927 assertions)

BUILD SUCCESSFUL in 15s
 2 actionable tasks: 1 executed, 1 up-to-date
 Resuming build at Wed Jan 24 15:57:45 CET 2018 after Jenkins restart
 Waiting to resume part of ACME-Pipeline (Branch) 20180124-155522-revUNKNOWN: Jenkins is about to shut down
 Waiting to resume part of ACME-Pipeline (Branch) 20180124-155522-revUNKNOWN: Jenkins is about to shut down
 Waiting to resume part of ACME-Pipeline (Branch) 20180124-155522-revUNKNOWN: Jenkins is about to shut down
 [Pipeline] End of Pipeline
 java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

      Here is a thread dump after "Manage Jenkins > Prepare for Shutdown" + waiting for the finishing of the "sh" step using gradle to execute PHPUnit and right before restarting the Jenkins master.

      Thread #18
    at com.ACME.php.PHPUnitViaGradle.execute([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/php/PHPUnitViaGradle.groovy:30|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/php/PHPUnitViaGradle.groovy:30])
    at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(jar:[file:/var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:129|file:///var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:129])
    at DSL.withDockerContainer(Native Method)
    at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(jar:[file:/var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:128|file:///var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:128])
    at org.jenkinsci.plugins.docker.workflow.Docker.node(jar:[file:/var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:63|file:///var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:63])
    at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(jar:[file:/var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:116|file:///var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:116])
    at com.ACME.php.PHPUnitViaGradle.execute([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/php/PHPUnitViaGradle.groovy:26|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/php/PHPUnitViaGradle.groovy:26])
    at ACME.executePHPUnitTests(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:191)
    at WorkflowScript.run(WorkflowScript:65)
    at com.ACME.Workspace.prepare([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Workspace.groovy:53|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Workspace.groovy:53])
    at DSL.ws(Native Method)
    at com.ACME.Workspace.prepare([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Workspace.groovy:30|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Workspace.groovy:30])
    at ACME.wsWithArtifacts(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:164)
    at WorkflowScript.run(WorkflowScript:64)
    at DSL.node(running on )
    at WorkflowScript.run(WorkflowScript:55)
    at ACME.skipableStage(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:76)
    at DSL.stage(Native Method)
    at ACME.skipableStage(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:67)
    at ACME.skipableStage(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:57)
    at ACME.skipableStage(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:52)
    at WorkflowScript.run(WorkflowScript:53)
    at com.ACME.Build.build([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Build.groovy:42|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Build.groovy:42])
    at ACME.build(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:48)
    at WorkflowScript.run(WorkflowScript:19)

      Not sure if it is relevant, but the Jenkins setup is very up to date: Jenins v2.103 with very recent update of these plugins:

          ---> Plugins installed:
    ...
    workflow-api:2.25
    workflow-cps:2.43
    workflow-durable-task-step:2.18
    workflow-job:2.17
    workflow-multibranch:2.17
    workflow-support:2.17

      ..., and thus this output at the top of the build log:

          Running in Durability level: MAX_SURVIVABILITY

      Is this supposed to work? If so, what am I missing?
      Or can it work at all?

            Unassigned Unassigned
            reinholdfuereder Reinhold Füreder
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: