Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62659

Intermittent NotSerializableException when iterating over a LinkedList in a Pipeline

XMLWordPrintable

    • workflow-cps 2.81

      Originally reported as https://github.com/cloudbees/groovy-cps/issues/105.

      Using a LinkedList in a Groovy for in loop in a Pipeline sometimes (but not always!) causes the following NotSerializableException:

      an exception which occurred:
    in field com.cloudbees.groovy.cps.impl.ForInLoopBlock$ContinuationImpl.itr
    in object com.cloudbees.groovy.cps.impl.ForInLoopBlock$ContinuationImpl@4365f5c7
    in field com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.target
    in object com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl@701a709d
    in field com.cloudbees.groovy.cps.impl.LoopBlockScopeEnv.continue_
    in object com.cloudbees.groovy.cps.impl.LoopBlockScopeEnv@44fc4585
    in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
    in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@323c441a
    in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
    in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@5e61e8aa
    in field com.cloudbees.groovy.cps.impl.CallEnv.caller
    in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@7e82466a
    in field com.cloudbees.groovy.cps.Continuable.e
    in object com.cloudbees.groovy.cps.Continuable@125a5d42
    in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
    in object org.jenkinsci.plugins.workflow.cps.CpsThread@3eda4dde
    in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@56f14ac9
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@56f14ac9
Caused: java.io.NotSerializableException: java.util.LinkedList$ListItr
    ...

      Here is a simple Pipeline that can reproduce the problem (make sure you are using the MAX_SURVIVABILITY durability level):

      def col = new LinkedList<>([1, 2, 3])
for (Integer i in col) {
  sleep(time: 500, unit: 'MILLISECONDS') // Force the Pipeline to be serialized.
}

      My diagnosis of the issue can be found in this and earlier comments. When resolving this call internally, sometimes Groovy treats it as a call to List.iterator, but sometimes it treats it as a call to Deque.iterator() (presumably because LinkedList implements both interfaces and there is some nondeterministic behavior somewhere). The method that Groovy chooses changes which IteratorHack category methods are valid (the class of the selected method much match the first parameter of the category method), so adding IteratorHack.iterator(Deque) fixes the issue.

      This probably also affects other types, and maybe we need to add other methods to IteratorHack as well.

      I am not sure about the root cause of the nondeterminism, and I was not able to reproduce it using a standalone Groovy script. See this comment for addition details. Notably, running a JenkinsRule-based reproduction test in a loop inside of an @Test method showed that the test either always fails or always passes in the same JVM instance, so maybe the nondeterminism is caused by something like ordering of methods in Java's reflection APIs, which I think is undefined but consistent during the lifetime of a single JVM instance.

            dnusbaum Devin Nusbaum
            dnusbaum Devin Nusbaum
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: