Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47071

BooleanClosureWrapper isn't serializable, causing .every and friends to fail

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: workflow-cps-plugin
    • Labels:
      None
    • Environment:
      workflow-cps 2.40, groovy-cps 1.19
    • Similar Issues:

      Description

      If you do something like

      ['a', 'b'].every { sleep 1; it != null }
      

      in a Pipeline script, you get the always-fun serialization error:

      WARNING	o.j.p.w.cps.CpsThreadGroup#saveProgramIfPossible: program state save failed
      an exception which occurred:
      	in field locals
      	in field parent
      	in field parent
      	in field caller
      	in field parent
      	in field parent
      	in field caller
      	in field e
      	in field program
      	in field threads
      	in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@65d813fb
      Caused: java.io.NotSerializableException: org.codehaus.groovy.runtime.callsite.BooleanClosureWrapper
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
      	at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
      	at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
      	at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
      	at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
      	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
      	at java.util.HashMap.internalWriteEntries(HashMap.java:1777)
      	at java.util.HashMap.writeObject(HashMap.java:1354)
      	at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:483)
      	at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
      	at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
      	at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
      	at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
      	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
      	at java.util.TreeMap.writeObject(TreeMap.java:2433)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:483)
      	at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      	at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
      	at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
      	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:140)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:458)
      Caused: java.io.IOException: Failed to persist /var/folders/pr/24nv8g910wg8vr4b4c33q34r0000gn/T/jenkinsTests.tmp/jenkins6845617186175380474test/jobs/p/builds/1/program.dat
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:472)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:434)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:422)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:362)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230)
      	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
      	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:744)
      

      This is because a bunch of the DGM methods (specifically any, count, every, findAll, findIndexOf, findIndexValues, findLastIndexOf, find, removeAll, retainAll, and split) use a BooleanClosureWrapper to actually evaluate their closures. And neither BooleanClosureWrapper nor BooleanReturningMethodInvoker (a field in BooleanClosureWrapper is of this class) are serializable. So as soon as a serialization attempt hits, bam, it falls over.

      On the upside, I think I can fix this by just adding new methods to Builder that provides the same behavior as BooleanClosureWrapper.call and BooleanClosureWrapper.callForMap.

        Attachments

          Issue Links

            Activity

            abayer Andrew Bayer created issue -
            abayer Andrew Bayer made changes -
            Field Original Value New Value
            Link This issue relates to JENKINS-47064 [ JENKINS-47064 ]
            abayer Andrew Bayer made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            abayer Andrew Bayer made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            Hide
            abayer Andrew Bayer added a comment -
            Show
            abayer Andrew Bayer added a comment - groovy-cps PR up at https://github.com/cloudbees/groovy-cps/pull/75
            abayer Andrew Bayer made changes -
            Remote Link This issue links to "groovy-cps PR #75 (Web Link)" [ 17760 ]
            Hide
            abayer Andrew Bayer added a comment -

            And downstream test PR in workflow-cps up at https://github.com/jenkinsci/workflow-cps-plugin/pull/177

            Show
            abayer Andrew Bayer added a comment - And downstream test PR in workflow-cps up at https://github.com/jenkinsci/workflow-cps-plugin/pull/177
            abayer Andrew Bayer made changes -
            Remote Link This issue links to "workflow-cps PR #177 (Web Link)" [ 17761 ]
            Hide
            abayer Andrew Bayer added a comment -

            Merged, should be in next groovy-cps (i.e., 1.20) and workflow-cps (i.e., 2.41) releases.

            Show
            abayer Andrew Bayer added a comment - Merged, should be in next groovy-cps (i.e., 1.20) and workflow-cps (i.e., 2.41) releases.
            abayer Andrew Bayer made changes -
            Status In Review [ 10005 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            laurent_malvert Laurent Malvert added a comment - - edited

            I may be wrong, but I think I'm also getting hit by this bug in a Jenkinsfile where I use a sh() call's return status as part of a when condition's expression, e.g.:

            stage("reset") {
              when { anyOf {
                expression { params.RESET_WORKSPACE }
                expression { sh(
                    script:"./scripts/build/stage-reset-check.sh ${params.VERSION}",
                    returnStatus:true
                  ) }
              } }
              steps {
                sh './scripts/build/stage-reset.sh'
              }
            }
            Show
            laurent_malvert Laurent Malvert added a comment - - edited I may be wrong, but I think I'm also getting hit by this bug in a Jenkinsfile where I use a sh()  call's return status as part of a when condition's expression , e.g.: stage( "reset" ) { when { anyOf { expression { params.RESET_WORKSPACE } expression { sh( script: "./scripts/build/stage-reset-check.sh ${params.VERSION}" , returnStatus: true ) } } } steps { sh './scripts/build/stage-reset.sh' } }
            Hide
            laurent_malvert Laurent Malvert added a comment -

            And indeed the workaround suggested in https://issues.jenkins-ci.org/browse/JENKINS-47064 by Nathan Vahrenberg worked (discard when conditions, write conditions as a guard in a script block):

            // temp fix using script block, see:
            //   https://issues.jenkins-ci.org/browse/JENKINS-47071
            //   https://issues.jenkins-ci.org/browse/JENKINS-47064
            //   when { anyOf {
            //     expression { params.RESET_WORKSPACE }
            //     expression { sh(
            //       script:"./scripts/build/stage-reset-check.sh ${params.VERSION}",
            //       returnStatus:true
            //     ) }
            //   } }
              steps {
                script {
                  if (params.RESET_WORKSPACE || sh(
                    script:"./scripts/build/stage-reset-check.sh ${params.VERSION}",
                    returnStatus:true
                  )) {
                    sh "./do/stuff.sh"
                  }
                }
              }

            Only satisfactory as a temp fix, but will do until workflow-cps:2.41

            Show
            laurent_malvert Laurent Malvert added a comment - And indeed the workaround suggested in https://issues.jenkins-ci.org/browse/JENKINS-47064  by Nathan Vahrenberg worked (discard when conditions, write conditions as a guard in a script block): // temp fix using script block, see: // https://issues.jenkins-ci.org/browse/JENKINS-47071 // https://issues.jenkins-ci.org/browse/JENKINS-47064 // when { anyOf { // expression { params.RESET_WORKSPACE } // expression { sh( // script: "./scripts/build/stage-reset-check.sh ${params.VERSION}" , // returnStatus: true // ) } // } } steps { script { if (params.RESET_WORKSPACE || sh( script: "./scripts/build/stage-reset-check.sh ${params.VERSION}" , returnStatus: true )) { sh "./ do /stuff.sh" } } } Only satisfactory as a temp fix, but will do until workflow-cps:2.41
            Hide
            abayer Andrew Bayer added a comment -

            Yup, that's the same root cause.

            Show
            abayer Andrew Bayer added a comment - Yup, that's the same root cause.
            Hide
            laurent_malvert Laurent Malvert added a comment - - edited

            I'd consider it slightly higher than minor though, considering this has the potential to break pipelines, and the error message is fairly tricky to troubleshoot to figure out the workaround.

            I see a PR has been sent (but is pending a valid CI build). I suppose workflow-cps:2.41can be expected somewhat shortly?

            Show
            laurent_malvert Laurent Malvert added a comment - - edited I'd consider it slightly higher than minor though, considering this has the potential to break pipelines, and the error message is fairly tricky to troubleshoot to figure out the workaround. I see a PR has been sent (but is pending a valid CI build). I suppose workflow-cps:2.41can be expected somewhat shortly?
            abayer Andrew Bayer made changes -
            Link This issue is duplicated by JENKINS-47064 [ JENKINS-47064 ]
            Hide
            chris5287 Chris Stylianou added a comment -

            @Andrew Bayer Any idea when an update will be release, as this issue is currently breaking all of our pipeline builds

            Show
            chris5287 Chris Stylianou added a comment - @ Andrew Bayer  Any idea when an update will be release, as this issue is currently breaking all of our pipeline builds
            Hide
            abayer Andrew Bayer added a comment -

            I believe it'll be released today - Sam Van Oort, are you still planning to do that workflow-cps release?

            Show
            abayer Andrew Bayer added a comment - I believe it'll be released today - Sam Van Oort , are you still planning to do that workflow-cps release?
            Hide
            svanoort Sam Van Oort added a comment -

            Andrew Bayer Yes – I'm testing since there's a lot of changes in this version and trying to get one community contribution in before cutting.

            Show
            svanoort Sam Van Oort added a comment - Andrew Bayer Yes – I'm testing since there's a lot of changes in this version and trying to get one community contribution in before cutting.
            Hide
            stefanthurnherr Stefan Thurnherr added a comment -

            Andrew Bayer Sam Van Oort Thanks for solving this! Can confirm that "Pipeline: Groovy" 2.41 together with Declarative/Model API/Stage Tags Metadata plugins 1.2.1 works fine for us.

            Show
            stefanthurnherr Stefan Thurnherr added a comment - Andrew Bayer Sam Van Oort Thanks for solving this! Can confirm that "Pipeline: Groovy" 2.41 together with Declarative/Model API/Stage Tags Metadata plugins 1.2.1 works fine for us.
            Hide
            externl Joe George added a comment - - edited

            I'm still getting this error in switch statements.

            switch(foobarstring) {
                case { it.contains("foo") }:
                    echo "foo"
                    break
                case { it.contains("bar") }:
                    echo "bar"
                    break
            }

            Causing:

            Caused: java.io.NotSerializableException: org.codehaus.groovy.runtime.callsite.BooleanClosureWrapper

            Pipeline: Groovy 2.35

            Pipeline: Declarative 1.2.7

            Show
            externl Joe George added a comment - - edited I'm still getting this error in switch statements. switch (foobarstring) { case { it.contains( "foo" ) }: echo "foo" break case { it.contains( "bar" ) }: echo "bar" break } Causing: Caused: java.io.NotSerializableException: org.codehaus.groovy.runtime.callsite.BooleanClosureWrapper Pipeline: Groovy 2.35 Pipeline: Declarative 1.2.7
            Hide
            abayer Andrew Bayer added a comment -

            Joe George - that looks like a different issue specific to closures in case statements. Could you open a new ticket for that in the workflow-cps-plugin component? Thanks!

            Show
            abayer Andrew Bayer added a comment - Joe George - that looks like a different issue specific to closures in case statements. Could you open a new ticket for that in the workflow-cps-plugin component? Thanks!

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                abayer Andrew Bayer
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: