Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47202

Using readFile step in environment scope causes error

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      The readFile step, when used inside a environment closure, whether top-level or in a stage, causes the following error:
      an exception which occurred:
      in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
      in object com.cloudbees.groovy.cps.impl.LoopBlockScopeEnv@29044815
      in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@25c9f135
      in field com.cloudbees.groovy.cps.impl.CallEnv.caller
      in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@307ab985
      in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@5a92c230
      in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@37a0a42f
      in field com.cloudbees.groovy.cps.impl.CallEnv.caller
      in object com.cloudbees.groovy.cps.impl.ClosureCallEnv@184a6ff5
      in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@676c6c8d
      in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@19f01356
      in field com.cloudbees.groovy.cps.impl.CallEnv.caller
      in object com.cloudbees.groovy.cps.impl.ClosureCallEnv@74d1467b
      in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@4d098490
      in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@28223d82
      in field com.cloudbees.groovy.cps.impl.CallEnv.caller
      in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@6e27611b
      in field com.cloudbees.groovy.cps.Continuable.e
      in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable@78ff9c41
      in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
      in object org.jenkinsci.plugins.workflow.cps.CpsThread@7841b6fe
      in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
      in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@4d2d90ce
      in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@4d2d90ce
      Caused: java.io.NotSerializableException: java.util.TreeMap$Entry
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
      at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
      at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
      at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
      at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
      at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
      at java.util.HashMap.internalWriteEntries(HashMap.java:1785)
      at java.util.HashMap.writeObject(HashMap.java:1362)
      at sun.reflect.GeneratedMethodAccessor134.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:273)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
      at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
      at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
      at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
      at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
      at java.util.TreeMap.writeObject(TreeMap.java:2438)
      at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:273)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
      at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
      at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
      at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:140)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:458)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:434)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:422)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:362)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230)
      at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
      at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)

      A test repo was created to replicate this.

      https://github.com/sflynn-dell/pipeline-test

      Branches:
      declarative-script - readFile is successful when used inside a script closure.
      declarative-env - readFile fails when used inside an environment enclosure.

        Attachments

          Issue Links

            Activity

            stephen_flynn Stephen Flynn created issue -
            Hide
            lucas_weka Lucas Kögel added a comment -

            All our pipelines are build like this:

            environment {
              POM_VERSION = readMavenPom().getVersion()
              VERSION = "${POM_VERSION}-${env.BUILD_NUMBER}"
            ...
            
            

            Every pipeline and every build is failing now, because of the 

            jenkins java.io.NotSerializableException: java.util.TreeMap$Entry
            
            Show
            lucas_weka Lucas Kögel added a comment - All our pipelines are build like this: environment { POM_VERSION = readMavenPom().getVersion() VERSION = "${POM_VERSION}-${env.BUILD_NUMBER}" ... Every pipeline and every build is failing now, because of the  jenkins java.io.NotSerializableException: java.util.TreeMap$Entry
            arakem Adam Arakelian made changes -
            Field Original Value New Value
            Priority Minor [ 4 ] Blocker [ 1 ]
            Hide
            arakem Adam Arakelian added a comment -

            Yes, this is an enormous issue.  It broke all our pipeline builds.  I'm not sure how someone could have released this with such an enormous bug.  It's not minor, it's critical, anyone who uses this is broken.  We need this resolved immediately please and we need someone to look at this ASAP.

            Show
            arakem Adam Arakelian added a comment - Yes, this is an enormous issue.  It broke all our pipeline builds.  I'm not sure how someone could have released this with such an enormous bug.  It's not minor, it's critical, anyone who uses this is broken.  We need this resolved immediately please and we need someone to look at this ASAP.
            Hide
            stephen_flynn Stephen Flynn added a comment -

            Checked against Jenkins 2.82, Pipeline Groovy Plugin 2.41.  Still throwing the same error.

            Show
            stephen_flynn Stephen Flynn added a comment - Checked against Jenkins 2.82, Pipeline Groovy Plugin 2.41.  Still throwing the same error.
            Hide
            bconner Bill Conner added a comment -

            This issue is breaking all of our pipeline builds too. Does anyone have a roll back procedure? 

            Show
            bconner Bill Conner added a comment - This issue is breaking all of our pipeline builds too. Does anyone have a roll back procedure? 
            abayer Andrew Bayer made changes -
            Component/s pipeline-model-definition-plugin [ 21706 ]
            Component/s workflow-cps-plugin [ 21713 ]
            Assignee Andrew Bayer [ abayer ]
            Hide
            abayer Andrew Bayer added a comment -

            I'm reproducing this with Declarative 1.2.1. Working on it now.

            Show
            abayer Andrew Bayer added a comment - I'm reproducing this with Declarative 1.2.1. Working on it now.
            abayer Andrew Bayer made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            abayer Andrew Bayer made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            abayer Andrew Bayer made changes -
            Remote Link This issue links to "PR #204 (Web Link)" [ 17815 ]
            Hide
            abayer Andrew Bayer added a comment -

            Thanks for the great reproduction case, Stephen Flynn - I'm 99% sure I've found and fixed the problem in https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/204. This is critical enough that once it gets merged, I'll cut a Declarative 1.2.2 release to pick it up.

            Show
            abayer Andrew Bayer added a comment - Thanks for the great reproduction case, Stephen Flynn - I'm 99% sure I've found and fixed the problem in https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/204 . This is critical enough that once it gets merged, I'll cut a Declarative 1.2.2 release to pick it up.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Andrew Bayer
            Path:
            pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Environment.groovy
            pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
            pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/EnvironmentTest.java
            pipeline-model-definition/src/test/resources/readFileInEnv-data.txt
            pipeline-model-definition/src/test/resources/readFileInEnv.groovy
            http://jenkins-ci.org/commit/pipeline-model-definition-plugin/03f4dcf0fc9dc9e72e03dd7ee0be82f8a9ee5357
            Log:
            [FIXED JENKINS-47202] Switch to HashMaps for env to fix serialization

            TreeMap#Entry isn't serializable, which can bite you in the weirdest
            places. Like here - if `environment` variable values end up triggering
            a program save, tada, you get a `NotSerializableException`. Which is
            bad. I can't reproduce this consistently - maybe 3 times out of 4,
            it'll fail. But this fix has yet to reproduce the error after 20+
            attempts, so I think it's good.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Environment.groovy pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/EnvironmentTest.java pipeline-model-definition/src/test/resources/readFileInEnv-data.txt pipeline-model-definition/src/test/resources/readFileInEnv.groovy http://jenkins-ci.org/commit/pipeline-model-definition-plugin/03f4dcf0fc9dc9e72e03dd7ee0be82f8a9ee5357 Log: [FIXED JENKINS-47202] Switch to HashMaps for env to fix serialization TreeMap#Entry isn't serializable, which can bite you in the weirdest places. Like here - if `environment` variable values end up triggering a program save, tada, you get a `NotSerializableException`. Which is bad. I can't reproduce this consistently - maybe 3 times out of 4, it'll fail. But this fix has yet to reproduce the error after 20+ attempts, so I think it's good.
            Hide
            abayer Andrew Bayer added a comment -

            Will be in Declarative 1.2.2, coming out in the next day or so.

            Show
            abayer Andrew Bayer added a comment - Will be in Declarative 1.2.2, coming out in the next day or so.
            abayer Andrew Bayer made changes -
            Status In Review [ 10005 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                stephen_flynn Stephen Flynn
              • Votes:
                10 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: