Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56343

Intermittent java.io.NotSerializableException when running scripts in parallel

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: pipeline
    • Labels:
      None
    • Environment:
    • Similar Issues:

      Description

      I receive an intermittent java.io.NotSerializableException when running a script in parallel

       

      an exception which occurred:
      	in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
      	in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@53ce7f91
      ...
      Caused: java.io.NotSerializableException: java.util.regex.Matcher
      	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:926)
      ...

       

      The stack trace completely masks the location of the offending code. The offending code is when I'm storing the result of a matcher in a local variable, the main issue is that it was very difficult to locate and that it occurs inconsistently.

      I've reduced the steps required to reproduce to the following script:

       

      import com.cloudbees.groovy.cps.NonCPS
      
      //Fails when it enters this method, only if method is annotated NonCPS
      @NonCPS
      def getMessage(def auth) {
          return '[abc][def]asdasd'
      }
      
      def getTags() {
      //Below line is the offending code, it seems to break CPS rules, but it only fails intermittently
          def tagMatches = getMessage(credentials) =~ /\[(.*?)\]/
          def tagsList = []
          for (def tagMatch in tagMatches) {
              tagsList << (tagMatch[1])?.toLowerCase()
          }
          tagsList
      }
      
      //This doesn't seem as though it would be relevant, but if I remove it when reproducing the failure doesn't seem to appear, perhaps it causes some necessary delay that causes serialisation in the parallel run?
      def getCredentials() {
          withCredentials([
                  [$class: 'StringBinding', credentialsId: 'SOME_CREDENTIALS', variable: 'TOKEN']
          ])
                  {
                      "any"
                  }
      }
      
      def script = {
          stage(it) {
              println tags
          }
      }
      
      def scripts = [:]
      
      for (int i = 0; i < 5; i++) {
          def runName = "run ${i}"
          scripts."run ${i}" = { script(runName) }
      }
      
      //Running the scripts sequentially does not cause issue
       for ( def _script:scripts.values()){
           _script()
       }
      //Running the scripts in parallel causes the issue
      parallel scripts
      

       

       

        Attachments

          Activity

          Hide
          aedwards Alun Edwards added a comment -

          Appears to be similar issue as https://issues.jenkins-ci.org/browse/JENKINS-46597

          Although JENKINS-46597 was resolved because they fixed their underlying issue (use of a tree map), e.g. and for me I could resolve this by storing the result of the matcher in a method not marked as non-cps, for me the outstanding issues are that

          1. The issue happens inconsistently, surely the aforementioned code should always be a violation or should always pass
          2. The issue is very hard to locate. The stack trace gives no indication as to where the cause is. In my case it was a regex matcher, which is used in many places in a pipeline library we have. Even adding logging statements on every line it was still difficult, as where the code is actually triggered is (when it enters the @NonCPS method) is not where the problematic variable is.
          Show
          aedwards Alun Edwards added a comment - Appears to be similar issue as https://issues.jenkins-ci.org/browse/JENKINS-46597 Although JENKINS-46597 was resolved because they fixed their underlying issue (use of a tree map), e.g. and for me I could resolve this by storing the result of the matcher in a method not marked as non-cps, for me the outstanding issues are that The issue happens inconsistently, surely the aforementioned code should always be a violation or should always pass The issue is very hard to locate. The stack trace gives no indication as to where the cause is. In my case it was a regex matcher, which is used in many places in a pipeline library we have. Even adding logging statements on every line it was still difficult, as where the code is actually triggered is (when it enters the @NonCPS method) is not where the problematic variable is.

            People

            • Assignee:
              Unassigned
              Reporter:
              aedwards Alun Edwards
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: