Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55040

Problem sharing data objects between parallel build stages

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Duplicate
    • Component/s: pipeline
    • Labels:
      None
    • Environment:
      Jenkins v2.148
      Pipeline v2.6
    • Similar Issues:

      Description

      I recently found a bug which causes local data created in two closures run by a parallel build operation in Jenkins Pipeline. It seems that variables within closures somehow share memory space or something with one another, causing conflicts when the code is run in parallel. Here is some example code that should exploit the problem I hit:

       
      parallel ( "thread1": {
      node()

      { def a = git(branch: 'BranchA', credentialsId: 'MyCreds', url: 'url/to/git/repo.git') def b = "1" def c = sh(returnStdout: true, script: 'echo 1').trim() echo "git commit " + a.GIT_COMMIT echo "b is " + b echo "c is " + c }

      }, "thread2": {
      node()

      { def a = git(branch: 'BranchB', credentialsId: 'MyCreds', url: 'url/to/git/repo.git') def b = "2" def c = sh(returnStdout: true, script: 'echo 2').trim() echo "git commit " + a.GIT_COMMIT echo "b is " + b echo "c is " + c }

      }
      )
      In this example, I would expect the output to be something like

      [thread1] git commit <BranchA_Hash>
      [thread1] b is 1
      [thread1] c is 1
      [thread2] git commit <BranchB_Hash>
      [thread2] b is 2
      [thread2] c is 2

      What I actually get is this:

      [thread1] git commit <BranchA_Hash>
      [thread1] b is 1
      [thread1] c is 1
      [thread2] git commit <BranchA_Hash>
      [thread2] b is 2
      [thread2] c is 2

      While the 'b' and 'c' variables seem to get unique values for each of the parallel threads, the 'a' value becomes ambiguous, always returning a single Git commit hash.

      Further, I've confirmed that which Git commit hash that is shown represents the one that is executed "first" in the parallel stages. So, for example, if you put a small "sleep 5" at the top of thread1, then you get the Git commit has from BranchB in both echo statements, and if you move the sleep statement to the top of thread2 you get the has from BranchA.

      Also interesting is that although the 'a' variable seems to get confused based on the thread execution, the other variables do not. I thought at first that maybe assigning a static value to the variable may have explained the discrepancy like in the case of the 'b' variable so I added a dynamically executed shell step to the mix for variable 'c' and I get the same results as b... so I'm not sure how / why / what is going on here.

        Attachments

          Issue Links

            Activity

            Hide
            dnusbaum Devin Nusbaum added a comment -

            I think this is a problem with the git step not working correctly when run in parallel (i.e. both times it runs it has the same result because it cannot be run safely in parallel), not with Groovy closure variables, which is why b and c work fine. I am going to close this as a duplicate of JENKINS-39968, but please feel free to reopen if you think this is a distinct issue. See also JENKINS-54732, which more directly describes the problem you are seeing but it also looks like a dupe of JENKINS-39968.

            Show
            dnusbaum Devin Nusbaum added a comment - I think this is a problem with the git step not working correctly when run in parallel (i.e. both times it runs it has the same result because it cannot be run safely in parallel), not with Groovy closure variables, which is why b and c work fine. I am going to close this as a duplicate of JENKINS-39968 , but please feel free to reopen if you think this is a distinct issue. See also JENKINS-54732 , which more directly describes the problem you are seeing but it also looks like a dupe of JENKINS-39968 .

              People

              • Assignee:
                Unassigned
                Reporter:
                leedega Kevin Phillips
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: