Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60801

Variable changes are shared between matrix (parallel) stages.

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Blocker
    • Resolution: Unresolved
    • Component/s: pipeline
    • Labels:
    • Environment:
      Jenkins ver. 2.190.2
      Pipeline: API 2.83
      Pipeline: Basic Steps 2.19
      Pipeline: Declarative 1.5.0
    • Similar Issues:

      Description

      New matrix step provides ability to run the same code in parallel cells. But if that code assigns ANY variable (not just env), it gets overwritten later by the next cell. This is huge deal, it basically prevents from assigning any variable in this kind of pipeline which makes it nearly useless.

      The condition is to set variable, have a piece of code that takes some time to execute and use the already set variable after. This can be simulated with sleep().

      I boiled down my code to easily reproduce it, here is the pipeline:

      pipeline {
          parameters {
              booleanParam(name: 'RHEL5', defaultValue: true)
              booleanParam(name: 'RHEL6', defaultValue: true)
              booleanParam(name: 'RHEL7', defaultValue: true)
          }
          stages {
              stage('Matrix') {
                  matrix {
                      when {
                          anyOf {
                              expression { params.RHEL5 && env.RHEL == '5' }
                              expression { params.RHEL6 && env.RHEL == '6' }
                              expression { params.RHEL7 && env.RHEL == '7' }
                          }
                      }
                      axes {
                          axis {
                              name 'RHEL'
                              values '5', '6', '7'
                          }
                      }
                      stages {
                          stage('Build single'){
                              steps {
                                  script {
                                      echo "RHEL version: " + env.RHEL
                                      myVar = "something.something.RHEL-${env.RHEL}"
                                      echo "myVar: " + myVar
                                      sleep(10)
                                      echo "AFTER RHEL version: " + env.RHEL 
                                      echo "AFTER myVar: " + myVar
                                  }
                              }
                          }
                      }
                  }
              }
          }
      }
      

      The result of this code:

       

      RHEL version: 5
      myVar: something.something.RHEL-5
      Sleeping for 10 sec
      RHEL version: 6
      myVar: something.something.RHEL-6
      Sleeping for 10 sec
      RHEL version: 7
      myVar: something.something.RHEL-7
      Sleeping for 10 sec
      No need to sleep any longer
      RHEL version: 5
      myVar: something.something.RHEL-7
      No need to sleep any longer
      RHEL version: 6
      myVar: something.something.RHEL-7
      No need to sleep any longer
      RHEL version: 7
      myVar: something.something.RHEL-7

      As you can see, myVar gets overwritten by the last run cell even for the other cells.

       

       

        Attachments

          Activity

          Hide
          kryslj Jakub Krysl added a comment - - edited

          Here is a script boiled down to simplest reproducer for both Matrix and Parallel pipeline:

          def getTasks() {
              def tasks = [:]
              //['5', '6', '7'].each { RHEL ->
              versions = ['5', '6', '7']
              for (int i=0; i<versions.size(); i++) {
                  def RHEL = versions[i]
                  tasks["${RHEL}"] = {
                      node('slave') {
                          echo "in PARALLEL"
                          TEST = "in_variable_" + RHEL
                          echo "RHEL: " + RHEL + ", TEST: " + TEST
                          sleep(5)
                          echo "RHEL: " + RHEL + ", TEST: " + TEST
                      }
                  }
              }
              return tasks
          }
          
          pipeline {
              agent { label 'slave' }
              stages {
                  stage('Build Matrix') {
                      matrix {
                          axes {
                              axis {
                                  name 'RHEL'
                                  values '5', '6', '7'
                              }
                          }
                          stages('RHEL') {
                              stage('Build single'){
                                  steps {
                                      script {
                                          echo "in MATRIX"
                                          TEST = "in_variable_" + RHEL
                                          echo "RHEL: " + RHEL + ", TEST: " + TEST
                                          sleep(5)
                                          echo "RHEL: " + RHEL + ", TEST: " + TEST
                                      }
                                  }
                              }
                          }
                      }
                  }
                  stage('Parallel build') {
                      steps {
                          script {
                              parallel getTasks()
                          }
                      }
                  }
              }
          }
          
          

          Output:

          in MATRIX
          RHEL: 5, TEST: in_variable_5
          Sleeping for 5 sec
          in MATRIX
          RHEL: 6, TEST: in_variable_6
          Sleeping for 5 sec
          in MATRIX
          RHEL: 7, TEST: in_variable_7
          Sleeping for 5 sec
          RHEL: 5, TEST: in_variable_7
          RHEL: 6, TEST: in_variable_7
          RHEL: 7, TEST: in_variable_7
          
          Running on jenkins-slave2 in /home/jenkins/workspace/TEST_PIPELINE
          Running on jenkins-slave2 in /home/jenkins/workspace/TEST_PIPELINE@2
          Running on jenkins-slave1 in /home/jenkins/workspace/TEST_PIPELINE@2
          in PARALLEL
          RHEL: 5, TEST: in_variable_5
          Sleeping for 5 sec
          in PARALLEL
          RHEL: 6, TEST: in_variable_6
          Sleeping for 5 sec
          in PARALLEL
          RHEL: 7, TEST: in_variable_7
          Sleeping for 5 sec
          RHEL: 5, TEST: in_variable_7
          RHEL: 6, TEST: in_variable_7
          RHEL: 7, TEST: in_variable_7
          Finished: SUCCESS
          

          Setting this issue to blocker as it prevents me from using any cell/branch specific variables.

          Show
          kryslj Jakub Krysl added a comment - - edited Here is a script boiled down to simplest reproducer for both Matrix and Parallel pipeline: def getTasks() { def tasks = [:] //[ '5' , '6' , '7' ].each { RHEL -> versions = [ '5' , '6' , '7' ] for ( int i=0; i<versions.size(); i++) { def RHEL = versions[i] tasks[ "${RHEL}" ] = { node( 'slave' ) { echo "in PARALLEL" TEST = "in_variable_" + RHEL echo "RHEL: " + RHEL + ", TEST: " + TEST sleep(5) echo "RHEL: " + RHEL + ", TEST: " + TEST } } } return tasks } pipeline { agent { label 'slave' } stages { stage( 'Build Matrix' ) { matrix { axes { axis { name 'RHEL' values '5' , '6' , '7' } } stages( 'RHEL' ) { stage( 'Build single' ){ steps { script { echo "in MATRIX" TEST = "in_variable_" + RHEL echo "RHEL: " + RHEL + ", TEST: " + TEST sleep(5) echo "RHEL: " + RHEL + ", TEST: " + TEST } } } } } } stage( 'Parallel build' ) { steps { script { parallel getTasks() } } } } } Output: in MATRIX RHEL: 5, TEST: in_variable_5 Sleeping for 5 sec in MATRIX RHEL: 6, TEST: in_variable_6 Sleeping for 5 sec in MATRIX RHEL: 7, TEST: in_variable_7 Sleeping for 5 sec RHEL: 5, TEST: in_variable_7 RHEL: 6, TEST: in_variable_7 RHEL: 7, TEST: in_variable_7 Running on jenkins-slave2 in /home/jenkins/workspace/TEST_PIPELINE Running on jenkins-slave2 in /home/jenkins/workspace/TEST_PIPELINE@2 Running on jenkins-slave1 in /home/jenkins/workspace/TEST_PIPELINE@2 in PARALLEL RHEL: 5, TEST: in_variable_5 Sleeping for 5 sec in PARALLEL RHEL: 6, TEST: in_variable_6 Sleeping for 5 sec in PARALLEL RHEL: 7, TEST: in_variable_7 Sleeping for 5 sec RHEL: 5, TEST: in_variable_7 RHEL: 6, TEST: in_variable_7 RHEL: 7, TEST: in_variable_7 Finished: SUCCESS Setting this issue to blocker as it prevents me from using any cell/branch specific variables.
          Hide
          cwoodcox Corey Woodcox added a comment -

          We also see this in a similar build script. We have a job that loops through a list of versions and fires off several docker image builds in parallel and at the end of the script the image name variables have overwritten each other.

          I can try and grab more information if you think it would help troubleshoot.

          Show
          cwoodcox Corey Woodcox added a comment - We also see this in a similar build script. We have a job that loops through a list of versions and fires off several docker image builds in parallel and at the end of the script the image name variables have overwritten each other. I can try and grab more information if you think it would help troubleshoot.

            People

            • Assignee:
              Unassigned
              Reporter:
              kryslj Jakub Krysl
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: