Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43336

Allow locking multiple stages in declarative pipeline

    XMLWordPrintable

    Details

    • Sprint:
      Declarative - 1.2
    • Similar Issues:

      Description

      It would be useful to be able to lock multiple stages as a single lock. For example, we usually have a stage to deploy to an environment and then another stage to run end-to-end tests on that environment, but there should be no other concurrent deployments until both stages have completed.

      Something like this:

      pipeline {
        stages {
          lock(resource: 'myResource', inversePrecedence: true) {
            stage('Deploy') {
              // deploy to environment
            }
      
            stage('E2E') {
              // run tests on the environment
              milestone 1
            }
          }
        }
      }

      Technically both stages could just be merged into a single stage but to me that defeats the purpose of stages.

        Attachments

          Issue Links

            Activity

            Hide
            abayer Andrew Bayer added a comment -

            The combination of JENKINS-46809 and JENKINS-48380 will let you define a list of stages to run within a single top level (or parallel) stage, which will let you get what's desired here.

            Show
            abayer Andrew Bayer added a comment - The combination of JENKINS-46809 and JENKINS-48380 will let you define a list of stages to run within a single top level (or parallel) stage, which will let you get what's desired here.
            Hide
            wortex17 Worry Darque added a comment - - edited

            Andrew Bayer I'm not so sure about this, as this only works if we would be allowed to wrap the substages inside an "lock" declaration, which, as of now, would be impossible as lock is not a valid direct child of a stage (has to be nested inside steps).

            I tried

             

            stage('Build Editor') {
                lock("myLock") {
                    steps {
                        //...
                    }
                }
            }
            

            ,

            stage('Build Editor') {
                lock("myLock") {
                    parallel {
                        stage("ReallyBuild") {
                            steps {
                                //...
                            }
                        }
                    }
                }
            }
            

             as well as 

            stage('Build Editor') {
                parallel {
                    lock("myLock") {
                        stage("ReallyBuild") {
                            steps {
                                //...
                            }
                        }
                    }
                }
            }
            

             to no avail, as it fails with

            No "steps" or "parallel" to execute within stage

             or

            No stages specified ...

            Show
            wortex17 Worry Darque added a comment - - edited Andrew Bayer I'm not so sure about this, as this only works if we would be allowed to wrap the substages inside an "lock" declaration, which, as of now, would be impossible as lock is not a valid direct child of a stage (has to be nested inside steps). I tried   stage( 'Build Editor' ) { lock( "myLock" ) { steps { //... } } } , stage( 'Build Editor' ) { lock( "myLock" ) { parallel { stage( "ReallyBuild" ) { steps { //... } } } } }  as well as  stage( 'Build Editor' ) { parallel { lock( "myLock" ) { stage( "ReallyBuild" ) { steps { //... } } } } }  to no avail, as it fails with No "steps" or "parallel" to execute within stage  or No stages specified ...
            Hide
            abayer Andrew Bayer added a comment -

            JENKINS-46809 will give you what you need on top of what's already available from JENKINS-48380, so that you could do something like:

            stage('Parent') {
              options {
                lock('myLock')
              }
              stages {
                stage('first child') {
                  ...
                }
                stage('second child') {
                  ...
                }
              }
            }
            
            Show
            abayer Andrew Bayer added a comment - JENKINS-46809 will give you what you need on top of what's already available from JENKINS-48380 , so that you could do something like: stage( 'Parent' ) { options { lock( 'myLock' ) } stages { stage( 'first child' ) { ... } stage( 'second child' ) { ... } } }
            Hide
            rrueth Ryan Rueth added a comment -

            Andrew Bayer, I'm trying to lock multiple stages based on the Jenkins node on which the stages will execute. It seems like the only way to do this is using the `options` block that you specified:

            stage('Parent') {
              options {
                lock("${env.NODE_NAME}")
              }
              stages {
                stage('first child') {
                  ...
                }
                stage('second child') {
                  ...
                }
              }
            }

            Unfortunately, the resource this attempts to lock is `null`:

            Trying to acquire lock on [null]

             

            It appears that this won't work because the stage's `options` directive occurs before entering the agent. The declarative pipeline syntax  documentation says: 

            Inside a stage, the steps in the options directive are invoked before entering the agent or checking any when conditions.

             
            Is there any way to lock multiple stages for a single env.NODE_NAME?  Ideally, other subsequent stages could still run in parallel with other builds (i.e. so limiting executors to 1 or disabling concurrent builds are not options).
             

            Show
            rrueth Ryan Rueth added a comment - Andrew Bayer , I'm trying to lock multiple stages based on the Jenkins node on which the stages will execute. It seems like the only way to do this is using the `options` block that you specified: stage( 'Parent' ) { options { lock( "${env.NODE_NAME}" ) } stages { stage( 'first child' ) { ... } stage( 'second child' ) { ... } } } Unfortunately, the resource this attempts to lock is `null`: Trying to acquire lock on [null]   It appears that this won't work because the stage's `options` directive occurs before entering the agent. The declarative pipeline syntax  documentation says:  Inside a  stage , the steps in the  options  directive are invoked before entering the  agent  or checking any  when  conditions.   Is there any way to lock multiple stages for a single env.NODE_NAME?  Ideally, other subsequent stages could still run in parallel with other builds (i.e. so limiting executors to 1 or disabling concurrent builds are not options).  
            Hide
            nikolodus Dusan Nikolov added a comment -

            Just wanted to +1 on Ryan Rueth's comment regarding locking multiple stages with lock named after $NODE_NAME. Pretty much in the same situation, and I'd like to know if this can be supported - either by enabling options block to execute after agent has been assigned for a given stage, or through some other means if this is not an option.

            Andrew Bayer do you know if this could be possible?

            Show
            nikolodus Dusan Nikolov added a comment - Just wanted to +1 on Ryan Rueth 's comment regarding locking multiple stages with lock named after $NODE_NAME. Pretty much in the same situation, and I'd like to know if this can be supported - either by enabling options block to execute after agent has been assigned for a given stage, or through some other means if this is not an option. Andrew Bayer do you know if this could be possible?

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                rochdev Roch Devost
              • Votes:
                19 Vote for this issue
                Watchers:
                34 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: