Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47286

Support skipping stages in scripted pipelines for nice visualization in blue ocean and classic UI stage view

    Details

    • Similar Issues:

      Description

      Created as response to comments of https://issues.jenkins-ci.org/browse/JENKINS-37781 as the visualization of skipped stages is very, very nice for declarative pipelines in Blue Ocean.

      1. Allow to skip stages in scripted pipelines leading to equally nice visualization in Blue Ocean: The current approaches mentioned by Mike Kobit in https://issues.jenkins-ci.org/browse/JENKINS-37781?focusedCommentId=294965&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-294965 and James Dumay in https://issues.jenkins-ci.org/browse/JENKINS-37781?focusedCommentId=294966&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-294966 lead to either misleading or just less obvious visualization...
      2. Improve visualization of stage view: instead of showing skipped stages (declarative pipelines) as always being green and allegedly executed, make them e.g. gray.

        Attachments

          Issue Links

            Activity

            Hide
            jamesdumay James Dumay added a comment -

            Hi there,
            As I mentioned on JENKINS-37781, we won't be adding the skipped feature to Pipeline Script. There are some features we intend that will remain to be Pipeline and Blue Ocean only.

            However, if someone from the community wants to pick this up and work on it, please reopen and assign this to yourself.

            Thanks,
            James

            Show
            jamesdumay James Dumay added a comment - Hi there, As I mentioned on JENKINS-37781 , we won't be adding the skipped feature to Pipeline Script. There are some features we intend that will remain to be Pipeline and Blue Ocean only. However, if someone from the community wants to pick this up and work on it, please reopen and assign this to yourself. Thanks, James
            Hide
            b_dean Ben Dean added a comment -

            James Dumay are scripted pipelines deprecated or something? Why is declarative prefered? I would really like to have the ability to use a when clause on stages in scripted pipeline, for the very same reasons people use it in declarative.

            Show
            b_dean Ben Dean added a comment - James Dumay are scripted pipelines deprecated or something? Why is declarative prefered? I would really like to have the ability to use a when clause on stages in scripted pipeline, for the very same reasons people use it in declarative.
            Hide
            hamiltb Bill Hamilton added a comment -

            James Dumay I want to throw in my support to this:
            1)The ability to skip a stage is very common in our organization. A repo with many branches might want to skip various stages (deployment, testing) for certain branches. They can obviously do nothing, but again this is misleading in the pipeline view since it looks like to performed the stage (just really really fast)
            2)This support needs to be in scripted as well as declarative. You guys know that scripted is alive and well right? And sometimes, the only option....

            Show
            hamiltb Bill Hamilton added a comment - James Dumay I want to throw in my support to this: 1)The ability to skip a stage is very common in our organization. A repo with many branches might want to skip various stages (deployment, testing) for certain branches. They can obviously do nothing, but again this is misleading in the pipeline view since it looks like to performed the stage (just really really fast) 2)This support needs to be in scripted as well as declarative. You guys know that scripted is alive and well right? And sometimes, the only option....
            Hide
            foutoucour Jordi Riera added a comment -

            Hello,

            Could you detail why the reasons the `skipped` feature would be only declarative? I always struggle with choosing between using declarative and scripted, but I most of the time end up with using scripted as our pipelines are pretty complex here. 

            I would be interested to see this feature in scripted for sure.

            Show
            foutoucour Jordi Riera added a comment - Hello, Could you detail why the reasons the `skipped` feature would be only declarative? I always struggle with choosing between using declarative and scripted, but I most of the time end up with using scripted as our pipelines are pretty complex here.  I would be interested to see this feature in scripted for sure.
            Hide
            philip Philip Stroh added a comment -

            I can just agree - from my perspective the ability to skip stages and visualize that stages are skipped is a (long awaited) must have feature for both declarative and scripted pipeline! Only having this feature in declarative pipeline would fall far short of the needs.

            Show
            philip Philip Stroh added a comment - I can just agree - from my perspective the ability to skip stages and visualize that stages are skipped is a (long awaited) must have feature for both declarative and scripted pipeline! Only having this feature in declarative pipeline would fall far short of the needs.
            Hide
            pmatignon Patrice Matignon added a comment -

            I find unfortunate that the ticket be rejected wholesale.

            I think it had 2 things potentially in its scope that could be considered independently, and both of them are pretty common sense IMHO.

            The first part is adding support for skipped stages (i.e. add an expression to the stage to determine whether to skip or not), which was shutdown, sadly.

            The second part is concerned solely on rendering a skipped stage in a pipeline visualization (classic or blue ocean). This applies to declarative pipelines right now, and I can't think there would be anyone arguing against it. IF skipped stages are one day available  on scripted pipelines, the same rendering would apply as well.

            Would there be support for splitting this ticket maybe ?

            Show
            pmatignon Patrice Matignon added a comment - I find unfortunate that the ticket be rejected wholesale. I think it had 2 things potentially in its scope that could be considered independently, and both of them are pretty common sense IMHO. The first part is adding support for skipped stages (i.e. add an expression to the stage to determine whether to skip or not), which was shutdown, sadly. The second part is concerned solely on rendering a skipped stage in a pipeline visualization (classic or blue ocean). This applies to declarative pipelines right now, and I can't think there would be anyone arguing against it. IF skipped stages are one day available  on scripted pipelines, the same rendering would apply as well. Would there be support for splitting this ticket maybe ?
            Hide
            simonwydooghe Simon Wydooghe added a comment -

            I think people with scripted pipelines could benefit greatly from this. Scripted pipeline users run more advanced pipelines I'm guessing, so it feels weird to not support this feature there as it seems they will be the first ones to implement this functionality in their pipelines.

            Show
            simonwydooghe Simon Wydooghe added a comment - I think people with scripted pipelines could benefit greatly from this. Scripted pipeline users run more advanced pipelines I'm guessing, so it feels weird to not support this feature there as it seems they will be the first ones to implement this functionality in their pipelines.
            Hide
            grafi71 Grzegorz Kotarski added a comment -

            Why is the issue marked as resolved? The declarative pipeline is a subset of scripted pipeline (if I understand correctly), so there should be also possible to skip the stage.

            Show
            grafi71 Grzegorz Kotarski added a comment - Why is the issue marked as resolved? The declarative pipeline is a subset of scripted pipeline (if I understand correctly), so there should be also possible to skip the stage.
            Hide
            lucasocio Leandro Lucarella added a comment -

            +1
            Even when the issue proper here concerns me, I find it very worrying that there are features that are planned to be only available in declarative when I thought declarative was just a convenient simplification of scripted pipelines for people only needing very simple pipelines. Is obvious that when it comes to complex pipelines, or even sharing code (reuse possibilities are very crippled in declarative, as one can't put stages or stuff like agent or triggers in a library), there is no other option but scripted pipelines.
            It seems crazy (and backwards) to make scripted pipelines a second-class citizen.

            Show
            lucasocio Leandro Lucarella added a comment - +1 Even when the issue proper here concerns me, I find it very worrying that there are features that are planned to be only available in declarative when I thought declarative was just a convenient simplification of scripted pipelines for people only needing very simple pipelines. Is obvious that when it comes to complex pipelines, or even sharing code (reuse possibilities are very crippled in declarative, as one can't put stages or stuff like agent or triggers in a library), there is no other option but scripted pipelines. It seems crazy (and backwards) to make scripted pipelines a second-class citizen.
            Hide
            benkeil83 Ben Keil added a comment -
            Show
            benkeil83 Ben Keil added a comment - Take a look at  https://github.com/comquent/imperative-when
            Hide
            vizzard Aron Smyk added a comment - - edited

            Works like a charm! Great find Ben Keil, thanks!

            Since the solution was 15 lines of Shared Library I hope it will be implemented.

            Show
            vizzard Aron Smyk added a comment - - edited Works like a charm! Great find Ben Keil , thanks! Since the solution was 15 lines of Shared Library I hope it will be implemented.
            Hide
            grafi71 Grzegorz Kotarski added a comment -

            I can confirm too, it works very nice, good work!

            Show
            grafi71 Grzegorz Kotarski added a comment - I can confirm too, it works very nice, good work!
            Hide
            reinholdfuereder Reinhold Füreder added a comment - - edited

            Ben Keil Great + thanks!

            Some questions and comments WRT https://github.com/comquent/imperative-when:

            1. James Dumay org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME) is annotated with @Restricted(NoExternalUse.class) => I assume one therefore should not and cannot rely on this method being available forever. However, since this approach seems to work very well or would be at least an acceptable workaround for scripted pipelines, how are the chances that this will be get part of the official API (via removing this annotation?) or might even get used/wrapped by a new official API?
              • When trying out the other similarly named methods in org.jenkinsci.plugins.pipeline.modeldefinition.Utils (i.e. markStageFailedAndContinued, markStageSkippedForFailure, markStageSkippedForUnstable, markStageSkippedForConditional) the behaviour is rather surprising and "proofs" the aforementioned annotation => so let's just stay with org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME)
            2. James Dumay The mouse over hovering text in blue ocean shows "common.state.skipped": Since I also get this in a little declarative pipeline prototype I guess this is a minor bug (missing translation, or – maybe even better/preferable – missing to state the reason for the skipping) and not a side effect of (ab-)using this method in the wrong context?
            3. @ All: I am wondering why the body closure evaluation is coded with delegation based on config map in this example!?
            4. @ All: The STAGE_NAME is an unknown property in class section of shared library (that is not in "vars" section) but works in "Jenkinsfile" – I guess that is due to some transformation magic applied on "vars" section?
            5. @ All: Is that unfortunately leading to such SEVERE log lines:
              2018-04-06 11:04:03 SEVERE [io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeGraphVisitor <init>]   Could not find execution for run ACME-Pipeline#2347
              • Update: No that is another Jenkins pipeline or Blue Ocean bug, because it also happens for builds that do NOT use this method/approach; or that even do not skip any stages...
            Show
            reinholdfuereder Reinhold Füreder added a comment - - edited Ben Keil Great + thanks! Some questions and comments WRT https://github.com/comquent/imperative-when: James Dumay org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME) is annotated with @Restricted(NoExternalUse.class) => I assume one therefore should not and cannot rely on this method being available forever. However, since this approach seems to work very well or would be at least an acceptable workaround for scripted pipelines, how are the chances that this will be get part of the official API (via removing this annotation?) or might even get used/wrapped by a new official API? When trying out the other similarly named methods in org.jenkinsci.plugins.pipeline.modeldefinition.Utils (i.e. markStageFailedAndContinued , markStageSkippedForFailure , markStageSkippedForUnstable , markStageSkippedForConditional ) the behaviour is rather surprising and "proofs" the aforementioned annotation => so let's just stay with org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME) James Dumay The mouse over hovering text in blue ocean shows "common.state.skipped": Since I also get this in a little declarative pipeline prototype I guess this is a minor bug (missing translation, or – maybe even better/preferable – missing to state the reason for the skipping) and not a side effect of (ab-)using this method in the wrong context ? @ All: I am wondering why the body closure evaluation is coded with delegation based on config map in this example!? @ All: The STAGE_NAME is an unknown property in class section of shared library (that is not in "vars" section) but works in "Jenkinsfile" – I guess that is due to some transformation magic applied on "vars" section? @ All: Is that unfortunately leading to such SEVERE log lines: 2018-04-06 11:04:03 SEVERE [io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeGraphVisitor <init>]   Could not find execution for run ACME-Pipeline#2347 Update: No that is another Jenkins pipeline or Blue Ocean bug, because it also happens for builds that do NOT use this method/approach; or that even do not skip any stages...
            Hide
            chrismaes Chris Maes added a comment -

            This issue is not solved. The skipping of stages in blue ocean view is well supported in Pipeline syntax, but not in scripted pipelines.

            The workaround proposed in https://github.com/comquent/imperative-when works correctly, but since I am using https://github.com/jenkinsci/JenkinsPipelineUnit it seems quite impossible for me to get my unit-tests working.

            Show
            chrismaes Chris Maes added a comment - This issue is not solved. The skipping of stages in blue ocean view is well supported in Pipeline syntax, but not in scripted pipelines. The workaround proposed in https://github.com/comquent/imperative-when  works correctly, but since I am using https://github.com/jenkinsci/JenkinsPipelineUnit  it seems quite impossible for me to get my unit-tests working.
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            This morning I was positively surprised that according to BlueOcean change log (https://plugins.jenkins.io/blueocean) this issue was solved:

            Change log
            1.11.0 (Feb 13, 2019)

            ...
            JENKINS-47286 Support skipping stages in scripted pipelines for nice visualization in blue ocean and classic UI stage view
            ...

            ... "only" to find out that it is still in status reopened, i.e. unresolved and that "only" one aspect was solved: namely the missing translation (https://github.com/jenkinsci/blueocean-plugin/commit/b0bf45c64afbf511971c7e79ff2dd317f1bddfc8) – of course I am happy/grateful about that too.

            However, this is a bit misleading

            Show
            reinholdfuereder Reinhold Füreder added a comment - This morning I was positively surprised that according to BlueOcean change log ( https://plugins.jenkins.io/blueocean) this issue was solved: Change log 1.11.0 (Feb 13, 2019) ... JENKINS-47286 Support skipping stages in scripted pipelines for nice visualization in blue ocean and classic UI stage view ... ... "only" to find out that it is still in status reopened, i.e. unresolved and that "only" one aspect was solved: namely the missing translation ( https://github.com/jenkinsci/blueocean-plugin/commit/b0bf45c64afbf511971c7e79ff2dd317f1bddfc8 ) – of course I am happy/grateful about that too. However, this is a bit misleading
            Hide
            ctaggart Cameron Taggart added a comment - - edited

            How do we use the workaround at https://github.com/comquent/imperative-when ? It does not appear to be a plugin that we can load. I'm a bit confused on how to use it. Any tips are appreciated. It would be great if this came out of the box. I'm migrating a declarative Jenkinsfile and got stuck on this.

            Show
            ctaggart Cameron Taggart added a comment - - edited How do we use the workaround at https://github.com/comquent/imperative-when ? It does not appear to be a plugin that we can load. I'm a bit confused on how to use it. Any tips are appreciated. It would be great if this came out of the box. I'm migrating a declarative Jenkinsfile and got stuck on this.
            Hide
            ctaggart Cameron Taggart added a comment -

            Adding it as a Global Shared Library is documented here. https://jenkins.io/doc/book/pipeline/shared-libraries/#global-shared-libraries
            It is working for me.

            Show
            ctaggart Cameron Taggart added a comment - Adding it as a Global Shared Library is documented here. https://jenkins.io/doc/book/pipeline/shared-libraries/#global-shared-libraries It is working for me.
            Hide
            mihaid Mihai Dinu added a comment - - edited

            There's a nice post here: **https://comquent.de/en/skipped-stages-in-jenkins-scripted-pipeline/ that describes how to use the https://github.com/comquent/imperative-when workaround. 

            I have a related question though: is it possible to show the stages that follow a failed stage as skipped (or failed)? My use case is something like this

            node {
                try {
                    stage('Will fail') {
                        sh 'exit 1'
                    }
                    stage('Some operations') {
                        echo 'This stage should run when previous stage is successful'
                        echo 'Should be skipped or failed when previous stage fails'
                    }
                } 
                catch (err) {
                    echo "Print the error from the first stage $err"
                }
                finally {
                    stage('Post build') {
                        echo 'Always do some cleanup ops'
                    }
                }
            }
            
            
            

             The problem that I'm facing is that any stage from my pipeline can fail and I want all my downstream stages to be skipped, except the post build one. When a stage fails, it is caught in the catch block and so all downstream stages are ignored, ruining the pipeline stage view from the job UI. 

            Any possible solutions for this?

            Show
            mihaid Mihai Dinu added a comment - - edited There's a nice post here: ** https://comquent.de/en/skipped-stages-in-jenkins-scripted-pipeline/  that describes how to use the https://github.com/comquent/imperative-when  workaround.  I have a related question though: is it possible to show the stages that follow a failed stage as skipped (or failed)? My use case is something like this node { try { stage( 'Will fail' ) { sh 'exit 1' } stage( 'Some operations' ) { echo 'This stage should run when previous stage is successful' echo 'Should be skipped or failed when previous stage fails' } } catch (err) { echo "Print the error from the first stage $err" } finally { stage( 'Post build' ) { echo 'Always do some cleanup ops' } } }  The problem that I'm facing is that any stage from my pipeline can fail and I want all my downstream stages to be skipped, except the post build one. When a stage fails, it is caught in the catch block and so all downstream stages are ignored, ruining the pipeline stage view from the job UI.  Any possible solutions for this?
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            While I am not sure if this is the correct/perfect/recommended place for this question, it admittedly fits to this issue and I can actually give you a little hint

            My approach for this is to have a skipableStage step in my shared Jenkins pipeline library that wraps the default/standard stage step, so the usage is almost like normally:

            ...
            acmeSharedLibrary.skipableStage("stage name") {
               ... // normal stage implementation
            }
            ...
            

            ... and the steps implementation in pseudo code is:

            stage(stageName) {
              if (currentBuild.result) {
                echo "Skipping stage '${stageName}' due to build result '${currentBuild.result}'..."
                Result result = Result.fromString(currentBuild.result)
                markStageSkippedForResult(stageName, result) // via the aformentioned workaround
              } else {
                body() // which is the closure block of this "skippableStage"
              }
            }
            
            Show
            reinholdfuereder Reinhold Füreder added a comment - While I am not sure if this is the correct/perfect/recommended place for this question, it admittedly fits to this issue and I can actually give you a little hint My approach for this is to have a skipableStage step in my shared Jenkins pipeline library that wraps the default/standard stage step, so the usage is almost like normally: ... acmeSharedLibrary.skipableStage( "stage name" ) { ... // normal stage implementation } ... ... and the steps implementation in pseudo code is: stage(stageName) { if (currentBuild.result) { echo "Skipping stage '${stageName}' due to build result '${currentBuild.result}' ..." Result result = Result.fromString(currentBuild.result) markStageSkippedForResult(stageName, result) // via the aformentioned workaround } else { body() // which is the closure block of this "skippableStage" } }
            Hide
            michaelneale Michael Neale added a comment -

            (can't leave this unassigned for some reason). 

            Show
            michaelneale Michael Neale added a comment - (can't leave this unassigned for some reason). 
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            Michael Neale I am a bit surprised by being the new assignee of this issue, as I am not sure what you are expecting from me. Considering the admittedly rather long history of this issue and its distractions/deviations I think I can only try to do a little update:

            Show
            reinholdfuereder Reinhold Füreder added a comment - Michael Neale I am a bit surprised by being the new assignee of this issue, as I am not sure what you are expecting from me. Considering the admittedly rather long history of this issue and its distractions/deviations I think I can only try to do a little update: the issue's title/goal is still completely the same and correct and IMHO the "only" (or main) thing certainly missing is a stable/public API for org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME) (as this is annotated with @Restricted(NoExternalUse.class) ) In the course of doing that other questions from https://issues.jenkins-ci.org/browse/JENKINS-47286?focusedCommentId=334032&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-334032 might be answered as well... (I think item #2 "mouse hover" was actually fixed; see https://issues.jenkins-ci.org/browse/JENKINS-47286?focusedCommentId=360479&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-360479 )
            Hide
            reinholdfuereder Reinhold Füreder added a comment - - edited

            Assigning this back to Michael Neale with the update/summary comment above...

            Show
            reinholdfuereder Reinhold Füreder added a comment - - edited Assigning this back to Michael Neale with the update/summary comment above...
            Hide
            michaelneale Michael Neale added a comment -

            can't seem to unassign it. 

             

            Show
            michaelneale Michael Neale added a comment - can't seem to unassign it.   

              People

              • Assignee:
                michaelneale Michael Neale
                Reporter:
                reinholdfuereder Reinhold Füreder
              • Votes:
                10 Vote for this issue
                Watchers:
                40 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: