Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55438

Allow sequential stages inside parallel in Scripted syntax

    Details

    • Similar Issues:

      Description

      Similar to JENKINS-46809, I would like the Blue Ocean GUI to properly visualize multiple stages in sequence that are all parallel of each other. Below is an example image of what this looks like using declarative syntax (working):

       

      Here is the declarative example code I wrote to generate the example image:

      pipeline {
        agent { label 'master' }
        stages {
          stage('Build and Test') {
            parallel {
              stage("Build and Test Linux") {
                stages {
                  stage("Build (Linux)") {
                    agent any
                    steps {
                      echo "Inside for loop 1"
                    }
                  }
                  stage("Test (Linux)") {
                    agent any
                    steps {
                      echo "Inside for loop 2"
                    }
                  }
                }
              }
              stage("Build and Test Windows") {
                stages {
                  stage("Build (Windows)") {
                    agent any
                    steps {
                      echo "Inside for loop 3"
                    }
                  }
                  stage("Test (Windows)") {
                    agent any
                    steps {
                      echo "Inside for loop 4"
                    }
                  }
                }
              }
            }
          }
        }
      }

       

      Here is example scripted Jenkins code that I would like to use. Linux and windows build/test flows happen in parallel. Inside each flow in a "build" stage and then a "test" stage. The Windows sequential build/test flow should be displayed in parallel with the Linux build/test flow, but right now the separate Build/Test sequential stages are combined into one circle/stage when using the scripted syntax.

       

      pipeline {
        agent any
        stages {
          stage("Build and Test") {
            steps {
              script {
                parallel linux: {
                  node("linux_build") {
                    echo "Inside for loop 1"
                  }
                  node("linux_test") {
                    echo "Inside for loop 2"
                  }
                },
                windows: {
                  node("windows_build") {
                    echo "Inside for loop 3"
                  }
                  node("windows_test") {
                    echo "Inside for loop 4"
                  }
                }
              }
            }
          }
        }
      }
      

       

      This is what the scripted example code currently generates:

       

        Attachments

          Issue Links

            Activity

            acarr468 Adam Carroll created issue -
            abayer Andrew Bayer made changes -
            Field Original Value New Value
            Component/s blueocean-plugin [ 21481 ]
            Component/s pipeline [ 21692 ]
            Component/s pipeline-model-definition-plugin [ 21706 ]
            abayer Andrew Bayer made changes -
            Assignee Andrew Bayer [ abayer ]
            Hide
            jglick Jesse Glick added a comment -

            First of all, the second example is not Scripted syntax; it is pseudo-Declarative (pipeline) with a script block.

            Second, node is not a stage. Try using the actual stage step.

            If Blue Ocean in fact refuses to display a valid Scripted build with an identical structure of parallel / stage steps that is displayed when run from a Declarative build, then I would consider that a bug rather an RFE.

            Show
            jglick Jesse Glick added a comment - First of all, the second example is not Scripted syntax; it is pseudo-Declarative ( pipeline ) with a script block. Second, node is not a stage . Try using the actual stage step. If Blue Ocean in fact refuses to display a valid Scripted build with an identical structure of parallel / stage steps that is displayed when run from a Declarative build, then I would consider that a bug rather an RFE.
            peacemoon An Tran made changes -
            Hide
            peacemoon An Tran added a comment - - edited

            I can confirm that I have the same behaviour like that of the author. Only one stage of scripted pipeline in  parallel branches is shown

            pipeline {
              stages {
                stage('testing') {
                  steps {
                    script {
                      parallel "task1": {
                          node("ios") {
                            stage("build") {
                                echo "windows-build"
                            }
                            stage("deploy") {
                                echo "windows-deploy"
                            }
                          }
                      },
                      "task2": {
                        node("ios") {
                          stage("build") {
                              echo "windows-build"
                          }
                          stage("deploy") {
                              echo "windows-deploy"
                          }
                        }
                      }
                    }
                  }
                }
              }
            }

             

            Show
            peacemoon An Tran added a comment - - edited I can confirm that I have the same behaviour like that of the author. Only one stage of scripted pipeline in  parallel branches is shown pipeline { stages { stage( 'testing' ) { steps { script { parallel "task1" : { node( "ios" ) { stage( "build" ) { echo "windows-build" } stage( "deploy" ) { echo "windows-deploy" } } }, "task2" : { node( "ios" ) { stage( "build" ) { echo "windows-build" } stage( "deploy" ) { echo "windows-deploy" } } } } } } } }  
            Hide
            jglick Jesse Glick added a comment -

            An Tran again that is not Scripted syntax. It is a script block inside Declarative syntax.

            Show
            jglick Jesse Glick added a comment - An Tran again that is not Scripted syntax. It is a script block inside Declarative syntax.
            Hide
            peacemoon An Tran added a comment -

            Jesse Glick Ok, so what is the correct way to do it?

            Show
            peacemoon An Tran added a comment - Jesse Glick Ok, so what is the correct way to do it?
            Hide
            jglick Jesse Glick added a comment -

            I do not know, you would need to experiment.

            Show
            jglick Jesse Glick added a comment - I do not know, you would need to experiment.
            Hide
            peacemoon An Tran added a comment -

            I did experiment a lot already before asking here

            Ok let's keep the terminology aside.

            Can you help me to understand why only one stage is shown in each parallel branch even though I habe 2 stages for each branch in my script? Is it a bug? Should I create a bug report for that?

            Show
            peacemoon An Tran added a comment - I did experiment a lot already before asking here Ok let's keep the terminology aside. Can you help me to understand why only one stage is shown in each parallel branch even though I habe 2 stages for each branch in my script? Is it a bug? Should I create a bug report for that?
            Hide
            jglick Jesse Glick added a comment -

            I do not know how Blue Ocean works in any detail. My comment was very much about the terminology. None of the examples given so far in this issue are using actual Scripted syntax—they all start with pipeline, which is Declarative syntax, and so all bets are off.

            At any rate, the closest Scripted analogue I can come up with for the original Declarative script would be

            parallel 'Build and Test Linux': {
                stage("Build (Linux)") {
                    echo "Inside for loop 1"
                }
                stage("Test (Linux)") {
                    echo "Inside for loop 2"
                }
            }, 'Build and Test Windows': {
                stage("Build (Windows)") {
                    echo "Inside for loop 3"
                }
                stage("Test (Windows)") {
                    echo "Inside for loop 4"
                }
            }
            

            which indeed also fails to display the stages—only the branches are shown. (Similarly when the whole script is enclosed in stage('Build and Test') {…}.)So I would consider that a bug in Blue Ocean: it should display a Pipeline build with a structure that its visualization layer supports, regardless of the syntax used to run that build. Perhaps there is some workaround that a B.O. developer could explain.

            Show
            jglick Jesse Glick added a comment - I do not know how Blue Ocean works in any detail. My comment was very much about the terminology. None of the examples given so far in this issue are using actual Scripted syntax—they all start with pipeline , which is Declarative syntax, and so all bets are off. At any rate, the closest Scripted analogue I can come up with for the original Declarative script would be parallel 'Build and Test Linux' : { stage( "Build (Linux)" ) { echo "Inside for loop 1" } stage( "Test (Linux)" ) { echo "Inside for loop 2" } }, 'Build and Test Windows' : { stage( "Build (Windows)" ) { echo "Inside for loop 3" } stage( "Test (Windows)" ) { echo "Inside for loop 4" } } which indeed also fails to display the stages—only the branches are shown. (Similarly when the whole script is enclosed in stage('Build and Test') {… }.)So I would consider that a bug in Blue Ocean: it should display a Pipeline build with a structure that its visualization layer supports, regardless of the syntax used to run that build. Perhaps there is some workaround that a B.O. developer could explain.
            jglick Jesse Glick made changes -
            Issue Type Improvement [ 4 ] Bug [ 1 ]
            Hide
            peacemoon An Tran added a comment -

            Jesse Glick From what I see in last comment int this ticket here https://issues.jenkins-ci.org/browse/JENKINS-53048?focusedCommentId=351304&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-351304 , Adam Carroll really wants to have an improvement instead of a bug report because the UI really on supports only declarative syntax of parallel command. If there is a bug, I would open a new one and keep this as improvement.

            Show
            peacemoon An Tran added a comment - Jesse Glick From what I see in last comment int this ticket here https://issues.jenkins-ci.org/browse/JENKINS-53048?focusedCommentId=351304&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-351304  , Adam Carroll really wants to have an improvement instead of a bug report because the UI really on supports only declarative syntax of parallel command. If there is a bug, I would open a new one and keep this as improvement.
            Hide
            jglick Jesse Glick added a comment -

            Contra Olivier Lamy’s comment there I consider this a bug. Blue Ocean should pay attention to the build metadata as provided by official Pipeline APIs and may not make assumptions about the mechanism used to run a build, except insofar as that is intrinsically required for a certain feature. (For example, a stage restart button should only appear next to a Declarative stage, because the feature of restarting a stage is tied to Declarative semantics.)

            Regardless of how the issue is classified, the question of whether it will be fixed or not depends entirely on the development team’s time and priorities.

            Show
            jglick Jesse Glick added a comment - Contra Olivier Lamy ’s comment there I consider this a bug. Blue Ocean should pay attention to the build metadata as provided by official Pipeline APIs and may not make assumptions about the mechanism used to run a build, except insofar as that is intrinsically required for a certain feature. (For example, a stage restart button should only appear next to a Declarative stage, because the feature of restarting a stage is tied to Declarative semantics.) Regardless of how the issue is classified, the question of whether it will be fixed or not depends entirely on the development team’s time and priorities.
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-53048 [ JENKINS-53048 ]
            Hide
            peacemoon An Tran added a comment -
            Show
            peacemoon An Tran added a comment - I found some tickets that maybe related: https://issues.jenkins-ci.org/browse/JENKINS-53751 https://issues.jenkins-ci.org/browse/JENKINS-53162  
            Hide
            acarr468 Adam Carroll added a comment -

            Thanks Jesse Glick and An Tran for digging into this more! I personally am not concerned whether or not this is a bug or improvement, as long as it is fixed . It does sound a lot like a bug to me, but I do understand the argument for it being an improvement. Whatever the best path to resolution is will be fine with me.

            Based on the tickets An Tran linked above, it looks like others desire this functionality as well.

            Show
            acarr468 Adam Carroll added a comment - Thanks Jesse Glick and An Tran for digging into this more! I personally am not concerned whether or not this is a bug or improvement, as long as it is fixed . It does sound a lot like a bug to me, but I do understand the argument for it being an improvement. Whatever the best path to resolution is will be fine with me. Based on the tickets An Tran linked above, it looks like others desire this functionality as well.
            mahima Mahima Mishra made changes -
            Attachment Screenshot from 2019-11-01 13-05-19.png [ 49371 ]
            mahima Mahima Mishra made changes -
            Attachment Screenshot from 2019-11-01 13-05-19.png [ 49371 ]
            Hide
            mahima Mahima Mishra added a comment - - edited

            I am also stuck in the same situation, where I need to use scripted parallel block within declarative and the blueocean view is doesn't display the stages for scripted block correctly.

            Show
            mahima Mahima Mishra added a comment - - edited I am also stuck in the same situation, where I need to use scripted parallel block within declarative and the blueocean view is doesn't display the stages for scripted block correctly.
            mahima Mahima Mishra made changes -
            zionyx KY Lee made changes -
            Link This issue relates to JENKINS-53751 [ JENKINS-53751 ]
            zionyx KY Lee made changes -
            Link This issue relates to JENKINS-53751 [ JENKINS-53751 ]
            zionyx KY Lee made changes -
            Link This issue is related to JENKINS-53751 [ JENKINS-53751 ]
            zionyx KY Lee made changes -
            Link This issue relates to JENKINS-53162 [ JENKINS-53162 ]
            betoz Alberto Arango made changes -
            betoz Alberto Arango made changes -
            Attachment mixed-declarative-and-scripted.PNG [ 52217 ]
            Hide
            betoz Alberto Arango added a comment -

            Here I share three code examples (bottom of this comment)

            1. declarative parallel within declarative pipeline
            2. scripted parallel within scripted pipeline
            3. scripted parallel within declarative pipeline

            The first two get visualized as expected.

            The third one gets a messed up visualization starting at the parallel part, i.e. also affects the next non-parallel stage.

            The code

            • declarative parallel within declarative pipeline
              pipeline {
                  agent("none")
                  stages {
                      stage("prepare") {steps {echo "prepare"}}
                      stage("build") {steps {echo "build"}}
                      stage("test") {
                          parallel {
                              stage("linux") {
                                  stages {
                                      stage("install") {steps {echo "installing in linux"}}
                                      stage("test") {steps {echo "testing in linux"}}
                                  }
                              }
                              stage("windows") {
                                  stages {
                                      stage("install") {steps {echo "installing in windows"}}
                                      stage("test") {steps {echo "testing in windows"}}
                                  }
                              }
                              stage("mac") {
                                  stages {
                                      stage("install") {steps {echo "installing in mac"}}
                                      stage("test") {steps {echo "testing in mac"}}
                                  }
                              }
                          }
                      }
                      stage("cleanup") {steps {echo "cleanup"}}
                  }
              }
              
            • scripted parallel within scripted pipeline
              stage("prepare") {echo "prepare"}
              stage("build") {echo "build"}
              stage("test") {
                  parallel ([
                      "linux": {
                          stage("install") {echo "installing in linux"}
                          stage("test") {echo "testing in linux"}
                      },
                      "windows": {
                          stage("install") {echo "installing in windows"}
                          stage("test") {echo "testing in windows"}
                      },
                      "mac": {
                          stage("install") {echo "installing in mac"}
                          stage("test") {echo "testing in mac"}
                      }
                  ])
              }
              stage("cleanup") {echo "cleanup"}
              
            • scripted parallel within declarative pipeline
              pipeline {
                  agent("none")
                  stages {
                      stage("prepare") {steps {echo "prepare"}}
                      stage("build") {steps {echo "build"}}
                      stage("test") {
                          steps {
                              script {
                                  parallel ([
                                      "linux": {
                                          stage("install") {echo "installing in linux"}
                                          stage("test") {echo "testing in linux"}
                                      },
                                      "windows": {
                                          stage("install") {echo "installing in windows"}
                                          stage("test") {echo "testing in windows"}
                                      },
                                      "mac": {
                                          stage("install") {echo "installing in mac"}
                                          stage("test") {echo "testing in mac"}
                                      }
                                  ])
                              }
                          }
                      }
                      stage("cleanup") {steps {echo "cleanup"}}
                  }
              }

               

            Show
            betoz Alberto Arango added a comment - Here I share three code examples (bottom of this comment) declarative parallel within declarative pipeline scripted parallel within scripted pipeline scripted parallel within declarative pipeline The first two get visualized as expected. The third one gets a messed up visualization starting at the parallel part, i.e. also affects the next non-parallel stage. The code declarative parallel within declarative pipeline pipeline { agent( "none" ) stages { stage( "prepare" ) {steps {echo "prepare" }} stage( "build" ) {steps {echo "build" }} stage( "test" ) { parallel { stage( "linux" ) { stages { stage( "install" ) {steps {echo "installing in linux" }} stage( "test" ) {steps {echo "testing in linux" }} } } stage( "windows" ) { stages { stage( "install" ) {steps {echo "installing in windows" }} stage( "test" ) {steps {echo "testing in windows" }} } } stage( "mac" ) { stages { stage( "install" ) {steps {echo "installing in mac" }} stage( "test" ) {steps {echo "testing in mac" }} } } } } stage( "cleanup" ) {steps {echo "cleanup" }} } } scripted parallel within scripted pipeline stage( "prepare" ) {echo "prepare" } stage( "build" ) {echo "build" } stage( "test" ) { parallel ([ "linux" : { stage( "install" ) {echo "installing in linux" } stage( "test" ) {echo "testing in linux" } }, "windows" : { stage( "install" ) {echo "installing in windows" } stage( "test" ) {echo "testing in windows" } }, "mac" : { stage( "install" ) {echo "installing in mac" } stage( "test" ) {echo "testing in mac" } } ]) } stage( "cleanup" ) {echo "cleanup" } scripted parallel within declarative pipeline pipeline { agent( "none" ) stages { stage( "prepare" ) {steps {echo "prepare" }} stage( "build" ) {steps {echo "build" }} stage( "test" ) { steps { script { parallel ([ "linux" : { stage( "install" ) {echo "installing in linux" } stage( "test" ) {echo "testing in linux" } }, "windows" : { stage( "install" ) {echo "installing in windows" } stage( "test" ) {echo "testing in windows" } }, "mac" : { stage( "install" ) {echo "installing in mac" } stage( "test" ) {echo "testing in mac" } } ]) } } } stage( "cleanup" ) {steps {echo "cleanup" }} } }  

              People

              • Assignee:
                Unassigned
                Reporter:
                acarr468 Adam Carroll
              • Votes:
                26 Vote for this issue
                Watchers:
                32 Start watching this issue

                Dates

                • Created:
                  Updated: