Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54010

Support the visualization of two levels of parallelity in stages

    Details

    • Similar Issues:

      Description

      Support scripted parallel blocks in parallel stages in a declarative pipeline.

      Example pipeline:

      pipeline {
          agent none
          stages {
              stage('Parallel Stage') {
                  parallel {
                      stage('Stage 1') {
                          steps {
                              echo "Stage 1"
                          }
                      }
                      stage('Stage 2') {
                          steps {
                              script {
                                  parallel (
                                      "Stage 2.1.": {
                                          echo "Stage 2.1."
                                      },
                                      "Stage 2.2.": {
                                          echo "Stage 2.2."
                                      }
                                  )
                              }
                          }
                      }
                  }
              }
          }
      }
      

      Currently it is not properly visualized in blue ocean as shown in parallel-stages.png

       

      The idea how it should be visualized is shown in parallel-stages-new.png

        Attachments

          Issue Links

            Activity

            kurzy Daniel Kurzynski created issue -
            kurzy Daniel Kurzynski made changes -
            Field Original Value New Value
            Description Support scripted parallel blocks in parallel stages in a declarative pipeline: 

            Example pipeline:
            ```
            pipeline {
                agent none
                stages {
                    stage('Parallel Stage') {
                        parallel {
                            stage('Stage 1') {
                                steps {
                                    echo "Stage 1"
                                }
                            }
                            stage('Stage 2') {
                                steps {
                                    script {
                                        parallel (
                                            "Stage 2.1.": {
                                                echo "Stage 2.1."
                                            },
                                            "Stage 2.2.": {
                                                echo "Stage 2.2."
                                            }
                                        )
                                    }
                                }
                            }
                        }
                    }
                }
            }
            ```

            Currently it is not properly visualized in blue ocean as shown in parallel-stages.png
            Support scripted parallel blocks in parallel stages in a declarative pipeline.

            Example pipeline:
            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Parallel Stage') {
                        parallel {
                            stage('Stage 1') {
                                steps {
                                    echo "Stage 1"
                                }
                            }
                            stage('Stage 2') {
                                steps {
                                    script {
                                        parallel (
                                            "Stage 2.1.": {
                                                echo "Stage 2.1."
                                            },
                                            "Stage 2.2.": {
                                                echo "Stage 2.2."
                                            }
                                        )
                                    }
                                }
                            }
                        }
                    }
                }
            }
            {code}

            Currently it is not properly visualized in blue ocean as shown in parallel-stages.png
            kurzy Daniel Kurzynski made changes -
            Attachment parallel-stages-new.png [ 44753 ]
            kurzy Daniel Kurzynski made changes -
            Description Support scripted parallel blocks in parallel stages in a declarative pipeline.

            Example pipeline:
            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Parallel Stage') {
                        parallel {
                            stage('Stage 1') {
                                steps {
                                    echo "Stage 1"
                                }
                            }
                            stage('Stage 2') {
                                steps {
                                    script {
                                        parallel (
                                            "Stage 2.1.": {
                                                echo "Stage 2.1."
                                            },
                                            "Stage 2.2.": {
                                                echo "Stage 2.2."
                                            }
                                        )
                                    }
                                }
                            }
                        }
                    }
                }
            }
            {code}

            Currently it is not properly visualized in blue ocean as shown in parallel-stages.png
            Support scripted parallel blocks in parallel stages in a declarative pipeline.

            Example pipeline:
            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Parallel Stage') {
                        parallel {
                            stage('Stage 1') {
                                steps {
                                    echo "Stage 1"
                                }
                            }
                            stage('Stage 2') {
                                steps {
                                    script {
                                        parallel (
                                            "Stage 2.1.": {
                                                echo "Stage 2.1."
                                            },
                                            "Stage 2.2.": {
                                                echo "Stage 2.2."
                                            }
                                        )
                                    }
                                }
                            }
                        }
                    }
                }
            }
            {code}
            Currently it is not properly visualized in blue ocean as shown in parallel-stages.png

             

            The idea how it should be visualized is shown in parallel-stages-new.png
            kurzy Daniel Kurzynski made changes -
            Priority Minor [ 4 ] Critical [ 2 ]
            philsegal Phil Segal made changes -
            Assignee Phil Segal [ philsegal ]
            philsegal Phil Segal made changes -
            Assignee Phil Segal [ philsegal ]
            Hide
            philsegal Phil Segal added a comment -

            I would hope this would also support the same for scripted pipelines too.

            Show
            philsegal Phil Segal added a comment - I would hope this would also support the same for scripted pipelines too.
            kurzy Daniel Kurzynski made changes -
            Link This issue is related to JENKINS-50532 [ JENKINS-50532 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-38442 [ JENKINS-38442 ]
            Hide
            cloudcompany Tristan Cladet added a comment -

            +100  Today, I am stucked to run different kind of tests (python, qtest, ...) in parallel on two configurations (Windows and Linux) in parallel.

            Show
            cloudcompany Tristan Cladet added a comment - +100  Today, I am stucked to run different kind of tests (python, qtest, ...) in parallel on two configurations (Windows and Linux) in parallel.
            Hide
            digaus Dirk Gausmann added a comment - - edited

            This picture demonstrates it better... running parallel within parallel without a step before makes no sense since you could just run them in the outer parallel task.

            https://issues.jenkins-ci.org/secure/attachment/36602/pipeline_mockup.png

            Show
            digaus Dirk Gausmann added a comment - - edited This picture demonstrates it better... running parallel within parallel without a step before makes no sense since you could just run them in the outer parallel task. https://issues.jenkins-ci.org/secure/attachment/36602/pipeline_mockup.png
            Hide
            asg Ahmed Sghaier added a comment -

            Dirk Gausmann Actually the complete idea from Ben Dean would be really nice to have. Check this comment here:

            https://issues.jenkins-ci.org/browse/JENKINS-38442?focusedCommentId=292647&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-292647

            It would be nice to have that not only for the declarative, but also the scripted pipeline.

            Sometimes the test suites to cover are not statically declared and can depend on different things for different branches, hens the need for the scripted pipeline.

            Show
            asg Ahmed Sghaier added a comment - Dirk Gausmann Actually the complete idea from Ben Dean would be really nice to have. Check this comment here: https://issues.jenkins-ci.org/browse/JENKINS-38442?focusedCommentId=292647&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-292647 It would be nice to have that not only for the declarative, but also the scripted pipeline. Sometimes the test suites to cover are not statically declared and can depend on different things for different branches, hens the need for the scripted pipeline.
            Hide
            mirceaalbu Mircea-Andrei Albu added a comment -

            We need this for the scripted pipeline as well.

            Show
            mirceaalbu Mircea-Andrei Albu added a comment - We need this for the scripted pipeline as well.
            Hide
            rthakkar Rishi Thakkar added a comment -

            Is there any progress on this task? My team has a highly parallel flow that requires multiple layers of nested parallels. It would make life easier for debugging purposes if this feature were implemented.

            Show
            rthakkar Rishi Thakkar added a comment - Is there any progress on this task? My team has a highly parallel flow that requires multiple layers of nested parallels. It would make life easier for debugging purposes if this feature were implemented.
            Hide
            freme Thomas William added a comment -

            same here - i have a 50+ different build configs to run in parallel, a dozen per OS (Linux, MacOS, iOS, Android, Windows . . .) - Blue Ocean graph is pretty useless this way . . .

            Show
            freme Thomas William added a comment - same here - i have a 50+ different build configs to run in parallel, a dozen per OS (Linux, MacOS, iOS, Android, Windows . . .) - Blue Ocean graph is pretty useless this way . . .
            Hide
            shahafd Shahaf Duenyas added a comment -

            This feature will help us a lot.

            Multiple levels of parallelizm are currently supported only by creating new jobs, which is a poor solution given that Jenkins groovy supports it out of the box.

            I'm sure this feature will help a lot of users since many applications test themselves on multiple platforms today.

            Show
            shahafd Shahaf Duenyas added a comment - This feature will help us a lot. Multiple levels of parallelizm are currently supported only by creating new jobs, which is a poor solution given that Jenkins groovy supports it out of the box. I'm sure this feature will help a lot of users since many applications test themselves on multiple platforms today.
            Hide
            asavanchuk Alaiksei Savanchuk added a comment -

            Shahaf Duenyas I'm not sure that you are talking about the BlueOcean plugin.

            Jenkins pipeline already supports multiple levels of parallelism, this is only about Blue Ocean's UI

            Show
            asavanchuk Alaiksei Savanchuk added a comment - Shahaf Duenyas  I'm not sure that you are talking about the BlueOcean plugin. Jenkins pipeline already supports multiple levels of parallelism, this is only about Blue Ocean's UI
            Hide
            shahafd Shahaf Duenyas added a comment -

            Alaiksei Savanchuk - I am talking about the BlueOcean plugin.

            We work with BlueOcean and we have multiple levels of parallelizm in our pipeline, and the only method to show it correctly using the GUI is creating multiple jobs for the different paltforms, otherwise it shows all of the parallel steps in the internal parallel block in the same level of the parallel steps of the external parallel block which is obviously wrong, furthermore, it also shows the serial steps of the internal job as parallel.

             

            For example - we have the following pipeline:

            Parallel:

                Linux:

                    Build

                    Parallel:

                        Fixed tests

                        Custom tests

                Windows:

                    Build

             

            We currently create a different job for windows and linux pipes and trigger them from the master job.

            When trying to run it in a single job, BlueOcean shows it as follows:

            Parallel:

                Linux

                Fixed tests

                Custom tests

                Windows

             

            I would very much like to see it as shown in JENKINS-38442 (pipeline_mockup.png)

            Show
            shahafd Shahaf Duenyas added a comment - Alaiksei Savanchuk - I am talking about the BlueOcean plugin. We work with BlueOcean and we have multiple levels of parallelizm in our pipeline, and the only method to show it correctly using the GUI is creating multiple jobs for the different paltforms, otherwise it shows all of the parallel steps in the internal parallel block in the same level of the parallel steps of the external parallel block which is obviously wrong, furthermore, it also shows the serial steps of the internal job as parallel.   For example - we have the following pipeline: Parallel:     Linux:         Build         Parallel:             Fixed tests             Custom tests     Windows:         Build   We currently create a different job for windows and linux pipes and trigger them from the master job. When trying to run it in a single job, BlueOcean shows it as follows: Parallel:     Linux     Fixed tests     Custom tests     Windows   I would very much like to see it as shown in  JENKINS-38442 (pipeline_mockup.png)
            Hide
            nedrebo Mathias Nedrebø added a comment - - edited

            Our team is also hoping for a solution to this, the setup we have is similar to what Shahaf Duenyas describes, starting with some sequential pre-work, then splitting into windows and linux build, then ideally further splitting several times down the pipeline (multiple tests that each is parameterized over OS and HW configurations and fully parallelizable).

            Right now our pipeline takes three times as long as it should because we really like to have the user friendly visualization we get from Blue Ocean. This is of course not ideal because time to failure is longer that it should be and the dev cycle time suffers.

            I think this issue should add support for any levels of parallelism, not hard code it for any arbitrarily magic number of parallelism.

            The issue should be renamed to "Support nested parallel stages"

            I also agree that the "pipeline_mockup.png" in JENKINS-38442 looks like the natural way to render this and it should scale well. Rendering the X(1 for example) top level parallel stages different then the N-X inner levels, as I have seen proposed, seems like a hack tailored to some specific use case.

            Show
            nedrebo Mathias Nedrebø added a comment - - edited Our team is also hoping for a solution to this, the setup we have is similar to what Shahaf Duenyas describes, starting with some sequential pre-work, then splitting into windows and linux build, then ideally further splitting several times down the pipeline (multiple tests that each is parameterized over OS and HW configurations and fully parallelizable). Right now our pipeline takes three times as long as it should because we really like to have the user friendly visualization we get from Blue Ocean. This is of course not ideal because time to failure is longer that it should be and the dev cycle time suffers. I think this issue should add support for any levels of parallelism, not hard code it for any arbitrarily magic number of parallelism. The issue should be renamed to "Support nested parallel stages" I also agree that the "pipeline_mockup.png" in  JENKINS-38442 looks like the natural way to render this and it should scale well. Rendering the X(1 for example) top level parallel stages different then the N-X inner levels, as I have seen proposed, seems like a hack tailored to some specific use case.
            rthakkar Rishi Thakkar made changes -
            rthakkar Rishi Thakkar made changes -
            Hide
            rthakkar Rishi Thakkar added a comment - - edited

            My set up is also very similar to the one that Shahaf Duenyas has described.

            A possible solution is to combine all nested stages into one node. When you click on this node, it expands and shows you everything else that was under it. This way the view is fairly clean and intuitive to use. See following images for examples:

            In the image below lets imagine I clicked on build 1:

            The UI will then transition to show the following nested pipeline:

             

            This basically steps away from only supporting nested parallels to just supporting nested pipelines.

            Is there anyone who can be considered the owner of this issue? It would be great if we could come up with a plan and start executing. As  Shahaf Duenyas  said, when you have 50+ builds in parallel, the UI is pretty much useless without nesting support.

            Show
            rthakkar Rishi Thakkar added a comment - - edited My set up is also very similar to the one that Shahaf Duenyas has described. A possible solution is to combine all nested stages into one node. When you click on this node, it expands and shows you everything else that was under it. This way the view is fairly clean and intuitive to use. See following images for examples: In the image below lets imagine I clicked on build 1: The UI will then transition to show the following nested pipeline:   This basically steps away from only supporting nested parallels to just supporting nested pipelines. Is there anyone who can be considered the owner of this issue? It would be great if we could come up with a plan and start executing. As  Shahaf Duenyas   said, when you have 50+ builds in parallel, the UI is pretty much useless without nesting support.
            Hide
            rthakkar Rishi Thakkar added a comment -

            Could someone from CloudBees Inc. possibly take a look at this? I have seen a few other tickets related to parallel visualization in blue ocean owned by you guys.

            Show
            rthakkar Rishi Thakkar added a comment - Could someone from CloudBees Inc.  possibly take a look at this? I have seen a few other tickets related to parallel visualization in blue ocean owned by you guys.
            Hide
            ianfixes Ian Katz added a comment -

            The essence of this issue is a mismatch between the ability to express a nested (and valid) workflow in the Jenkinsfile and the UI's ability to render it.  I don't know of a limit on nesting in the Jenkinsfile, so it follows that there shouldn't be such a limit in the UI either.

             

            To me, the significance of this issue is that BlueOcean is used for CI – its job is to help us execute, understand, and troubleshoot some complex procedures in dev/release.  Failing to represent the procedure itself undermines this entire concept.

            Show
            ianfixes Ian Katz added a comment - The essence of this issue is a mismatch between the ability to express a nested (and valid) workflow in the Jenkinsfile and the UI's ability to render it.  I don't know of a limit on nesting in the Jenkinsfile, so it follows that there shouldn't be such a limit in the UI either.   To me, the significance of this issue is that BlueOcean is used for CI – its job is to help us execute, understand, and troubleshoot some complex procedures in dev/release.  Failing to represent the procedure itself undermines this entire concept.
            Hide
            rthakkar Rishi Thakkar added a comment -

            Is it possible to bump the priority of this issue to a blocker or do we need someone special to do that?

            Show
            rthakkar Rishi Thakkar added a comment - Is it possible to bump the priority of this issue to a blocker or do we need someone special to do that?
            rthakkar Rishi Thakkar made changes -
            Priority Critical [ 2 ] Blocker [ 1 ]
            Hide
            rthakkar Rishi Thakkar added a comment -

            I am bumping priority to blocker. Please let me know if this is something that I was not allowed to do.

            Hopefully, this will bring a little more attention to this issue.

            Show
            rthakkar Rishi Thakkar added a comment - I am bumping priority to blocker. Please let me know if this is something that I was not allowed to do. Hopefully, this will bring a little more attention to this issue.
            Hide
            luckyhorang Hokwang Lee added a comment -

            I am looking forward parallel in parallel feature, whole code of declarative syntax.

            Show
            luckyhorang Hokwang Lee added a comment - I am looking forward parallel in parallel feature, whole code of declarative syntax.
            Hide
            trymb Trym Bremnes added a comment -

            This is a very important issue for us to be resolved. Without this our pipeline takes so much longer to complete, since almost everything has to be done in series instead of parallel. 

            Show
            trymb Trym Bremnes added a comment - This is a very important issue for us to be resolved. Without this our pipeline takes so much longer to complete, since almost everything has to be done in series instead of parallel. 
            Hide
            jeremfg Jérémie Faucher-Goulet added a comment -

            I'm surprised this isn't supported by the UI.

            We develop firmware and cross-compile to different architecture using agent of different OS. This equates to a lot of builds in parallels, with internal stages in parallel as well (such as static code analysis, running tests, etc.).

            Parallelism is a must otherwise build times are way too long and a pain for developers who want rapid cycles of feedback and not wait for a build to complete.

            Show
            jeremfg Jérémie Faucher-Goulet added a comment - I'm surprised this isn't supported by the UI. We develop firmware and cross-compile to different architecture using agent of different OS. This equates to a lot of builds in parallels, with internal stages in parallel as well (such as static code analysis, running tests, etc.). Parallelism is a must otherwise build times are way too long and a pain for developers who want rapid cycles of feedback and not wait for a build to complete.
            Hide
            shahafd Shahaf Duenyas added a comment -

            Is there any reason why this issue doesn't get prioritized ?

            It is already marked as a blocker for 6 months, and was critical for another 6 months earlier.

            Can you please update on the status of this request ? Is it in development ? Do you plan to add this feature soon ?

            It is very crucial for many people using BlueOcean plugin.

            Show
            shahafd Shahaf Duenyas added a comment - Is there any reason why this issue doesn't get prioritized ? It is already marked as a blocker for 6 months, and was critical for another 6 months earlier. Can you please update on the status of this request ? Is it in development ? Do you plan to add this feature soon ? It is very crucial for many people using BlueOcean plugin.
            Hide
            ianfixes Ian Katz added a comment -

            I think a better question to ask would be "what technical hurdles block this request".  My guess is that when you run things in parallel, each parallel item counts as a "stage" to the UI, and so what we are really discussing here is a need to support sub-stages – something that seems a bit more complicated than a UI fix.

            Is that accurate? 

            Show
            ianfixes Ian Katz added a comment - I think a better question to ask would be "what technical hurdles block this request".  My guess is that when you run things in parallel, each parallel item counts as a "stage" to the UI, and so what we are really discussing here is a need to support sub-stages – something that seems a bit more complicated than a UI fix. Is that accurate? 
            Hide
            rpuvvada Rajiv Puvvada added a comment -

            +1 I'm currently running 20 parallel stages in my pipeline for speed but the BlueOcean visualization of a tall column of nodes isn't the most user friendly. Nested parallel would be one solution to this. I'd also be happy with a purely UI construct to categorize my stages for better visualization. This has been mentioned in a series of tickets resolved as dupes that ultimately end up here. An example based on the description:

             

            pipeline {
                agent none
                stages {
                    stage('Parallel Stage') {
                        parallel {
                            stage('Stage 1') {
                                steps {
                                    echo "Stage 1"
                                }
                            }
                            category('Stage 2') {
                                stage("Stage 2.1"): {
                                    steps {
                                        echo "Stage 2.1."
                                    }
                                },
                                stage(Stage 2.2") {
                                    steps {
                                        echo "Stage 2.2."
                                    }
                                }
                            }
                        }
                    }
                }
            }
            
            Show
            rpuvvada Rajiv Puvvada added a comment - +1 I'm currently running 20 parallel stages in my pipeline for speed but the BlueOcean visualization of a tall column of nodes isn't the most user friendly. Nested parallel would be one solution to this. I'd also be happy with a purely UI construct to categorize my stages for better visualization. This has been mentioned in a series of tickets resolved as dupes that ultimately end up here. An example based on the description:   pipeline { agent none stages { stage( 'Parallel Stage' ) { parallel { stage( 'Stage 1' ) { steps { echo "Stage 1" } } category( 'Stage 2' ) { stage( "Stage 2.1" ): { steps { echo "Stage 2.1." } }, stage(Stage 2.2") { steps { echo "Stage 2.2." } } } } } } }
            Hide
            rockmax Rajat Saxena added a comment -

            Blueocean team - Any updates on this? Are you guys working on this issue since its a blocker for us?

             

            • Adobe Developer
            Show
            rockmax Rajat Saxena added a comment - Blueocean team - Any updates on this? Are you guys working on this issue since its a blocker for us?   Adobe Developer
            shanidar1 Shani Dar made changes -
            Attachment image-2019-12-18-10-50-54-224.png [ 49808 ]
            shanidar1 Shani Dar made changes -
            Attachment Selection_090.png [ 49809 ]
            shanidar1 Shani Dar made changes -
            Attachment Selection_093.png [ 49810 ]
            Hide
            shanidar1 Shani Dar added a comment -

            What I did as a workaround is building a parallel stage dynamically - 

            We deploy 5 services for two clouds, we want to deploy all in parallel.  Let's say each service takes 5 min to deploy, we had to wait 10 minutes because we couldn't run two cloud stages in parallel that runs multiple services stages also in parallel:

            So I built one parallel stage with all 10 services, and named each step with a relevant prefix:

             

            stage('Deploy') {
                                steps {
                                    script {
                                        def parallelStagesMap = [:]
                                            AWS_CLUSTER_NAME.collect {
                                               parallelStagesMap.put("AWS ${it}", buildUtils.Deploy('AWS', it))
                                            }
                                            GCP_CLUSTER_NAME.collect {
                                               parallelStagesMap.put("GCP ${it}", buildUtils.Deploy('GCP', it))
                                            }
            
                                         parallel parallelStagesMap
                                    }
                                }
                            }
            

             

             

             

             

            Show
            shanidar1 Shani Dar added a comment - What I did as a workaround is building a parallel stage dynamically -  We deploy 5 services for two clouds, we want to deploy all in parallel.  Let's say each service takes 5 min to deploy, we had to wait 10 minutes because we couldn't run two cloud stages in parallel that runs multiple services stages also in parallel: So I built one parallel stage with all 10 services, and named each step with a relevant prefix:   stage( 'Deploy' ) { steps { script { def parallelStagesMap = [:] AWS_CLUSTER_NAME.collect { parallelStagesMap.put( "AWS ${it}" , buildUtils.Deploy( 'AWS' , it)) } GCP_CLUSTER_NAME.collect { parallelStagesMap.put( "GCP ${it}" , buildUtils.Deploy( 'GCP' , it)) } parallel parallelStagesMap } } }        
            Hide
            ianfixes Ian Katz added a comment - - edited

            That's a good workaround. I can't speak for everyone here, but my personal interest in this issue is more to do with the first attached screenshot in the original issue – the ability to have multiple sequential stages shown in each parallel branch. In other words, to not have it all squish horizontally into a single stage that represents the entire branch.

            Show
            ianfixes Ian Katz added a comment - - edited That's a good workaround. I can't speak for everyone here, but my personal interest in this issue is more to do with the first attached screenshot in the original issue – the ability to have multiple sequential stages shown in each parallel branch. In other words, to not have it all squish horizontally into a single stage that represents the entire branch.
            Hide
            shanidar1 Shani Dar added a comment -

            Ian Katz I agree, I'm also waiting for this solution, that's what I did meanwhile until this issue will be resolved..

            Show
            shanidar1 Shani Dar added a comment - Ian Katz I agree, I'm also waiting for this solution, that's what I did meanwhile until this issue will be resolved..
            Hide
            rvnairme Rahul Nair added a comment -

            Is there a plan to update the visualization as shown in parallel-stages-new.png ? Or is it available in already in any of the new releases ?

            Show
            rvnairme Rahul Nair added a comment - Is there a plan to update the visualization as shown in parallel-stages-new.png ? Or is it available in already in any of the new releases ?
            Hide
            amidar Amit Dar added a comment -

            Hi,

             

            any update regarding this issue?

            Show
            amidar Amit Dar added a comment - Hi,   any update regarding this issue?
            Hide
            henryborchers Henry Borchers added a comment -

            Having multiple levels of parallel stages would optimize the use of my executors and significantly reduce my pipeline times. 

            This would be very helpful to me for stages that create multiple platform specific artifacts. When I do this now, I have to have one matrix stage that builds Windows and Linux packages (using creative use of Dockerfile agents, I'm able to get these to run in parallel), and one stage dedicated to building Mac packages. Because matrix stages seem to use up a parallel stage under the hood, I can only run these stages one after another, even though the physical servers could easily run these steps independently .  

            Show
            henryborchers Henry Borchers added a comment - Having multiple levels of parallel stages would optimize the use of my executors and significantly reduce my pipeline times.  This would be very helpful to me for stages that create multiple platform specific artifacts. When I do this now, I have to have one matrix stage that builds Windows and Linux packages (using creative use of Dockerfile agents, I'm able to get these to run in parallel), and one stage dedicated to building Mac packages. Because matrix stages seem to use up a parallel stage under the hood, I can only run these stages one after another, even though the physical servers could easily run these steps independently .  
            Hide
            alonmiz alon miz added a comment -

            that's a really important feature for our team as well. we have an optimized ci with multiple parallelized steps, which look really awkward without that capability

            Henry Borchers in your case there's no limitation in using parallelism inside another. the downside is that it's really hard to view the process in the UI and in case of a failure step, the UI just don't understand how to show it.

            Show
            alonmiz alon miz added a comment - that's a really important feature for our team as well. we have an optimized ci with multiple parallelized steps, which look really awkward without that capability Henry Borchers in your case there's no limitation in using parallelism inside another. the downside is that it's really hard to view the process in the UI and in case of a failure step, the UI just don't understand how to show it.
            batmat Baptiste Mathus made changes -
            Summary Support two levels of parallelity in stages Support the visualization of two levels of parallelity in stages

              People

              • Assignee:
                Unassigned
                Reporter:
                kurzy Daniel Kurzynski
              • Votes:
                111 Vote for this issue
                Watchers:
                114 Start watching this issue

                Dates

                • Created:
                  Updated: