Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33185

Visualize parallel steps within a Pipeline Stage

    Details

    • Similar Issues:

      Description

      Support for visualizing parallel steps is unplanned

      A common thread in the feedback we have received is to represent parallel steps in the Stage View as cells within the Stage column.

      The use cases supported by Pipeline Stage View, such as visualizing stages across multiple runs and detecting changes in execution time or stages that fail regularly, work well due to the use of a table to visualize this data. Tables are excellent at allowing the eye to scan over a lot of related data.

      There has some debate within CloudBees on how parallel could be added to Pipeline Stage View without breaking the advantages that a table brings to the visualization. We believe that adding additional cells within the table column would reduce the usability of the overall experience, as scanning many rows becomes much more difficult.

      From a technical point of view it would be non-trivial to retrofit the Stage View with the parallel information.

      For the time being solving this problem has been delayed by CloudBees. However, we encourage any community members who are interested in solving this problem to pick it up.

      Over the last year CloudBees and the Jenkins community have been heavily invested in bringing a new user interface to Jenkins that is designed for Pipeline called Blue Ocean.

      Blue Ocean offers an entirely new way of visualizing Pipelines including parallel steps , and makes it much easier to navigate the Pipelines log by Stage and Step.

      We aim that over time that many of the use cases you have come to enjoy in Stage View will be available in Blue Ocean. We would like to encourage people interested in this feature to try out Blue Ocean and provide us feedback.

      You can find out more about Blue Ocean at https://jenkins.io/projects/blueocean/

      Thanks,
      James Dumay
      jdumay@cloudbees.com

      Original Request
      1) Parallel steps are the common workflow case
      2) Build logs for parallel steps are not informative
      3) A user would expect Stage View to somehow address the issue

      Current state:

      • stage declarations within parallel sections are not being considered as parallel
      • steps within one parallelized step are being mixed => no info in the view (see the screenshot)

      Test script (move stage declarations where you want):

      for (int i=0; i< 2; ++i) {
        stage "Stage #"+i
        print 'Hello, world $i!'
      }
      
      stage "Stage Parallel"
      def branches = [:]
      for (int i = 0; i < numHelloMessages.toInteger(); i++) {
        branches["split${i}"] = {
          stage "Stage parallel- #"+i
          node('remote') {
           echo  'Starting sleep'
           sleep 10
           echo  'Finished sleep'
          }
        }
      }
      parallel branches
      

      comment from JGlick:
      So concretely: the stage view would, for a given build (row), continue to display a linear sequence of stages (columns) with their associated times and so on; but each cell (stage of a build) would be some rendering of an st-planar graph, with structure determined according to the presence of steps of interest like parallel or a proposed label. Sort of like the existing Workflow Steps, but rendered as a graph rather than a tree table, restricted to displaying certain steps, and divided by stage.
      A simpler request would be to only display branches of one top-level parallel step, if any. (I.e., an st-planar graph which is either trivial—one vertex—or where all vertices other than the source and sink are directly connected to both the source and the sink.) This more limited UI would match that of the Build Pipeline plugin.
      Note that there is also a request JENKINS-29892 which would allow several stages to share concurrency semantics, but I think that need not have any effect on visualization; it would just be an internal change in the stage step.

        Attachments

        1. blue-ocean-parallel-generated-view.png
          blue-ocean-parallel-generated-view.png
          76 kB
        2. blue-ocean-parallel-WORKS.png
          blue-ocean-parallel-WORKS.png
          8 kB
        3. parallel.png
          parallel.png
          58 kB
        4. pipeline.png
          pipeline.png
          15 kB
        5. Screen Shot 2016-11-21 at 15.41.33.png
          Screen Shot 2016-11-21 at 15.41.33.png
          14 kB
        6. stage-duration.png
          stage-duration.png
          52 kB
        7. stages.png
          stages.png
          359 kB
        8. successful-pipeline.png
          successful-pipeline.png
          140 kB

          Issue Links

            Activity

            Hide
            michaelneale Michael Neale added a comment -

            usually that means some plugins have been removed/need to be updated (all blue ocean plugins).

            Show
            michaelneale Michael Neale added a comment - usually that means some plugins have been removed/need to be updated (all blue ocean plugins).
            Hide
            trejkaz trejkaz added a comment -

            It turned out that "Blue Ocean" plugin itself was never installed, but all the other parts somehow ended up being on the system even though nobody had manually installed them. I'm assuming Jenkins itself pulled in the individual parts without pulling in the parent. After manually adding it, the page finally loads, after never working the entire time since the link appeared on the site.

             

            Show
            trejkaz trejkaz added a comment - It turned out that "Blue Ocean" plugin itself was never installed, but all the other parts somehow ended up being on the system even though nobody had manually installed them. I'm assuming Jenkins itself pulled in the individual parts without pulling in the parent. After manually adding it, the page finally loads, after never working the entire time since the link appeared on the site.  
            Hide
            michaelneale Michael Neale added a comment -

            trejkaz ok that is good to hear - although I don't think other plugins yet depend on blue ocean parts, so not quite sure how it ended up doing that. There are other tickets on better generic error reporting that may make this error more obvious (although like with all error cases, edge cases mean it sometimes just wont work,but hopefully will at least give a hint).

            Show
            michaelneale Michael Neale added a comment - trejkaz ok that is good to hear - although I don't think other plugins yet depend on blue ocean parts, so not quite sure how it ended up doing that. There are other tickets on better generic error reporting that may make this error more obvious (although like with all error cases, edge cases mean it sometimes just wont work,but hopefully will at least give a hint).
            Hide
            trejkaz trejkaz added a comment -

            I'm not sure this view is really much better. I have to move down multiple pages of "pipelines" which aren't even pipelines to get to the only actual pipeline we have configured so far ... then when we do get to the one we want and look at some of the builds, the visualisation on each page doesn't even seem to show all the stages, just the one it's currently working on and then "End".

            Oh well.

            Show
            trejkaz trejkaz added a comment - I'm not sure this view is really much better. I have to move down multiple pages of "pipelines" which aren't even pipelines to get to the only actual pipeline we have configured so far ... then when we do get to the one we want and look at some of the builds, the visualisation on each page doesn't even seem to show all the stages, just the one it's currently working on and then "End". Oh well.
            Hide
            jeinstei Joshua Einstein-Curtis added a comment -

            I was wondering whatever happened to this? We have an architecture like the pseudo-code below, where the parallel is a stage, but the stages in the parallel aren't broken out. We need to do this to get different nodes within a stage. If there is a better way to do this so the visualization works, we'd love to know, but it is currently an ugly visual representation with only the first stage and the parallel stage, basically.

             

             

            def jobMap (
                  some['thing'] = {node (label) {
                     stage1
                     stage2
                  } }
            )
             
            node {
               def jobs = jobMap()
               stage1
               stage2 {
                  parallel jobs
               }
            }
            

             

            Show
            jeinstei Joshua Einstein-Curtis added a comment - I was wondering whatever happened to this? We have an architecture like the pseudo-code below, where the parallel is a stage, but the stages in the parallel aren't broken out. We need to do this to get different nodes within a stage. If there is a better way to do this so the visualization works, we'd love to know, but it is currently an ugly visual representation with only the first stage and the parallel stage, basically.     def jobMap (       some[ 'thing' ] = {node (label) { stage1 stage2       } } )   node { def jobs = jobMap()    stage1    stage2 {       parallel jobs    } }  

              People

              • Assignee:
                cloudbees CloudBees Inc.
                Reporter:
                hrmpw Patrick Wolf
              • Votes:
                201 Vote for this issue
                Watchers:
                210 Start watching this issue

                Dates

                • Created:
                  Updated: