Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49050

I can view my sequential declarative pipeline stages in the pipeline visualization plugin

    Details

    • Similar Issues:
    • Released As:
      Blue Ocean 1.8.0

      Description

      For Declarative 1.3 (JENKINS-46809), we're adding support for sequential groups of stages, either as part of a set of parallel stages or on its own in a top-level stage. I won't bother going into all the rationale behind it. =) See the Declarative PR for the relevant changes, examples, discussion, etc.

      This will require changes for both visualization and the editor:

      • Visualization - see my pathetic attempts at mockups attached here. James Dumay probably will come up with something better in like five seconds. =)
      • Editor - Should be something like we have for parallel stages - reordering would be good too.

        Attachments

          Issue Links

            Activity

            Hide
            michaelneale Michael Neale added a comment -

            hey Josh McDonald any guesses on how much work this would be for both vis and editor? 

            Show
            michaelneale Michael Neale added a comment - hey Josh McDonald any guesses on how much work this would be for both vis and editor? 
            Hide
            sophistifunk Josh McDonald added a comment -

            Renderer: zero for multi-stage-parallels, as that's already done. As for "grouping" at the top level, it's mostly about cooking up a sensible visual language rather than layout changes.

             

            Editor-wise, I'm not sure. We need to replace the editor's mixed-in layout code with the renderer layout class, which was extracted from the renderer for this purpose and should be done either way.

             

            Also, we would need changes to the `PipelineNodeGraphVisitor` to support it. I have ideas about refactoring that to make it less... intimidating... to change, but I'll need to run them by Vivek and probably a few people on pipeline to make sure I'm not talking out of my ass

            Show
            sophistifunk Josh McDonald added a comment - Renderer: zero for multi-stage-parallels, as that's already done. As for "grouping" at the top level, it's mostly about cooking up a sensible visual language rather than layout changes.   Editor-wise, I'm not sure. We need to replace the editor's mixed-in layout code with the renderer layout class, which was extracted from the renderer for this purpose and should be done either way.   Also, we would need changes to the `PipelineNodeGraphVisitor` to support it. I have ideas about refactoring that to make it less... intimidating... to change, but I'll need to run them by Vivek and probably a few people on pipeline to make sure I'm not talking out of my ass
            Hide
            michaelneale Michael Neale added a comment -

            thanks Josh McDonald - so it sounds like "not a small thing" 

            Show
            michaelneale Michael Neale added a comment - thanks Josh McDonald - so it sounds like "not a small thing" 
            Hide
            vivek Vivek Pandey added a comment -

            Yeah we need changes in nodes  API, opened this ticket: https://issues.jenkins-ci.org/browse/JENKINS-49779.

            Show
            vivek Vivek Pandey added a comment - Yeah we need changes in nodes  API, opened this ticket:  https://issues.jenkins-ci.org/browse/JENKINS-49779 .
            Hide
            horangs Hokwang Lee added a comment -

            Hello.
            I was looking forward to implementing this sequential in parallel feature. Yesterday finally I can use this in Jenkinsfile as code. But I didn't know visualization does not support yet!

            Please assign right person and consider very critical thing to do.

            Show
            horangs Hokwang Lee added a comment - Hello. I was looking forward to implementing this sequential in parallel feature. Yesterday finally I can use this in Jenkinsfile as code. But I didn't know visualization does not support yet! Please assign right person and consider very critical thing to do.
            Hide
            jbriden Jenn Briden added a comment -

            We will be focusing first on visualization of sequential stages within the UI. We've begun that work. Editing needs significant review so that we can create an intuitive user experience. In the sake of hyper focusing on delivering high quality experiences for the community very quickly, we are taking one UX task at a time. Stay tuned!

            Show
            jbriden Jenn Briden added a comment - We will be focusing first on visualization of sequential stages within the UI. We've begun that work. Editing needs significant review so that we can create an intuitive user experience. In the sake of hyper focusing on delivering high quality experiences for the community very quickly, we are taking one UX task at a time. Stay tuned!
            Hide
            olamy Olivier Lamy added a comment -

            first we need to improve backend rest api.

            Show
            olamy Olivier Lamy added a comment - first we need to improve backend rest api.
            Show
            olamy Olivier Lamy added a comment - pr https://github.com/jenkinsci/blueocean-plugin/pull/1778
            Hide
            sophistifunk Josh McDonald added a comment -

            Vivek Pandey I can take this over at the end of the week when I'm done with these last few PoC tasks if you like, let Nicu enjoy his time seeing the sights. The code's reasonably fresh in my mind because I've been using parts of it for the demo.

            Show
            sophistifunk Josh McDonald added a comment - Vivek Pandey I can take this over at the end of the week when I'm done with these last few PoC tasks if you like, let Nicu enjoy his time seeing the sights. The code's reasonably fresh in my mind because I've been using parts of it for the demo.
            Hide
            vivek Vivek Pandey added a comment -

            Josh McDonald Fine with me, Nicolae Pascu ok with you? We would rather have you enjoy your vacation

            Show
            vivek Vivek Pandey added a comment - Josh McDonald Fine with me, Nicolae Pascu ok with you? We would rather have you enjoy your vacation
            Hide
            nicu Nicolae Pascu added a comment -

            Thanks Josh McDonald. I have all the debugging code already written locally, and I think I have an easy fix for this. I just talked with Olivier Lamy about a small bug that prevents me from testing my fix, once that is done, I think I can open a PR without much trouble. If I can't finish by the end of the week or it requires too much time I will contact you and hand over what I already have. I will keep everyone updated here with the progress.

            Show
            nicu Nicolae Pascu added a comment - Thanks Josh McDonald . I have all the debugging code already written locally, and I think I have an easy fix for this. I just talked with Olivier Lamy about a small bug that prevents me from testing my fix, once that is done, I think I can open a PR without much trouble. If I can't finish by the end of the week or it requires too much time I will contact you and hand over what I already have. I will keep everyone updated here with the progress.
            Hide
            nicu Nicolae Pascu added a comment -

            Chad Bercea here is the preview of this Jenkinsfile that we talked about in yesterdays meeting. Let me know what you think.

            Show
            nicu Nicolae Pascu added a comment - Chad Bercea here is the preview of this Jenkinsfile that we talked about in yesterdays meeting. Let me know what you think.
            Hide
            stevenfoster Steven Foster added a comment -

            From the screenshot, I guess that the stage group name ('multiple-stages' in the example Jenkinsfile) won't be displayed anywhere?

            Show
            stevenfoster Steven Foster added a comment - From the screenshot, I guess that the stage group name ('multiple-stages' in the example Jenkinsfile) won't be displayed anywhere?
            Hide
            cbercea Chad Bercea added a comment -

            Nicolae Pascu I really like the idea of no longer combining the top most stage name with the name of the "stack". In cases where this happens, what would we name that "parent"? Just curious. Otherwise this looks exactly like what we talked about. Nice!

            Show
            cbercea Chad Bercea added a comment - Nicolae Pascu I really like the idea of no longer combining the top most stage name with the name of the "stack". In cases where this happens, what would we name that "parent"? Just curious. Otherwise this looks exactly like what we talked about. Nice!
            Hide
            nicu Nicolae Pascu added a comment -

            Steven Foster that is correct, the container stage ('multiple-stages') can't have any steps so that is why we decided to not include it anywhere.

            Show
            nicu Nicolae Pascu added a comment - Steven Foster that is correct, the container stage ('multiple-stages') can't have any steps so that is why we decided to not include it anywhere.
            Hide
            nicu Nicolae Pascu added a comment -

            Thanks Chad Bercea, as for the edge case that you mention, I don't really understand it, I will PM you later so we can clarify.

            Show
            nicu Nicolae Pascu added a comment - Thanks  Chad Bercea , as for the edge case that you mention, I don't really understand it, I will PM you later so we can clarify.
            Hide
            nicu Nicolae Pascu added a comment -

            Opened PR#1784

            Show
            nicu Nicolae Pascu added a comment - Opened  PR#1784
            Hide
            mgrinberg Marcin Grinberg added a comment - - edited

            Hi Nicolae Pascu,

             

            Are you aware in which version of blue-ocean can we expect this feature?

             

            Thanks for your work on this feature btw.

            Show
            mgrinberg Marcin Grinberg added a comment - - edited Hi Nicolae Pascu ,   Are you aware in which version of blue-ocean can we expect this feature?   Thanks for your work on this feature btw.
            Hide
            nicu Nicolae Pascu added a comment -

            Hi Marcin Grinberg, this feature will land in the next version of blueocean, which will be 1.8

            Show
            nicu Nicolae Pascu added a comment - Hi  Marcin Grinberg , this feature will land in the next version of blueocean, which will be 1.8
            Hide
            feliwir Stephan Vedder added a comment - - edited

            This new feature in BlueOcean 1.8 did definetly not fix visualization of parallel sequential stages, but instead break it completly:

            Each of Windows, Linux etc. is one sequential stage. The corresponding Jenkinsfile looks like this:
            https://gist.github.com/feliwir/75864cb641c448446f836056fc698d8a

            While the build is running everything is bugging out, things are moving all over the screen, but not where they're supposed to be. Any advice why the visualization is completly bugged now is welcome!

            Show
            feliwir Stephan Vedder added a comment - - edited This new feature in BlueOcean 1.8 did definetly not fix visualization of parallel sequential stages, but instead break it completly: Each of Windows, Linux etc. is one sequential stage. The corresponding Jenkinsfile looks like this: https://gist.github.com/feliwir/75864cb641c448446f836056fc698d8a While the build is running everything is bugging out, things are moving all over the screen, but not where they're supposed to be. Any advice why the visualization is completly bugged now is welcome!
            Hide
            nicu Nicolae Pascu added a comment - - edited

            Hi Stephan Vedder, I looked over your Jenkinsfile and saw a problem. Per the sequential stages documentation each sequential parallel group needs to be contained by a named Stage that doesn't have any steps.

            As an example, your sequential stages called 'Build' and 'Test' need to have another container stage with a name that is the immediate child of the 'stages' container.

            Please take a look at this Jenkinsfile for an example.

            Show
            nicu Nicolae Pascu added a comment - - edited Hi Stephan Vedder , I looked over your Jenkinsfile and saw a problem. Per the  sequential stages documentation  each sequential parallel group needs to be contained by a named Stage that doesn't have any steps. As an example, your sequential stages called 'Build' and 'Test' need to have another container stage with a name that is the immediate child of the 'stages' container. Please take a look at this Jenkinsfile  for an example.
            Hide
            carelc Carel Combrink added a comment -

            Will this also work for "Scripted Pipeline syntax" or do one need to start migrating all "scripted" scripts to "declarative"?

            Show
            carelc Carel Combrink added a comment - Will this also work for "Scripted Pipeline syntax" or do one need to start migrating all "scripted" scripts to "declarative"?
            Hide
            olamy Olivier Lamy added a comment -

            Carel Combrink it's only supported with declarative

            Show
            olamy Olivier Lamy added a comment - Carel Combrink it's only supported with declarative
            Hide
            feliwir Stephan Vedder added a comment - - edited

            Nicolae Pascu i did look at the jenkinsfile you've linked above very closely, but i can't spot any differences. The "Build" and "Test" stage are directly in a stages block, just like your sequential stages (here). The nesting doesn't seem to be different in any way or did i miss sth?

            Show
            feliwir Stephan Vedder added a comment - - edited Nicolae Pascu i did look at the jenkinsfile you've linked above very closely, but i can't spot any differences. The "Build" and "Test" stage are directly in a stages block, just like your sequential stages ( here ). The nesting doesn't seem to be different in any way or did i miss sth?
            Hide
            abayer Andrew Bayer added a comment -

            Yeah, Stephan Vedder’s Jenkinsfile looks right to me - here’s how the stages are laid out:

            Continuous (children are parallel) ->
              Windows (children are sequential) ->
                Build
                Test
              Linux (children are sequential) ->
                Build
                Test
                Coverage
              MemCheck (children are sequential) ->
                Build
                Memcheck 
              Android (children are sequential) ->
                Build
              iOS (children are sequential) ->
                Build
            

            It’s a valid Jenkinsfile and should render correctly. Stephan Vedder, could you please open a new JIRA for this and include your Jenkinsfile there? Thanks.

            Show
            abayer Andrew Bayer added a comment - Yeah, Stephan Vedder ’s Jenkinsfile looks right to me - here’s how the stages are laid out: Continuous (children are parallel) -> Windows (children are sequential) -> Build Test Linux (children are sequential) -> Build Test Coverage MemCheck (children are sequential) -> Build Memcheck Android (children are sequential) -> Build iOS (children are sequential) -> Build It’s a valid Jenkinsfile and should render correctly. Stephan Vedder , could you please open a new JIRA for this and include your Jenkinsfile there? Thanks.
            Hide
            feliwir Stephan Vedder added a comment -
            Show
            feliwir Stephan Vedder added a comment - I've create the new issue: Andrew Bayer Nicolae Pascu See https://issues.jenkins-ci.org/browse/JENKINS-53048
            Hide
            hasselmm Mathias Hasselmann added a comment -

            Looks awesome! Thanks a lot!

            Show
            hasselmm Mathias Hasselmann added a comment - Looks awesome! Thanks a lot!
            Hide
            olamy Olivier Lamy added a comment -

            Hokwang Lee any Jenkinsfile to reproduce that?

            Show
            olamy Olivier Lamy added a comment - Hokwang Lee any Jenkinsfile to reproduce that?
            Hide
            horangs Hokwang Lee added a comment - - edited

            Nicolae Pascu and Andrew Bayer

             

            Today I can test this visualization, and it shows me like below. Nice works.

            But there's one big concern.

            In my pipeline code,

            stage('Parallel') : it really does not have meaning.

            stage('Feature A') and stage('Feature B') and stage('Feature C') : now there's no way to see, (it means sequential actions group(type) name)

            but this is more important and user should be aware of it.

             

            How do you think of this? And please let me know related issue number if exists. Or not, can I create new issue for this?

             

            Thank you.

            Show
            horangs Hokwang Lee added a comment - - edited Nicolae Pascu and Andrew Bayer   Today I can test this visualization, and it shows me like below. Nice works. But there's one big concern. In my pipeline code, stage('Parallel') : it really does not have meaning. stage('Feature A') and stage('Feature B') and stage('Feature C') : now there's no way to see, (it means sequential actions group(type) name) but this is more important and user should be aware of it.   How do you think of this? And please let me know related issue number if exists. Or not, can I create new issue for this?   Thank you.
            Hide
            horangs Hokwang Lee added a comment -

            Olivier Lamy Oh, please forget it, now I guess there's no problem. Please look at my comment anyway upside.

            Show
            horangs Hokwang Lee added a comment - Olivier Lamy Oh, please forget it, now I guess there's no problem. Please look at my comment anyway upside.
            Hide
            hasselmm Mathias Hasselmann added a comment -

            Hokwang Lee see JENKINS-53127. Unfortunately nobody triaged that issue yet.

            Also much more important: JENKINS-53547

            Show
            hasselmm Mathias Hasselmann added a comment - Hokwang Lee see  JENKINS-53127 . Unfortunately nobody triaged that issue yet. Also much more important:  JENKINS-53547

              People

              • Assignee:
                nicu Nicolae Pascu
                Reporter:
                abayer Andrew Bayer
              • Votes:
                18 Vote for this issue
                Watchers:
                39 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: