Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39256

Allow stages to appear within nodes

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • pipeline-view-plugin
    • None

      I think this issue may have already been raised or at least a similar version of it, but I couldnt find an example of exactly what I'm looking for. In a way this is the inverse of this other issue I logged: https://issues.jenkins-ci.org/browse/JENKINS-39119.

      In pre-pipeline Jenkins we'd have a single job that does git checkout, build, unit test, archive, etc. For pipeline we liked having these nice pretty stages for each of these steps. The problem is that the stash/unstash workflow really slows things down with the only benefit being the pretty stage display.

      If we put all of these steps into a single stage then it all works great, but then we just have this single green or red box which doesn't feel very 'pipeline-y'. We thought we solved everything when realizing we could save the node information between stages and force downstream stages to run on the same node. That way we could run in the same workspace and avoid the need for all the unstashing.

      The big catch with this approach is that between stages the node is briefly available so if another job is queued up then it will immediately grab it and the test stage will be stuck waiting for the now busy node. Our team has spent way too long trying to solve this core problem by trying our own methods of reserving a node by adding and removing node labels at the beginning and end of the pipeline. That however has created its own headaches in that modifying the labels doesnt seem reliable.

      We've also taken a look at the External Workspace Manager plugin but compiling a build over a network share is slower than doing stash/unstash.

      What would really solve things is if the stage view would let us parallelize for platform x config (so we're running say Mac-Debug, Mac-Release, Win-Debug, WinRelease in parallel), and then go linear from there so that for example on the Mac-Debug flavor we could see a nice stage view separation of checkout, build, test, etc all running on the same node/workspace.

      Is such an approach possible? It's been maddening to have to decide between performance/stability vs pretty stage view. On one hand it's a no brainer that stability/performance is most important, but it's hard to market an exciting new pipeline upgrade when you only display a single stage.

            Unassigned Unassigned
            rainwaj Justin Rainwater
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: