Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39839

Missing StandardChunkVisitor.parallelBranchEnd() and parallelEnd() events if there are nested parallels

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      workflow-api plugin version 2.6.

      For the given pipeline script, secondLevelNestedBranch1 and secondLevelNestedBranch2 parallel branches are nested inside nestedBranch.

      There are missing StandardChunkVisitor.parallelEnd() and StandardChunkVisitor.parallelBranchEnd() events for nestedBranch. This causes computing incorrect DAG and status of individual branches. Blueocean extends StandardChunkVisitor to process events and create it's DAG.

      node {
         stage ('test') { 
           echo ('Testing'); 
           parallel nestedBranch: {
             echo 'nested Branch'
             stage('nestedBranchStage') {
               echo 'running nestedBranchStage'
               parallel secondLevelNestedBranch1: {
                 echo 'secondLevelNestedBranch1'
               },secondLevelNestedBranch2: {
                 echo 'secondLevelNestedBranch2'
               }
             }
           },
          failFast: false
         } 
      }
      

      If there are only one parallel branches in nested parallel then no parallelEnd or parallelBranchEnd event is fired. See the script below:

      node {
         stage ('test') { 
           echo ('Testing'); 
           parallel nestedBranch: {
             echo 'nested Branch'
             stage('nestedBranchStage') {
               echo 'running nestedBranchStage'
               parallel secondLevelNestedBranch1: {
                 echo 'secondLevelNestedBranch1'
               }
             }
           },
          failFast: false
         } 
      }
      

        Attachments

          Issue Links

            Activity

            Hide
            vivek Vivek Pandey added a comment -

            Looks like, firing of parallel events are broken in general. Below script has two stages, each stage with couple parallels, parallelStart event is not called. The pipeline script below has s2.0 and s2.1 are still running so parallelBranchEnd is not fired, understandable, but I expect parallelStart should have been fired.

            I collect branches in parallelStart and associate them with relevant stage, reset states etc. so this breaks association of parallel branches with the stage it belongs to. See https://issues.jenkins-ci.org/browse/JENKINS-39847

            chunkEnd=> id: 39, name: End of Pipeline, function: End of Pipeline
            chunkEnd=> id: 35, name: Stage : Body : End, function: }
              StartNode: org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode[id=24]
            
            parallelEnd=> id: 20, StartNode: 12, name: Execute in parallel : Start, function: parallel
            parallelBranchStart=> id: 31, name: Branch: S2.1, function: { (Branch: S2.1)
            parallelBranchStart=> id: 26, name: Branch: S2.0, function: { (Branch: S2.0)
            
            
            handleChunkDone=> id: 24, name: S2, function: { (S2)
            chunkStart=> id: 24, name: S2, function: { (S2)
            
            
            chunkEnd=> id: 21, name: Stage : Body : End, function: }
              StartNode: org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode[id=6]
            
            parallelEnd=> id: 20, StartNode: 12, name: Execute in parallel : Start, function: parallel
            
            parallelBranchEnd=> id: 19, StartNode: 15, name: Branch: S1.1.2, function: { (Branch: S1.1.2)
            parallelBranchStart=> id: 15, name: Branch: S1.1.2, function: { (Branch: S1.1.2)
            
            parallelBranchEnd=> id: 18, StartNode: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1)
            parallelBranchStart=> id: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1)
            
            parallelStart=> id: 12, name: Execute in parallel : Start, function: parallel
              branch=> id: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1)
            
            parallelBranchStart=> id: 8, name: Branch: S1.0, function: { (Branch: S1.0)
            handleChunkDone=> id: 6, name: S1, function: { (S1)
            chunkStart=> id: 6, name: S1, function: { (S1)
            
            node {
              stage('S1') {  
                parallel 'S1.0': { sh('sleep 10') }
                
                parallel 'S1.1.1': { sh('sleep 10') }, 'S1.1.2': { sh('sleep 10') }
              }
                stage('S2') {  
                parallel 'S2.0': { sh('sleep 10') }
                
                parallel 'S2.1': { sh('sleep 10') }
              }
            }
            
            Show
            vivek Vivek Pandey added a comment - Looks like, firing of parallel events are broken in general. Below script has two stages, each stage with couple parallels, parallelStart event is not called. The pipeline script below has s2.0 and s2.1 are still running so parallelBranchEnd is not fired, understandable, but I expect parallelStart should have been fired. I collect branches in parallelStart and associate them with relevant stage, reset states etc. so this breaks association of parallel branches with the stage it belongs to. See https://issues.jenkins-ci.org/browse/JENKINS-39847 chunkEnd=> id: 39, name: End of Pipeline, function: End of Pipeline chunkEnd=> id: 35, name: Stage : Body : End, function: } StartNode: org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode[id=24] parallelEnd=> id: 20, StartNode: 12, name: Execute in parallel : Start, function: parallel parallelBranchStart=> id: 31, name: Branch: S2.1, function: { (Branch: S2.1) parallelBranchStart=> id: 26, name: Branch: S2.0, function: { (Branch: S2.0) handleChunkDone=> id: 24, name: S2, function: { (S2) chunkStart=> id: 24, name: S2, function: { (S2) chunkEnd=> id: 21, name: Stage : Body : End, function: } StartNode: org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode[id=6] parallelEnd=> id: 20, StartNode: 12, name: Execute in parallel : Start, function: parallel parallelBranchEnd=> id: 19, StartNode: 15, name: Branch: S1.1.2, function: { (Branch: S1.1.2) parallelBranchStart=> id: 15, name: Branch: S1.1.2, function: { (Branch: S1.1.2) parallelBranchEnd=> id: 18, StartNode: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1) parallelBranchStart=> id: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1) parallelStart=> id: 12, name: Execute in parallel : Start, function: parallel branch=> id: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1) parallelBranchStart=> id: 8, name: Branch: S1.0, function: { (Branch: S1.0) handleChunkDone=> id: 6, name: S1, function: { (S1) chunkStart=> id: 6, name: S1, function: { (S1) node { stage( 'S1' ) { parallel 'S1.0' : { sh( 'sleep 10' ) } parallel 'S1.1.1' : { sh( 'sleep 10' ) }, 'S1.1.2' : { sh( 'sleep 10' ) } } stage( 'S2' ) { parallel 'S2.0' : { sh( 'sleep 10' ) } parallel 'S2.1' : { sh( 'sleep 10' ) } } }
            Hide
            svanoort Sam Van Oort added a comment -

            Vivek Pandey I probably discussed this in the phone at the time, but for the record: it's impossible to guarantee parallel/branch end events are always fired in the case where there is a parallel with only one branch, because:

            1. For an incomplete branch block (still running) you may not have an end block
            2. If there is not more than one current head node (current execution) you have no way to know it's inside a parallel

            This is best handled in your visitor by retaining nodes in case a parallel branch start is discovered (for incomplete flows at least).

            I can, however provide guarantees that the parallel starts are correctly handled (and they need to be), and am hardening tests for this case.

            Show
            svanoort Sam Van Oort added a comment - Vivek Pandey I probably discussed this in the phone at the time, but for the record: it's impossible to guarantee parallel/branch end events are always fired in the case where there is a parallel with only one branch, because: For an incomplete branch block (still running) you may not have an end block If there is not more than one current head node (current execution) you have no way to know it's inside a parallel This is best handled in your visitor by retaining nodes in case a parallel branch start is discovered (for incomplete flows at least). I can, however provide guarantees that the parallel starts are correctly handled (and they need to be), and am hardening tests for this case.
            Hide
            svanoort Sam Van Oort added a comment -

            Vivek Pandey This is awaiting your review (along with other fixes) in https://github.com/jenkinsci/workflow-api-plugin/pull/33

            Show
            svanoort Sam Van Oort added a comment - Vivek Pandey This is awaiting your review (along with other fixes) in https://github.com/jenkinsci/workflow-api-plugin/pull/33
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Sam Van Oort
            Path:
            src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScanner.java
            src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScannerTest.java
            http://jenkins-ci.org/commit/workflow-api-plugin/96e2c2ed99384a9c9f4fb80cc8f8876d9b225504
            Log:
            Fix a couple forkscanner testcase quirks, and put JENKINS-39839 to bed along with JENKINS-39841

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScanner.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScannerTest.java http://jenkins-ci.org/commit/workflow-api-plugin/96e2c2ed99384a9c9f4fb80cc8f8876d9b225504 Log: Fix a couple forkscanner testcase quirks, and put JENKINS-39839 to bed along with JENKINS-39841
            Hide
            svanoort Sam Van Oort added a comment -

            Fixed with explicit test coverage in workflow-api 2.12

            Show
            svanoort Sam Van Oort added a comment - Fixed with explicit test coverage in workflow-api 2.12

              People

              • Assignee:
                svanoort Sam Van Oort
                Reporter:
                vivek Vivek Pandey
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: