Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37573

Create way for pipeline graph analysis to return full nested structures

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      As a consumer of pipeline graph analysis it would be helpful to have a way to get the full block chunk structure (with nesting). This is useful for cases where we need to find the properties of a FlowNode based on block contents (example its label, or the node used to run it).

      In cases where we need scope, we may have nested blocks with following nodes belonging to the enclosing parent block.

      This needs:

      • A SimpleChunkVisitor implementation using an ArrayDeque to store the context (open and nested chunks). If finished with a current chunk (popping out of the scope & removing from ArrayDeque), it is added as a nested chunk in the parent scope, when hitting the ChunkStart
      • A NestableMemoryFlowChunk that can have other chunks added within it or contain lists of nodes nodes.
      • NestableMemoryFlowChunk needs something like "addChunk(NestableMemoryFlowChunk)" and "addNode(FlowNode)".
      • Two container types, RecursiveNestableMemoryFlowChunk has a list of FlowChunk, ContainerNestableMemoryFlowChunk has an ArrayList of flownodes, and is converted to recursive when you add the first nested chunk
      • I've described this as the DOMLike model – this is because you might provide tree-traversal APIs
      • AdvancedVisitor is one way to collect these data models (and couple to datastructures for custom storage). it would invoke a factory to create chunks implementing these interfaces and store data.

        Attachments

          Activity

          Hide
          abayer Andrew Bayer added a comment -

          Sam Van Oort - this is related to the stuff you're working on for my junit work, right?

          Show
          abayer Andrew Bayer added a comment - Sam Van Oort - this is related to the stuff you're working on for my junit work, right?
          Hide
          svanoort Sam Van Oort added a comment -

          Andrew Bayer kinda – I'll link it in the PR but it's intended to be more full-featured.

          Show
          svanoort Sam Van Oort added a comment - Andrew Bayer kinda – I'll link it in the PR but it's intended to be more full-featured.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java
          src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java
          http://jenkins-ci.org/commit/workflow-api-plugin/4ba6b42b651d9a5853a32681e9582f9649ad8fa2
          Log:
          Merge pull request #50 from svanoort/jenkins-27395-block-structure-lookup

          JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph

          Compare: https://github.com/jenkinsci/workflow-api-plugin/compare/63e8ad0c2715...4ba6b42b651d

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java http://jenkins-ci.org/commit/workflow-api-plugin/4ba6b42b651d9a5853a32681e9582f9649ad8fa2 Log: Merge pull request #50 from svanoort/jenkins-27395-block-structure-lookup JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph Compare: https://github.com/jenkinsci/workflow-api-plugin/compare/63e8ad0c2715...4ba6b42b651d
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java
          src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java
          http://jenkins-ci.org/commit/workflow-api-plugin/70a05dc4d4a330b797ce370fd587fccfe74355d9
          Log:
          Revert "Revert "JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph""

          This reverts commit 88ffdfc69c43bd4dde21a6578b5ac466999b4fd4.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java http://jenkins-ci.org/commit/workflow-api-plugin/70a05dc4d4a330b797ce370fd587fccfe74355d9 Log: Revert "Revert " JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"" This reverts commit 88ffdfc69c43bd4dde21a6578b5ac466999b4fd4.
          Hide
          svanoort Sam Van Oort added a comment -

          Closing this out because we have a bit chunk of the tree traversal APIs available now (at least getting parents – getting children can be done via the AbstractFlowScanner classes using bounds set from the previous). 

          Show
          svanoort Sam Van Oort added a comment - Closing this out because we have a bit chunk of the tree traversal APIs available now (at least getting parents – getting children can be done via the AbstractFlowScanner classes using bounds set from the previous). 
          Hide
          svanoort Sam Van Oort added a comment - - edited

          Close enough, or at least many of the useful bits – workflow-api 2.22

          Show
          svanoort Sam Van Oort added a comment - - edited Close enough, or at least many of the useful bits – workflow-api 2.22
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java
          src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java
          http://jenkins-ci.org/commit/workflow-cps-plugin/88ffdfc69c43bd4dde21a6578b5ac466999b4fd4
          Log:
          Revert "JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/88ffdfc69c43bd4dde21a6578b5ac466999b4fd4 Log: Revert " JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java
          src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java
          http://jenkins-ci.org/commit/workflow-cps-plugin/c0daeb5ce9ba55e6f51cb6c8db903cc5fbba324b
          Log:
          Merge pull request #52 from jenkinsci/revert-50-jenkins-27395-block-structure-lookup

          Revert "JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/c0daeb5ce9ba55e6f51cb6c8db903cc5fbba324b Log: Merge pull request #52 from jenkinsci/revert-50-jenkins-27395-block-structure-lookup Revert " JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"

            People

            • Assignee:
              svanoort Sam Van Oort
              Reporter:
              svanoort Sam Van Oort
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: