Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30551

Make Warnings Plugin compatible with parallel Workflow

    Details

    • Similar Issues:
    • Released As:
      5.0.0-beta2

      Description

      29959 made it compatible with a single build.
      But it doesn't yet work nicely for multiple parallel builds.

      If one puts the WarningsPublisher inside each parallel task, then the build page contains a link to the warning output once per task. But each link points to the same page and only contains the warnings of one task.

      If instead one runs the WarningsPublisher step only once after all all parallel tasks, then the links to the source code do not work (because the workspace path is different on the different node) and the list of warnings doesn't show for which task/node the warning occurred.

      Based on how other plugins (e.g. junit) work, I assume the first is how it should work. To it work, the warnings plugin should summarize the warnings before reporting them.

      To make the 2nd work it would need to parse the task label in the console output (to display the task) and resolve the workspace relative to the node of that task.

      Is anyone working on this? Any suggestions of how to get started on fixing this?

        Attachments

          Issue Links

            Activity

            Hide
            drulli Ulli Hafner added a comment -

            I think nobody is working on this issue. Are you are interested in providing a pull request? I don't use the workflow-plugin on my own but may get give you some hints on how to implement it here...

            Show
            drulli Ulli Hafner added a comment - I think nobody is working on this issue. Are you are interested in providing a pull request? I don't use the workflow-plugin on my own but may get give you some hints on how to implement it here...
            Hide
            tmurtola Teemu Murtola added a comment -

            I think there are two slightly separate issues here:

            1. If the warning plugin is run more than once in a workflow, it should merge the results into a single report, producing results similar to what it does for a matrix/multiconfiguration build, or similar to what the JUnit plugin does in a workflow.
            2. It should be possible to run the warnings plugin only on a subset of the console output in a workflow job. For the original case where this is reported from, the desire would be to scan the warnings from the console output of each of the parallel branches separately. Currently, the warnings plugin just scans the whole console log up to the point where it is invoked, which means that each invocation will see all the warnings from all the branches. To make this work, changes to the workflow plugin can be required.
            Show
            tmurtola Teemu Murtola added a comment - I think there are two slightly separate issues here: If the warning plugin is run more than once in a workflow, it should merge the results into a single report, producing results similar to what it does for a matrix/multiconfiguration build, or similar to what the JUnit plugin does in a workflow. It should be possible to run the warnings plugin only on a subset of the console output in a workflow job. For the original case where this is reported from, the desire would be to scan the warnings from the console output of each of the parallel branches separately. Currently, the warnings plugin just scans the whole console log up to the point where it is invoked, which means that each invocation will see all the warnings from all the branches. To make this work, changes to the workflow plugin can be required.
            Hide
            drulli Ulli Hafner added a comment - - edited

            It would be helpful, if someone can provide an ATH test case for such a build. I never used parallel pipelines up to now.

            Show
            drulli Ulli Hafner added a comment - - edited It would be helpful, if someone can provide an ATH test case for such a build. I never used parallel pipelines up to now.
            Hide
            drulli Ulli Hafner added a comment -
            Show
            drulli Ulli Hafner added a comment - I found some documentation about doing this: https://github.com/jenkinsci/workflow-step-api-plugin/blob/master/README.md .
            Hide
            jekeller Jacob Keller added a comment -

            I solved this part of the problem by using separate files for each parallel step, so that they didn't need to scan the entire console output but only one file.

            Show
            jekeller Jacob Keller added a comment - I solved this part of the problem by using separate files for each parallel step, so that they didn't need to scan the entire console output but only one file.
            Hide
            drulli Ulli Hafner added a comment -

            I'm trying to consolidate the requirements for the static analysis suite in pipeline jobs in a wiki page. Can you please read it carefully and comment or change it accordingly?

            Show
            drulli Ulli Hafner added a comment - I'm trying to consolidate the requirements for the static analysis suite in pipeline jobs in a wiki page . Can you please read it carefully and comment or change it accordingly?
            Hide
            jekeller Jacob Keller added a comment -

            Overall that looks good. I think the main issue is that it might be broken down into too many steps. I think some of the steps can be done without making an explicit step (setting build status and health info, can probably be done directly?)

            I'm not sure if we can easily combined any of those steps, or maybe provide some common variations that can help avoid too much confusion. Overall I think the initial steps for computing, filtering, and such all make enough sense.

            I left some comments on the wiki page itself also.

            Show
            jekeller Jacob Keller added a comment - Overall that looks good. I think the main issue is that it might be broken down into too many steps. I think some of the steps can be done without making an explicit step (setting build status and health info, can probably be done directly?) I'm not sure if we can easily combined any of those steps, or maybe provide some common variations that can help avoid too much confusion. Overall I think the initial steps for computing, filtering, and such all make enough sense. I left some comments on the wiki page itself also.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Ulli Hafner
            Path:
            src/main/java/io/jenkins/plugins/analysis/core/views/ResultAction.java
            src/test/java/io/jenkins/plugins/analysis/core/views/ResultActionTest.java
            http://jenkins-ci.org/commit/analysis-core-plugin/17c4d9f1e018b25814da27b75994dcb8c31fe2b8
            Log:
            JENKINS-30551 Make ResultAction serializable.

            *NOTE:* This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

            Functionality will be removed from GitHub.com on January 31st, 2019.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ulli Hafner Path: src/main/java/io/jenkins/plugins/analysis/core/views/ResultAction.java src/test/java/io/jenkins/plugins/analysis/core/views/ResultActionTest.java http://jenkins-ci.org/commit/analysis-core-plugin/17c4d9f1e018b25814da27b75994dcb8c31fe2b8 Log: JENKINS-30551 Make ResultAction serializable. * NOTE: * This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Ulli Hafner
            Path:
            src/test/java/io/jenkins/plugins/analysis/core/testutil/IntegrationTest.java
            http://jenkins-ci.org/commit/analysis-core-plugin/59d761b521ff2eb229e828215750a9d9e562c475
            Log:
            JENKINS-30551 Added a test case.

            *NOTE:* This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

            Functionality will be removed from GitHub.com on January 31st, 2019.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ulli Hafner Path: src/test/java/io/jenkins/plugins/analysis/core/testutil/IntegrationTest.java http://jenkins-ci.org/commit/analysis-core-plugin/59d761b521ff2eb229e828215750a9d9e562c475 Log: JENKINS-30551 Added a test case. * NOTE: * This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Ulli Hafner
            Path:
            src/test/java/io/jenkins/plugins/analysis/warnings/FreestyleJobITest.java
            src/test/java/io/jenkins/plugins/analysis/warnings/IssuesRecorderITest.java
            src/test/java/io/jenkins/plugins/analysis/warnings/PipelineITest.java
            src/test/java/io/jenkins/plugins/analysis/warnings/StepsITest.java
            src/test/resources/io/jenkins/plugins/analysis/warnings/parallel.jenkinsfile
            http://jenkins-ci.org/commit/warnings-plugin/0315547bc1a3084c0ef34a9153a7ac2d81ac87a4
            Log:
            JENKINS-30551 Added a test case.

            *NOTE:* This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

            Functionality will be removed from GitHub.com on January 31st, 2019.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ulli Hafner Path: src/test/java/io/jenkins/plugins/analysis/warnings/FreestyleJobITest.java src/test/java/io/jenkins/plugins/analysis/warnings/IssuesRecorderITest.java src/test/java/io/jenkins/plugins/analysis/warnings/PipelineITest.java src/test/java/io/jenkins/plugins/analysis/warnings/StepsITest.java src/test/resources/io/jenkins/plugins/analysis/warnings/parallel.jenkinsfile http://jenkins-ci.org/commit/warnings-plugin/0315547bc1a3084c0ef34a9153a7ac2d81ac87a4 Log: JENKINS-30551 Added a test case. * NOTE: * This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.
            Hide
            drulli Ulli Hafner added a comment -

            What still needs to be checked:

            • If a build is done in parallel, then the log files might be distributed on different machines. Where is the scanner invoked, how can the scanner access the different log files.
            • See JENKINS-45774
            Show
            drulli Ulli Hafner added a comment - What still needs to be checked: If a build is done in parallel, then the log files might be distributed on different machines. Where is the scanner invoked, how can the scanner access the different log files. See JENKINS-45774
            Hide
            drulli Ulli Hafner added a comment -

            Released in 5.0.0-beta2.

            Show
            drulli Ulli Hafner added a comment - Released in 5.0.0-beta2.

              People

              • Assignee:
                drulli Ulli Hafner
                Reporter:
                rschulz Roland Schulz
              • Votes:
                4 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: