Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59659

XUnit plugin result displays confused by aggregation of parallel builds

    Details

    • Similar Issues:

      Description

      So we use pipeline to build several platforms of our codebase, and prod/nonprod etc.  They all produce a unit test report, and the paths to the result files all contain the specific parallel branch they were built in.

      I was hoping this would be enough for it to properly identify the tests, but it seems that unless the tests have the build specific info in the test name, the graphing and result display of the tests gets very confused and will combine some tests based on race conditions which leads to a very spiky looking graph.

      I confirmed this by using the jest config in a node project we build the same way.  Jest lets you set the pattern used to build the suite name and test name. Once I set this to include the build info, the graph flattened out and remained consistent.

      In other projects we use other frameworks that you support googletest, etc.  I don't know if each of them support the option to configure the test suite/name patterns...     Could you please look at how the plugin could properly handle parallel builds for test display and graphing?

        Attachments

          Issue Links

            Activity

            Hide
            nfalco Nikolas Falco added a comment -

            I have some questions:

            • Do you have a jenkins job per branch (multibranch pipeline) or one job that build a specific branch?
            • Is the workspace folder shred among all build? Is the workspace wipe out?
            • What do you mean with parallel? Do you have a single job that where you have parallelized the test stage (which one that produce reports) or you have one job but parallel executor of the same job in the same workspace?
            Show
            nfalco Nikolas Falco added a comment - I have some questions: Do you have a jenkins job per branch (multibranch pipeline) or one job that build a specific branch? Is the workspace folder shred among all build? Is the workspace wipe out? What do you mean with parallel? Do you have a single job that where you have parallelized the test stage (which one that produce reports) or you have one job but parallel executor of the same job in the same workspace?
            Hide
            nfalco Nikolas Falco added a comment - - edited

            Any news? If not I will close this issue.
            In pipeline there is the association with the stage. This means you can run parallel step and process reports. Each report is associated with a specific stage. Path of the reports are not taken into account.

            In non pipeline environment we can not support stage association. This means that tests name have to be unique across parallel execution.

            Show
            nfalco Nikolas Falco added a comment - - edited Any news? If not I will close this issue. In pipeline there is the association with the stage. This means you can run parallel step and process reports. Each report is associated with a specific stage. Path of the reports are not taken into account. In non pipeline environment we can not support stage association. This means that tests name have to be unique across parallel execution.
            Hide
            nfalco Nikolas Falco added a comment -

            Path of report file could not be used to distinguish test. Perfom a xunit step in a different atep to differentiate them as explained in the improvement of linked issue.

            Show
            nfalco Nikolas Falco added a comment - Path of report file could not be used to distinguish test. Perfom a xunit step in a different atep to differentiate them as explained in the improvement of linked issue.

              People

              • Assignee:
                nfalco Nikolas Falco
                Reporter:
                ddumont Dan Dumont
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: