Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37598

Inaccurate aggregation of multiple xmls containing case information about the same testsuite

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: junit-plugin
    • Labels:
      None
    • Environment:
      junit plugin 1.18
    • Similar Issues:

      Description

      When the same testsuite appears in multiple result xmls; the current aggregation logic fails to aggregate neither the sum of the testcases - and neither the total testclass runtime - instead it reports only 1 testclass'es runtime.

      example xml:

      <?xml version="1.0" encoding="UTF-8"?>
      <testsuites name="Automation Tests" tests="2" errors="0" failures="0" ignored="0">
          <testsuite name="test.fs.FileSystemTests" time="50">
            <testcase name="testPrefix1" classname="test.fs.FileSystemTest1" time="10"/>
          </testsuite>
          <testsuite name="test.fs.FileSystemTests" time="50">
            <testcase name="testPrefix2" classname="test.fs.FileSystemTest1" time="10"/>
          </testsuite>
      </testsuites>
      

      in the above example the runtime of test.fs.FileSystemTests should be 100

      Encountered during hive / maven 2.19.1 upgrade..details:
      https://issues.apache.org/jira/browse/HIVE-14572

        Attachments

          Issue Links

            Activity

            Hide
            kirk Zoltán Haindrich added a comment -

            I'm not sure which stats should I leave this ticket in...i've looked into a few...and haven't seen any in IN REVIEW
            I think the code in the pull request should take care of the issue

            Show
            kirk Zoltán Haindrich added a comment - I'm not sure which stats should I leave this ticket in...i've looked into a few...and haven't seen any in IN REVIEW I think the code in the pull request should take care of the issue
            Hide
            kirk Zoltán Haindrich added a comment -

            Jesse Glick can you take a look at the update changes?

            Show
            kirk Zoltán Haindrich added a comment - Jesse Glick can you take a look at the update changes?
            Hide
            kirk Zoltán Haindrich added a comment -

            Oleg Nenashev, Jesse Glick Can you please take a look at the changes...it would be great to have this fixed:

            I would like to propose a new way to run tests in parallel which would not need any effort on the user side - by extending the parallel test executor plugin with a different approach (which would not be prone to lose testcases)....this problem would arise in that scenario as well because that will possibly "split" a junit4 testcase in 2 or more pieces...which would be affected by the same timing inconsistency issues which this issue solves.

            Show
            kirk Zoltán Haindrich added a comment - Oleg Nenashev , Jesse Glick Can you please take a look at the changes...it would be great to have this fixed: I would like to propose a new way to run tests in parallel which would not need any effort on the user side - by extending the parallel test executor plugin with a different approach (which would not be prone to lose testcases)....this problem would arise in that scenario as well because that will possibly "split" a junit4 testcase in 2 or more pieces...which would be affected by the same timing inconsistency issues which this issue solves.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Haindrich Zoltán (kirk)
            Path:
            src/main/java/hudson/tasks/junit/ClassResult.java
            src/main/java/hudson/tasks/junit/SuiteResult.java
            src/main/java/hudson/tasks/junit/TestResult.java
            src/test/java/hudson/tasks/junit/TestResultTest.java
            src/test/resources/hudson/tasks/junit/junit-report-time-aggregation-invalid.xml
            src/test/resources/hudson/tasks/junit/junit-report-time-aggregation.xml
            src/test/resources/hudson/tasks/junit/junit-report-time-aggregation2.xml
            http://jenkins-ci.org/commit/junit-plugin/8f3473b215130ea3e537ca41a97df9a5572d1c2d
            Log:
            JENKINS-37598: Inaccurate aggregation of multiple xmls containing case information about the same testsuite

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Haindrich Zoltán (kirk) Path: src/main/java/hudson/tasks/junit/ClassResult.java src/main/java/hudson/tasks/junit/SuiteResult.java src/main/java/hudson/tasks/junit/TestResult.java src/test/java/hudson/tasks/junit/TestResultTest.java src/test/resources/hudson/tasks/junit/junit-report-time-aggregation-invalid.xml src/test/resources/hudson/tasks/junit/junit-report-time-aggregation.xml src/test/resources/hudson/tasks/junit/junit-report-time-aggregation2.xml http://jenkins-ci.org/commit/junit-plugin/8f3473b215130ea3e537ca41a97df9a5572d1c2d Log: JENKINS-37598 : Inaccurate aggregation of multiple xmls containing case information about the same testsuite
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Zoltan Haindrich
            Path:
            src/main/java/hudson/tasks/junit/SuiteResult.java
            src/test/java/hudson/tasks/junit/TestResultTest.java
            src/test/resources/hudson/tasks/junit/junit-report-time-aggregation-invalid.xml
            http://jenkins-ci.org/commit/junit-plugin/319280f67394b2b0eda1fba589912f30bbd3ebcc
            Log:
            JENKINS-37598: address review comments; put back time field, doesn't throw exception

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Zoltan Haindrich Path: src/main/java/hudson/tasks/junit/SuiteResult.java src/test/java/hudson/tasks/junit/TestResultTest.java src/test/resources/hudson/tasks/junit/junit-report-time-aggregation-invalid.xml http://jenkins-ci.org/commit/junit-plugin/319280f67394b2b0eda1fba589912f30bbd3ebcc Log: JENKINS-37598 : address review comments; put back time field, doesn't throw exception
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oliver Gondža
            Path:
            src/main/java/hudson/tasks/junit/ClassResult.java
            src/main/java/hudson/tasks/junit/SuiteResult.java
            src/main/java/hudson/tasks/junit/TestResult.java
            src/test/java/hudson/tasks/junit/TestResultTest.java
            src/test/resources/hudson/tasks/junit/junit-report-time-aggregation.xml
            src/test/resources/hudson/tasks/junit/junit-report-time-aggregation2.xml
            http://jenkins-ci.org/commit/junit-plugin/d083744e64c6f1a084bc032f9ef8f8a9e45cf38e
            Log:
            Merge pull request #54 from kgyrtkirk/JENKINS-37598

            JENKINS-37598 Inaccurate aggregation of multiple xmls containing case information about the same testsuite

            Compare: https://github.com/jenkinsci/junit-plugin/compare/6243fa595ab3...d083744e64c6

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oliver Gondža Path: src/main/java/hudson/tasks/junit/ClassResult.java src/main/java/hudson/tasks/junit/SuiteResult.java src/main/java/hudson/tasks/junit/TestResult.java src/test/java/hudson/tasks/junit/TestResultTest.java src/test/resources/hudson/tasks/junit/junit-report-time-aggregation.xml src/test/resources/hudson/tasks/junit/junit-report-time-aggregation2.xml http://jenkins-ci.org/commit/junit-plugin/d083744e64c6f1a084bc032f9ef8f8a9e45cf38e Log: Merge pull request #54 from kgyrtkirk/ JENKINS-37598 JENKINS-37598 Inaccurate aggregation of multiple xmls containing case information about the same testsuite Compare: https://github.com/jenkinsci/junit-plugin/compare/6243fa595ab3...d083744e64c6
            Hide
            kirk Zoltán Haindrich added a comment -

            merged; will be fixed in >=1.22

            Show
            kirk Zoltán Haindrich added a comment - merged; will be fixed in >=1.22

              People

              • Assignee:
                kirk Zoltán Haindrich
                Reporter:
                kirk Zoltán Haindrich
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: