Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53236

junit.xml testsuite element shows incorrect number of failures when you run tests with Jenkins parallel jobs

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Blocker Blocker
    • junit-plugin
    • None

      It seems like the "testsuite" element in my junit.xml file that Jenkins aggregated is not correct.

      Somehow, the "Test Result" page I see in Jenkins shows me the correct numbers - good job with that! However, when I examine the junit.xml file, the errors, failures, skips, and tests elements do not match what I see in Jenkins.

      I use Jenkins parallel jobs to run my tests because I have thousands of python unit tests that are run with py.test. I use py.test's junitxml output file which I later ingest into jenkins.

      My projects has over 5,000 tests. In a recent run, the junit file suggests there were 0 failures and 1 skips which is wrong....

      <?xml version="1.0" encoding="utf-8"?><testsuite errors="0" failures="0" name="pytest" skips="1" tests="33".... 

      However, the Jenkins rendering engine was able to correctly deduce that there really are 11 failures at 525 skipped tests as you can see in my attached screenshot.

      I think the bug here is that as each parallel shard is ingested by the jenkins master, that the errors, failures, skips, and tests count needs to be updated.

            davidvanlaatum David van Laatum
            piratejohnny Jon B
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: