Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6545

JUnit report parsing cannot handle nested suites

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Component/s: junit-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      JUnit allows to nest testsuites indefinetly (using TestSuite.addTestSuite()) which results in xml files that have <testsuites> element as root and under that a hierarchy of <testsuite> elements. Currently the SuiteResult class in Hudson can only handle a single level of nesting, which results in the "None of the test reports contained any result" error in the build.

      So my junit result looks like this:

      <?xml version="1.0" encoding="UTF-8"?>
      <testsuites name="Automated Tests" tests="186" errors="0" failures="0" ignored="0">
        <testsuite name="test.AutomatedTests" time="4.486">
          <testsuite name="test.fs.FileSystemTest" time="0.014">
            <testcase name="testPrefix" classname="test.fs.FileSystemTest" time="0.003"/>
          </testsuite>
          <testsuite name="test.db.DatabaseTest" time="0.014">
            <testcase name="testTable" classname="test.db.DatabaseTest" time="0.003"/>
          </testsuite>
        </testsuite>
      </testsuites
      

        Attachments

          Activity

          Hide
          ewysp ewysp added a comment -

          Try the xUnit plug in. I ran into the same issue as you, and found this plugin to suit our needs ... for now.

          http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin

          Show
          ewysp ewysp added a comment - Try the xUnit plug in. I ran into the same issue as you, and found this plugin to suit our needs ... for now. http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin
          Hide
          michaelbreu MichaelBreu added a comment -

          Hello ewvsg,

          It would be helpful if you could include here your XSLT-file needed for the xUnit-Workaround.

          Michael

          Show
          michaelbreu MichaelBreu added a comment - Hello ewvsg, It would be helpful if you could include here your XSLT-file needed for the xUnit-Workaround. Michael
          Hide
          russell_francis Russell Francis added a comment -

          Here is a description of the XSL needed to transform the nested junit xml output into something which can be parsed by Jenkins.

          http://stackoverflow.com/questions/4640300/hudson-ci-and-phpunit-none-of-the-test-reports-contained-any-result

          Show
          russell_francis Russell Francis added a comment - Here is a description of the XSL needed to transform the nested junit xml output into something which can be parsed by Jenkins. http://stackoverflow.com/questions/4640300/hudson-ci-and-phpunit-none-of-the-test-reports-contained-any-result
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/junit/SuiteResult.java
          http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be
          Log:
          [FIXED JENKINS-6545] support nested test suites

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/tasks/junit/SuiteResult.java http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be Log: [FIXED JENKINS-6545] support nested test suites
          Hide
          dogfood dogfood added a comment -

          Integrated in jenkins_main_trunk #648
          [FIXED JENKINS-6545] support nested test suites

          Kohsuke Kawaguchi : 4129ac50a39ae2522c20ce73e2b10872a07c04be
          Files :

          • core/src/main/java/hudson/tasks/junit/SuiteResult.java
          • changelog.html
          Show
          dogfood dogfood added a comment - Integrated in jenkins_main_trunk #648 [FIXED JENKINS-6545] support nested test suites Kohsuke Kawaguchi : 4129ac50a39ae2522c20ce73e2b10872a07c04be Files : core/src/main/java/hudson/tasks/junit/SuiteResult.java changelog.html
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545
          Hide
          dogfood dogfood added a comment -

          Integrated in jenkins_main_trunk #649
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Stephen Connolly : 97f1d02cbdc0aa85d2e907d878b126e80fd0880b
          Files :

          • test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          Show
          dogfood dogfood added a comment - Integrated in jenkins_main_trunk #649 Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Stephen Connolly : 97f1d02cbdc0aa85d2e907d878b126e80fd0880b Files : test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          Hide
          victorgarcia Victor Garcia Perez added a comment -

          This is my fix, at least you could mention my name in the changeset... since you copied it from the Hudson changeset i made few days ago

          https://github.com/hudson/hudson/commit/3d79ac89df3a3107904fa3b91b5d86075c86a5dd

          I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...

          Show
          victorgarcia Victor Garcia Perez added a comment - This is my fix, at least you could mention my name in the changeset... since you copied it from the Hudson changeset i made few days ago https://github.com/hudson/hudson/commit/3d79ac89df3a3107904fa3b91b5d86075c86a5dd I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          I'm sorry you felt that your code was taken without credit.

          You might not believe this, but I really did not copy your change. The pull request in Hudson caught my attention, so I looked at the bug, and the fix seemed simple enough, so I went ahead. I think you'd agree that it's hard to fix this bug in any other way.

          With that said, this is a sensitive problem, and I don't want to be accused of stealing the changes, so I've added your authorship in the copyright header to show my respect: https://github.com/jenkinsci/jenkins/commit/46994700e4d2b3a5ea84607035bb954ed3c4c750

          Show
          kohsuke Kohsuke Kawaguchi added a comment - I'm sorry you felt that your code was taken without credit. You might not believe this, but I really did not copy your change. The pull request in Hudson caught my attention, so I looked at the bug, and the fix seemed simple enough, so I went ahead. I think you'd agree that it's hard to fix this bug in any other way. With that said, this is a sensitive problem, and I don't want to be accused of stealing the changes, so I've added your authorship in the copyright header to show my respect: https://github.com/jenkinsci/jenkins/commit/46994700e4d2b3a5ea84607035bb954ed3c4c750
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -
          I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...
          

          Forgot to reply to the last topic. I'm sorry that we failed to respond to your inquiry — I just checked the dev archive but couldn't find your post — did it actually get delivered? Can you tell me the subject?

          Currently, sending in a pull request is the best way to get an attention on patches. I hope this didn't discourage you from continuing to contribute to Jenkins.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list... Forgot to reply to the last topic. I'm sorry that we failed to respond to your inquiry — I just checked the dev archive but couldn't find your post — did it actually get delivered? Can you tell me the subject? Currently, sending in a pull request is the best way to get an attention on patches. I hope this didn't discourage you from continuing to contribute to Jenkins.
          Hide
          dogfood dogfood added a comment -

          Integrated in jenkins_main_trunk #734
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Kohsuke Kawaguchi : d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
          Files :

          • test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          Show
          dogfood dogfood added a comment - Integrated in jenkins_main_trunk #734 Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Kohsuke Kawaguchi : d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Files : test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
          core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
          http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
          Log:
          finally added a test for JENKINS-6545

          Compare: https://github.com/jenkinsci/jenkins/compare/3192dfb...073bd72

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545 Compare: https://github.com/jenkinsci/jenkins/compare/3192dfb...073bd72
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/junit/SuiteResult.java
          http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be
          Log:
          [FIXED JENKINS-6545] support nested test suites

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/tasks/junit/SuiteResult.java http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be Log: [FIXED JENKINS-6545] support nested test suites
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
          core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
          http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
          Log:
          finally added a test for JENKINS-6545

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
          core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
          http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
          Log:
          finally added a test for JENKINS-6545

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545
          Hide
          costescuandrei Andrei Costescu added a comment -

          I've been searching for a while for a tool that would generate this kind of nested <testsuite> report for JUnit.
          Did any of you actually have such a nested report xml generated from nested JUnit tests suites (either 3.x or 4.x)? Or it it only generated by other testing frameworks?

          For example ant <junit> xml formatter flattens test suites, doesn't keep nesting.
          Maven surefire does the same.
          Even tried TestNG wrapped around JUnit with the same result.
          (junit 4.11 (used 3 like style and 4 like style), testNG 6.8, ant 1.9.4, maven-surefire-plugin 2.17)

          So which is the test runner that would generate such a nested report for a nested JUnit suite?

          Show
          costescuandrei Andrei Costescu added a comment - I've been searching for a while for a tool that would generate this kind of nested <testsuite> report for JUnit. Did any of you actually have such a nested report xml generated from nested JUnit tests suites (either 3.x or 4.x)? Or it it only generated by other testing frameworks? For example ant <junit> xml formatter flattens test suites, doesn't keep nesting. Maven surefire does the same. Even tried TestNG wrapped around JUnit with the same result. (junit 4.11 (used 3 like style and 4 like style), testNG 6.8, ant 1.9.4, maven-surefire-plugin 2.17) So which is the test runner that would generate such a nested report for a nested JUnit suite?
          Hide
          costescuandrei Andrei Costescu added a comment - - edited

          Actually I found one but not very useful for Jenkins (the eclipse jUnit results view has an Export in a menu that will generate such xml).
          So I need one that can be used in Jenkins.

          Show
          costescuandrei Andrei Costescu added a comment - - edited Actually I found one but not very useful for Jenkins (the eclipse jUnit results view has an Export in a menu that will generate such xml). So I need one that can be used in Jenkins.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins-test-harness/c9f1e5f27a87a26ff7e6b86e0adb1d6f77dfc948
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Originally-Committed-As: 97f1d02cbdc0aa85d2e907d878b126e80fd0880b

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins-test-harness/c9f1e5f27a87a26ff7e6b86e0adb1d6f77dfc948 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Originally-Committed-As: 97f1d02cbdc0aa85d2e907d878b126e80fd0880b
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins-test-harness/fd2c85b6aeb64003677d5faa6c2cdfc32c4f6a14
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Originally-Committed-As: d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins-test-harness/fd2c85b6aeb64003677d5faa6c2cdfc32c4f6a14 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Originally-Committed-As: d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9

            People

            • Assignee:
              Unassigned
              Reporter:
              nukleon nukleon
            • Votes:
              3 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: