Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6545

JUnit report parsing cannot handle nested suites

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Component/s: junit-plugin
    • Labels:
      None

      Description

      JUnit allows to nest testsuites indefinetly (using TestSuite.addTestSuite()) which results in xml files that have <testsuites> element as root and under that a hierarchy of <testsuite> elements. Currently the SuiteResult class in Hudson can only handle a single level of nesting, which results in the "None of the test reports contained any result" error in the build.

      So my junit result looks like this:

      <?xml version="1.0" encoding="UTF-8"?>
      <testsuites name="Automated Tests" tests="186" errors="0" failures="0" ignored="0">
        <testsuite name="test.AutomatedTests" time="4.486">
          <testsuite name="test.fs.FileSystemTest" time="0.014">
            <testcase name="testPrefix" classname="test.fs.FileSystemTest" time="0.003"/>
          </testsuite>
          <testsuite name="test.db.DatabaseTest" time="0.014">
            <testcase name="testTable" classname="test.db.DatabaseTest" time="0.003"/>
          </testsuite>
        </testsuite>
      </testsuites
      

        Activity

        Hide
        ewysp ewysp added a comment -

        Try the xUnit plug in. I ran into the same issue as you, and found this plugin to suit our needs ... for now.

        http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin

        Show
        ewysp ewysp added a comment - Try the xUnit plug in. I ran into the same issue as you, and found this plugin to suit our needs ... for now. http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin
        Hide
        michaelbreu MichaelBreu added a comment -

        Hello ewvsg,

        It would be helpful if you could include here your XSLT-file needed for the xUnit-Workaround.

        Michael

        Show
        michaelbreu MichaelBreu added a comment - Hello ewvsg, It would be helpful if you could include here your XSLT-file needed for the xUnit-Workaround. Michael
        Hide
        russell_francis Russell Francis added a comment -

        Here is a description of the XSL needed to transform the nested junit xml output into something which can be parsed by Jenkins.

        http://stackoverflow.com/questions/4640300/hudson-ci-and-phpunit-none-of-the-test-reports-contained-any-result

        Show
        russell_francis Russell Francis added a comment - Here is a description of the XSL needed to transform the nested junit xml output into something which can be parsed by Jenkins. http://stackoverflow.com/questions/4640300/hudson-ci-and-phpunit-none-of-the-test-reports-contained-any-result
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        changelog.html
        core/src/main/java/hudson/tasks/junit/SuiteResult.java
        http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be
        Log:
        [FIXED JENKINS-6545] support nested test suites

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/tasks/junit/SuiteResult.java http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be Log: [FIXED JENKINS-6545] support nested test suites
        Hide
        dogfood dogfood added a comment -

        Integrated in jenkins_main_trunk #648
        [FIXED JENKINS-6545] support nested test suites

        Kohsuke Kawaguchi : 4129ac50a39ae2522c20ce73e2b10872a07c04be
        Files :

        • core/src/main/java/hudson/tasks/junit/SuiteResult.java
        • changelog.html
        Show
        dogfood dogfood added a comment - Integrated in jenkins_main_trunk #648 [FIXED JENKINS-6545] support nested test suites Kohsuke Kawaguchi : 4129ac50a39ae2522c20ce73e2b10872a07c04be Files : core/src/main/java/hudson/tasks/junit/SuiteResult.java changelog.html
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Stephen Connolly
        Path:
        test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
        http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b
        Log:
        Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545
        Hide
        dogfood dogfood added a comment -

        Integrated in jenkins_main_trunk #649
        Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

        Stephen Connolly : 97f1d02cbdc0aa85d2e907d878b126e80fd0880b
        Files :

        • test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
        Show
        dogfood dogfood added a comment - Integrated in jenkins_main_trunk #649 Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Stephen Connolly : 97f1d02cbdc0aa85d2e907d878b126e80fd0880b Files : test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
        Hide
        victorgarcia Victor Garcia Perez added a comment -

        This is my fix, at least you could mention my name in the changeset... since you copied it from the Hudson changeset i made few days ago

        https://github.com/hudson/hudson/commit/3d79ac89df3a3107904fa3b91b5d86075c86a5dd

        I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...

        Show
        victorgarcia Victor Garcia Perez added a comment - This is my fix, at least you could mention my name in the changeset... since you copied it from the Hudson changeset i made few days ago https://github.com/hudson/hudson/commit/3d79ac89df3a3107904fa3b91b5d86075c86a5dd I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...
        Hide
        kohsuke Kohsuke Kawaguchi added a comment -

        I'm sorry you felt that your code was taken without credit.

        You might not believe this, but I really did not copy your change. The pull request in Hudson caught my attention, so I looked at the bug, and the fix seemed simple enough, so I went ahead. I think you'd agree that it's hard to fix this bug in any other way.

        With that said, this is a sensitive problem, and I don't want to be accused of stealing the changes, so I've added your authorship in the copyright header to show my respect: https://github.com/jenkinsci/jenkins/commit/46994700e4d2b3a5ea84607035bb954ed3c4c750

        Show
        kohsuke Kohsuke Kawaguchi added a comment - I'm sorry you felt that your code was taken without credit. You might not believe this, but I really did not copy your change. The pull request in Hudson caught my attention, so I looked at the bug, and the fix seemed simple enough, so I went ahead. I think you'd agree that it's hard to fix this bug in any other way. With that said, this is a sensitive problem, and I don't want to be accused of stealing the changes, so I've added your authorship in the copyright header to show my respect: https://github.com/jenkinsci/jenkins/commit/46994700e4d2b3a5ea84607035bb954ed3c4c750
        Hide
        kohsuke Kohsuke Kawaguchi added a comment -
        I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...
        

        Forgot to reply to the last topic. I'm sorry that we failed to respond to your inquiry — I just checked the dev archive but couldn't find your post — did it actually get delivered? Can you tell me the subject?

        Currently, sending in a pull request is the best way to get an attention on patches. I hope this didn't discourage you from continuing to contribute to Jenkins.

        Show
        kohsuke Kohsuke Kawaguchi added a comment - I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list... Forgot to reply to the last topic. I'm sorry that we failed to respond to your inquiry — I just checked the dev archive but couldn't find your post — did it actually get delivered? Can you tell me the subject? Currently, sending in a pull request is the best way to get an attention on patches. I hope this didn't discourage you from continuing to contribute to Jenkins.
        Hide
        dogfood dogfood added a comment -

        Integrated in jenkins_main_trunk #734
        Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

        Kohsuke Kawaguchi : d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
        Files :

        • test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
        Show
        dogfood dogfood added a comment - Integrated in jenkins_main_trunk #734 Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Kohsuke Kawaguchi : d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Files : test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Christoph Kutzinski
        Path:
        core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
        core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
        http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
        Log:
        finally added a test for JENKINS-6545

        Compare: https://github.com/jenkinsci/jenkins/compare/3192dfb...073bd72

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545 Compare: https://github.com/jenkinsci/jenkins/compare/3192dfb...073bd72
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        changelog.html
        core/src/main/java/hudson/tasks/junit/SuiteResult.java
        http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be
        Log:
        [FIXED JENKINS-6545] support nested test suites

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/tasks/junit/SuiteResult.java http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be Log: [FIXED JENKINS-6545] support nested test suites
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Stephen Connolly
        Path:
        test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
        http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b
        Log:
        Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Stephen Connolly
        Path:
        test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
        http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
        Log:
        Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Christoph Kutzinski
        Path:
        core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
        core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
        http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
        Log:
        finally added a test for JENKINS-6545

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Stephen Connolly
        Path:
        test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
        http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
        Log:
        Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Christoph Kutzinski
        Path:
        core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
        core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
        http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
        Log:
        finally added a test for JENKINS-6545

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545
        Hide
        costescuandrei Andrei Costescu added a comment -

        I've been searching for a while for a tool that would generate this kind of nested <testsuite> report for JUnit.
        Did any of you actually have such a nested report xml generated from nested JUnit tests suites (either 3.x or 4.x)? Or it it only generated by other testing frameworks?

        For example ant <junit> xml formatter flattens test suites, doesn't keep nesting.
        Maven surefire does the same.
        Even tried TestNG wrapped around JUnit with the same result.
        (junit 4.11 (used 3 like style and 4 like style), testNG 6.8, ant 1.9.4, maven-surefire-plugin 2.17)

        So which is the test runner that would generate such a nested report for a nested JUnit suite?

        Show
        costescuandrei Andrei Costescu added a comment - I've been searching for a while for a tool that would generate this kind of nested <testsuite> report for JUnit. Did any of you actually have such a nested report xml generated from nested JUnit tests suites (either 3.x or 4.x)? Or it it only generated by other testing frameworks? For example ant <junit> xml formatter flattens test suites, doesn't keep nesting. Maven surefire does the same. Even tried TestNG wrapped around JUnit with the same result. (junit 4.11 (used 3 like style and 4 like style), testNG 6.8, ant 1.9.4, maven-surefire-plugin 2.17) So which is the test runner that would generate such a nested report for a nested JUnit suite?
        Hide
        costescuandrei Andrei Costescu added a comment - - edited

        Actually I found one but not very useful for Jenkins (the eclipse jUnit results view has an Export in a menu that will generate such xml).
        So I need one that can be used in Jenkins.

        Show
        costescuandrei Andrei Costescu added a comment - - edited Actually I found one but not very useful for Jenkins (the eclipse jUnit results view has an Export in a menu that will generate such xml). So I need one that can be used in Jenkins.

          People

          • Assignee:
            Unassigned
            Reporter:
            nukleon nukleon
          • Votes:
            3 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: