Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Component/s: junit-plugin
    • Labels:
      None
    • Environment:
      Platform: All, OS: All
    • Similar Issues:
      Show 5 results

      Description

      Ignored tests are not part of the reporting.
      The number of ignored tests is not counted and not part of the trendgraph.
      Also it is not available in the overview table with total and failed tests.

      This would be very useful to have these as well.
      We sometimes put tests on Ingore if some functionality is under construction.
      Ignored tests are not counted as executed tests.

        Attachments

          Activity

          cbos Cees Bos created issue -
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          I suppose this is a new JUnit 4.0 functionality.

          Do you have any XML report file that contains ignored tests? If you do, please
          attach it here so that we can improve Hudson to handle it accordingly.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - I suppose this is a new JUnit 4.0 functionality. Do you have any XML report file that contains ignored tests? If you do, please attach it here so that we can improve Hudson to handle it accordingly.
          Hide
          wayfarer3130 wayfarer3130 added a comment -

          Created an attachment (id=516)
          Example JUnit with several ignored tests

          Show
          wayfarer3130 wayfarer3130 added a comment - Created an attachment (id=516) Example JUnit with several ignored tests
          Hide
          basil3whitehouse basil3whitehouse added a comment -

          This seems to be partially implemented, at least in version 1.288+. I see the
          number of skipped tests in the bar summary, and at test class detail level.
          Just seems to be missing from the package level, probably need another column there.

          I'll attach some screenshots. BTW this was generated with JUnit 4 tests using
          Maven surefire. The xml file is similar to what's already attached.

          Show
          basil3whitehouse basil3whitehouse added a comment - This seems to be partially implemented, at least in version 1.288+. I see the number of skipped tests in the bar summary, and at test class detail level. Just seems to be missing from the package level, probably need another column there. I'll attach some screenshots. BTW this was generated with JUnit 4 tests using Maven surefire. The xml file is similar to what's already attached.
          Hide
          basil3whitehouse basil3whitehouse added a comment -

          Created an attachment (id=721)
          Screen cap of test result bar showing skipped and passed tests.

          Show
          basil3whitehouse basil3whitehouse added a comment - Created an attachment (id=721) Screen cap of test result bar showing skipped and passed tests.
          Hide
          basil3whitehouse basil3whitehouse added a comment -

          Created an attachment (id=722)
          Screen cap of test class detail showing skipped and passed tests.

          Show
          basil3whitehouse basil3whitehouse added a comment - Created an attachment (id=722) Screen cap of test class detail showing skipped and passed tests.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : huybrechts
          Path:
          branches/junit/main/core/src/main/resources/hudson/tasks/junit/MetaTabulatedResult/body.jelly
          http://fisheye4.cenqua.com/changelog/hudson/?cs=20538
          Log:
          fix for JENKINS-1820 – extra column displaying skipped tests

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : huybrechts Path: branches/junit/main/core/src/main/resources/hudson/tasks/junit/MetaTabulatedResult/body.jelly http://fisheye4.cenqua.com/changelog/hudson/?cs=20538 Log: fix for JENKINS-1820 – extra column displaying skipped tests
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : huybrechts
          Path:
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/CaseResult.java
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/ClassResult.java
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/PackageResult.java
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/SuiteResult.java
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TabulatedResult.java
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TestObject.java
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TestResult.java
          trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TestResultAction.java
          trunk/hudson/main/core/src/main/resources/hudson/model/AbstractBuild/sidepanel.jelly
          trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/CaseResult/index.jelly
          trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/ClassResult/body.jelly
          trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly
          trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/MetaTabulatedResult/body.jelly
          trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/TabulatedResult/index.jelly
          trunk/hudson/main/core/src/main/resources/hudson/tasks/test/MatrixTestResult/index.jelly
          trunk/hudson/plugins/claim/pom.xml
          trunk/hudson/plugins/claim/src/main/java/hudson/plugins/claim/ClaimBuildAction.java
          trunk/hudson/plugins/claim/src/main/java/hudson/plugins/claim/ClaimPublisher.java
          trunk/hudson/plugins/claim/src/main/webapp/help-sticky.html
          trunk/hudson/pom.xml
          http://fisheye4.cenqua.com/changelog/hudson/?cs=20580
          Log:
          Merging in the junit branch. Lots of new features and bugfixes, including:

          • support for TestAction that can contribute various things to any TestObject (packages, classes and methods), using the TestDataPublisher extension point.
            Features include UI on the TestObject page, badges on the overview page, tasks in the side panel and annotating test output.
          • a junit-attachments plugin that uses this to record test attachments (files)
          • an extension to the claim plugin that offers claims on failing tests
          • editable test descriptions (on any TestObject)
          • a history page for all TestObjects (including test count and duration graph)
          • error details for failed tests are shown in a collapsed panel in test overview pages (package+class)
          • [FIXED JENKINS-3149] if file is .xml but does not parse, record a fake failing test
          • [FIXED JENKINS-1820] extra column displaying skipped tests
          • JENKINS-2046 new test packages, classes and methods are shown in bold
          • [FIXED JENKINS-2228] test duration charts on all levels
          • [FIXED JENKINS-3451] expand environment variables in test results path
          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : huybrechts Path: trunk/hudson/main/core/src/main/java/hudson/tasks/junit/CaseResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/ClassResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/PackageResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/SuiteResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TabulatedResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TestObject.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TestResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TestResultAction.java trunk/hudson/main/core/src/main/resources/hudson/model/AbstractBuild/sidepanel.jelly trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/CaseResult/index.jelly trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/ClassResult/body.jelly trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/MetaTabulatedResult/body.jelly trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/TabulatedResult/index.jelly trunk/hudson/main/core/src/main/resources/hudson/tasks/test/MatrixTestResult/index.jelly trunk/hudson/plugins/claim/pom.xml trunk/hudson/plugins/claim/src/main/java/hudson/plugins/claim/ClaimBuildAction.java trunk/hudson/plugins/claim/src/main/java/hudson/plugins/claim/ClaimPublisher.java trunk/hudson/plugins/claim/src/main/webapp/help-sticky.html trunk/hudson/pom.xml http://fisheye4.cenqua.com/changelog/hudson/?cs=20580 Log: Merging in the junit branch. Lots of new features and bugfixes, including: support for TestAction that can contribute various things to any TestObject (packages, classes and methods), using the TestDataPublisher extension point. Features include UI on the TestObject page, badges on the overview page, tasks in the side panel and annotating test output. a junit-attachments plugin that uses this to record test attachments (files) an extension to the claim plugin that offers claims on failing tests editable test descriptions (on any TestObject) a history page for all TestObjects (including test count and duration graph) error details for failed tests are shown in a collapsed panel in test overview pages (package+class) [FIXED JENKINS-3149] if file is .xml but does not parse, record a fake failing test [FIXED JENKINS-1820] extra column displaying skipped tests JENKINS-2046 new test packages, classes and methods are shown in bold [FIXED JENKINS-2228] test duration charts on all levels [FIXED JENKINS-3451] expand environment variables in test results path
          Hide
          cbos Cees Bos added a comment -

          In one of the attachements I see that skipped junit tests are reported.
          But JUnit reported with ant does not report them. How did you manage to get them
          reported?

          In 1.320 I see that this is implemented, so we really like to use it!

          Show
          cbos Cees Bos added a comment - In one of the attachements I see that skipped junit tests are reported. But JUnit reported with ant does not report them. How did you manage to get them reported? In 1.320 I see that this is implemented, so we really like to use it!
          Hide
          huybrechts huybrechts added a comment -

          can you attach an Ant JUnit .xml file that has the problem ?

          Show
          huybrechts huybrechts added a comment - can you attach an Ant JUnit .xml file that has the problem ?
          Hide
          cbos Cees Bos added a comment -

          The point is with default ant junit runs it does not report skipped tests.
          Ant junit is based on JUnit 3.8 and does not know @Ignore, that is introduces in
          Junit4.x
          Our junit reports only show successful and failed tests. The ignore tests are
          not mentioned at all.
          In the example above I see this:
          <testcase time="0.296" classname="org.antlr.stringtemplate.js.JSTemplateTest"
          name="includeTest">
          <skipped/>
          </testcase>

          So somehow it can get logged. So is this done with ANT or probably with Maven or
          another runner.

          Show
          cbos Cees Bos added a comment - The point is with default ant junit runs it does not report skipped tests. Ant junit is based on JUnit 3.8 and does not know @Ignore, that is introduces in Junit4.x Our junit reports only show successful and failed tests. The ignore tests are not mentioned at all. In the example above I see this: <testcase time="0.296" classname="org.antlr.stringtemplate.js.JSTemplateTest" name="includeTest"> <skipped/> </testcase> So somehow it can get logged. So is this done with ANT or probably with Maven or another runner.
          Hide
          mindless Alan Harder added a comment -

          marking fixed as per above commit comment.. if there is a remaining problem please open a new issue, thanks.

          Show
          mindless Alan Harder added a comment - marking fixed as per above commit comment.. if there is a remaining problem please open a new issue, thanks.
          mindless Alan Harder made changes -
          Field Original Value New Value
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee huybrechts [ huybrechts ]
          Resolution Fixed [ 1 ]
          abayer Andrew Bayer made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              huybrechts huybrechts
              Reporter:
              cbos Cees Bos
            • Votes:
              1 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: