Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-9246

NUnit plugin occasionally under-reports the number of tests and failures.

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: nunit-plugin
    • Labels:
      None
    • Environment:
      Jenkins 1.4.0.4
      NUnit 0.13
    • Similar Issues:

      Description

      The NUnit plugin occasionally under-reports the total number of tests and failures.
      I have two seperate installs of Jenkins, each with NUnit 1.3, both of which are occasionally showing the number of tests (and failures, if there are any) as much smaller than they should be.

      The example shows my most recent build info (TestResults-current.xml and junitResult-current.xml) and the previous build's results (junitResults-previous.xml). The previous build correctly reported the number of tests and failures. The current build is showing about 4000 fewer tests than it should.

        Attachments

          Activity

          otac0n otac0n created issue -
          Hide
          otac0n otac0n added a comment -

          One extra note. This seems to happen most often when the previous test results are exactly the same as the current results.

          Show
          otac0n otac0n added a comment - One extra note. This seems to happen most often when the previous test results are exactly the same as the current results.
          otac0n otac0n made changes -
          Field Original Value New Value
          Priority Major [ 3 ] Critical [ 2 ]
          Hide
          otac0n otac0n added a comment -

          Also, this seems to have been introduced in the last update. Also also, it looks like it mostly affects parametrized tests.

          Show
          otac0n otac0n added a comment - Also, this seems to have been introduced in the last update. Also also, it looks like it mostly affects parametrized tests.
          Hide
          redsolo redsolo added a comment -

          Thanks for the report and examples. I will have a look at it

          Show
          redsolo redsolo added a comment - Thanks for the report and examples. I will have a look at it
          Hide
          otac0n otac0n added a comment -

          This is another example of the issue, from here at work.

          Build #83 was Mar 23, 2011 11:03:35 AM

          Show
          otac0n otac0n added a comment - This is another example of the issue, from here at work. Build #83 was Mar 23, 2011 11:03:35 AM
          otac0n otac0n made changes -
          Attachment trend.png [ 20404 ]
          Hide
          otac0n otac0n added a comment -

          I have attached an image that demonstrates what I'm seeing at work. During the whole history of builds, no tests were changed.

          Show
          otac0n otac0n added a comment - I have attached an image that demonstrates what I'm seeing at work. During the whole history of builds, no tests were changed.
          Hide
          bartensud Thomas Bartensud added a comment -

          Missing parameterized tests on conversion to junit result format.

          Show
          bartensud Thomas Bartensud added a comment - Missing parameterized tests on conversion to junit result format.
          bartensud Thomas Bartensud made changes -
          Hide
          bartensud Thomas Bartensud added a comment -

          I'm also facing the issue for missing tests from a parameterized test suite.
          See attached parameterizedTests_smallExample.zip for a small example.

          I'm running NUnit plugin 0.13 + Jenkins 1.4.0.9 (and Hudson v1.381).

          Show
          bartensud Thomas Bartensud added a comment - I'm also facing the issue for missing tests from a parameterized test suite. See attached parameterizedTests_smallExample.zip for a small example. I'm running NUnit plugin 0.13 + Jenkins 1.4.0.9 (and Hudson v1.381).
          Hide
          otac0n otac0n added a comment -

          It looks like this is fixed in Pull Request #1: https://github.com/jenkinsci/nunit-plugin/pull/1

          How soon can we get this fix implemented?

          Show
          otac0n otac0n added a comment - It looks like this is fixed in Pull Request #1: https://github.com/jenkinsci/nunit-plugin/pull/1 How soon can we get this fix implemented?
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: redsolo
          Path:
          src/test/java/hudson/plugins/nunit/NUnitIntegrationTest.java
          src/test/resources/hudson/plugins/nunit/NUnitIntegrationTest/testIssue9246.zip
          http://jenkins-ci.org/commit/nunit-plugin/8a09775a2899819cde03724e566656d6324b468a
          Log:
          JENKINS-9246 Added test case that was attached to the issue report.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: redsolo Path: src/test/java/hudson/plugins/nunit/NUnitIntegrationTest.java src/test/resources/hudson/plugins/nunit/NUnitIntegrationTest/testIssue9246.zip http://jenkins-ci.org/commit/nunit-plugin/8a09775a2899819cde03724e566656d6324b468a Log: JENKINS-9246 Added test case that was attached to the issue report.
          Hide
          redsolo redsolo added a comment -

          Thanks for the fix for this issue. Im releasing a new version asap.

          Show
          redsolo redsolo added a comment - Thanks for the fix for this issue. Im releasing a new version asap.
          redsolo redsolo made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 139407 ] JNJira + In-Review [ 188483 ]

            People

            • Assignee:
              redsolo redsolo
              Reporter:
              otac0n otac0n
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: