Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53186

NUnit 2.x test cases with result Error are reported as Success

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: xunit-plugin
    • Labels:
      None
    • Environment:
      xunit-plugin 2.2.3
      nunit3-console.exe 3.7.0
    • Similar Issues:

      Description

      We are running `nunit3-console -result:TestResult.xml;format=nunit2` to produce NUnit2 compatible test reports with the NUnit3 runner.

      With the earlier xunit-plugin 1.102 version this worked just fine. Now with xunit-plugin 2.2.3, if we have errors / failures in the test results, these are not reported as failures anymore. Even worse, they are reported as success!

      TestResults.xml report file says:

      Jenkins / xunit-plugin 2.2.3 says:

      See the full example XML report attached

        Attachments

          Activity

          tknerr Torben Knerr created issue -
          tknerr Torben Knerr made changes -
          Field Original Value New Value
          Attachment Nunit3-in-nunit2-format-with-failures-TestReport.xml [ 43819 ]
          tknerr Torben Knerr made changes -
          tknerr Torben Knerr made changes -
          Attachment screenshot-1.png [ 43821 ]
          tknerr Torben Knerr made changes -
          Attachment screenshot-2.png [ 43822 ]
          tknerr Torben Knerr made changes -
          Description We are running `nunit3-console -result:TestResult.xml;format=nunit2` to produce NUnit2 compatible test reports with the NUnit3 runner.

          With the earlier xunit-plugin 1.102 version this worked just fine. Now with xunit-plugin 2.2.3, if we have errors / failures in the test results, these are not reported anymore.

          See the example XML report attached
          We are running `nunit3-console -result:TestResult.xml;format=nunit2` to produce NUnit2 compatible test reports with the NUnit3 runner.

          With the earlier xunit-plugin 1.102 version this worked just fine. Now with xunit-plugin 2.2.3, if we have errors / failures in the test results, these are not reported as failures anymore. Even worse, they are reported as success!

          TestResults.xml report file says:

           !screenshot-2.png|thumbnail!

          Jenkins / xunit-plugin 2.2.3 says:

           !screenshot-1.png|thumbnail!

          See the full example XML report attached
          tknerr Torben Knerr made changes -
          Summary Test Failures not reported when using nunit3-console with nunit2 output formatter Test Failures reported as Success when using nunit3-console with nunit2 output formatter
          tknerr Torben Knerr made changes -
          Issue Type Improvement [ 4 ] Bug [ 1 ]
          tknerr Torben Knerr made changes -
          Priority Minor [ 4 ] Major [ 3 ]
          nfalco Nikolas Falco made changes -
          Summary Test Failures reported as Success when using nunit3-console with nunit2 output formatter NUnit 2.x test cases with result Error are reported as Success
          nfalco Nikolas Falco made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          nfalco Nikolas Falco made changes -
          Status In Progress [ 3 ] Fixed but Unreleased [ 10203 ]
          Resolution Fixed [ 1 ]
          nfalco Nikolas Falco made changes -
          Status Fixed but Unreleased [ 10203 ] Resolved [ 5 ]

            People

            • Assignee:
              nfalco Nikolas Falco
              Reporter:
              tknerr Torben Knerr
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: