Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18642

Job build is marked as failed if NUnit test result contains only ignored tests

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: nunit-plugin
    • Labels:
      None
    • Environment:
      Windows 7, Jenkins, NUnit Publisher 0.14, NUnit 2.5.10
    • Similar Issues:

      Description

      Job build is marked as failed if NUnit test result contains only ignored tests.

      It could be said that a test suite with only ignored tests is useless and should be marked as failed. But I have an use case in a highly distributed test environment where it's a blocker for me:
      A high number of a parameterized unit test is splitted up (by a filtering mechanism) in order to distribute the test executions on jenkins slaves (via matrix build). In certain cases the dynamic test splitting results in only one test case per jenkins job. And if this test case is marked as ignored then the whole (distributed) build fails

      This can be easily fixed in NUnitPublisher.java by adding the red part:

      if (result.getPassCount()==0 && result.getFailCount()==0 && result.getSkipCount()==0) {
      throw new AbortException("None of the test reports contained any result");
      }

      I tested this fix in my environment and it works without any issues for several days now.

        Attachments

          Activity

          bartensud Thomas Bartensud created issue -
          bartensud Thomas Bartensud made changes -
          Field Original Value New Value
          Description Job build is marked as failed if NUnit test result contains only ignored tests.

          It could be said that a test suite with only ignored tests is useless and should be marked as failed. But I have an use case in a highly distributed test environment where it's a blocker for me:
          A high number of a parameterized unit test is splitted up (by a filtering mechanism) in order to distribute the test executions on jenkins slaves (via matrix build). In certain cases the dynamic test splitting results in only one test case per jenkins job. And if this test case is marked as ignored then the whole (distributed) build fails:(


          This can be easily fixed in NUnitPublisher.java by adding the {color:red}red part{color}:
          {quote}
          if (result.getPassCount()==0 && result.getFailCount()==0 {color:red}&& result.getSkipCount()==0{color}) {
             throw new AbortException("None of the test reports contained any result");
          }
          {quote}
          I tested this fix in my environment and it works without any issues for several days now.
          Job build is marked as failed if NUnit test result contains only ignored tests.

          It could be said that a test suite with only ignored tests is useless and should be marked as failed. But I have an use case in a highly distributed test environment where it's a blocker for me:
          A high number of a parameterized unit test is splitted up (by a filtering mechanism) in order to distribute the test executions on jenkins slaves (via matrix build). In certain cases the dynamic test splitting results in only one test case per jenkins job. And if this test case is marked as ignored then the whole (distributed) build fails:(


          This can be easily fixed in NUnitPublisher.java by adding the {color:red}red part{color}:
          {quote}
          if (result.getPassCount()==0 && result.getFailCount()==0 {color:red}&& result.getSkipCount()==0{color}) \{
             throw new AbortException("None of the test reports contained any result");
          }
          {quote}
          I tested this fix in my environment and it works without any issues for several days now.
          kinow Bruno P. Kinoshita made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          kinow Bruno P. Kinoshita made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 149952 ] JNJira + In-Review [ 193369 ]

            People

            • Assignee:
              redsolo redsolo
              Reporter:
              bartensud Thomas Bartensud
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: