Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-11980

some unit test failed but build is passed

    Details

    • Similar Issues:

      Description

      we have a tool that generates test report in CppUnit-1.x format.
      about a week ago we've noticed that sometimes jenkins marks build as "Passed" in spite of some tests were failed. currently we have jenkins 1.441 installed, but it seems that the problem appeared on previous versions of jenkins.

        Attachments

          Activity

          Hide
          gbois Gregory Boissinot added a comment -

          Could you specify your plugins versions (xUnit, CppUnit, ...)?
          Could you then attach your job configuration file (config.xml in $JENKINS_HOME/jobs/<job_name>)?
          And could you also upload your CppUnit result files?
          I'll try to reproduce the issue.
          Thanks

          Show
          gbois Gregory Boissinot added a comment - Could you specify your plugins versions (xUnit, CppUnit, ...)? Could you then attach your job configuration file (config.xml in $JENKINS_HOME/jobs/<job_name>)? And could you also upload your CppUnit result files? I'll try to reproduce the issue. Thanks
          Hide
          andreyalexeenko Andrey Alexeenko added a comment -

          Our plugin versions are:
          Jenkins xUnit plugin 1.32
          Hudson cppunit plugin 1.10
          Jenkins Cppcheck Plug-in 1.0.2

          We've done some investigations and found that the problem is in xUnit plugin. Here is cut from job console:

          <Statistics>
          <Tests>14548</Tests>
          <FailuresTotal>1</FailuresTotal>
          <Errors>0</Errors>
          <Failures>1</Failures>
          </Statistics>
          </TestRun>
          [xUnit] [INFO] - Starting to record.
          [xUnit] [INFO] - Processing CppUnit-1.x
          [xUnit] [INFO] - [CppUnit-1.x] - 1 test report file(s) were found with the pattern '**/results.xml' relative to '<<<Project path>>>' for the testing framework 'CppUnit-1.x'.
          [xUnit] [INFO] - Setting the build status to SUCCESS
          [xUnit] [INFO] - Stopping recording.

          as you can see xUnit sets build status to SUCCESS inspite of failed test.

          Show
          andreyalexeenko Andrey Alexeenko added a comment - Our plugin versions are: Jenkins xUnit plugin 1.32 Hudson cppunit plugin 1.10 Jenkins Cppcheck Plug-in 1.0.2 We've done some investigations and found that the problem is in xUnit plugin. Here is cut from job console: <Statistics> <Tests>14548</Tests> <FailuresTotal>1</FailuresTotal> <Errors>0</Errors> <Failures>1</Failures> </Statistics> </TestRun> [xUnit] [INFO] - Starting to record. [xUnit] [INFO] - Processing CppUnit-1.x [xUnit] [INFO] - [CppUnit-1.x] - 1 test report file(s) were found with the pattern '**/results.xml' relative to '<<<Project path>>>' for the testing framework 'CppUnit-1.x'. [xUnit] [INFO] - Setting the build status to SUCCESS [xUnit] [INFO] - Stopping recording. as you can see xUnit sets build status to SUCCESS inspite of failed test.

            People

            • Assignee:
              gbois Gregory Boissinot
              Reporter:
              andreyalexeenko Andrey Alexeenko
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated: