Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52253

xUnit in case of multiple tool type processes only the first one

    Details

    • Similar Issues:

      Description

      After upgrade xUnit to version 2.0.3 from 2.0.2 only first configured unit test result is being processed. Other configured test results are ignored.

      xUnit 2.0.3:

      INFO: Starting to record.
      INFO: Processing PHPUnit-3.x (default)
      INFO: [PHPUnit-3.x (default)] - 1 test report file(s) were found with the pattern 'build/logs/junit.xml' relative to '/var/lib/jenkins/jobs/app/workspace' for the testing framework 'PHPUnit-3.x (default)'.
      INFO: Check 'Failed Tests' threshold.
      INFO: Check 'Skipped Tests' threshold.
      INFO: Setting the build status to SUCCESS
      INFO: Stopping recording.

      xUnit 2.0.2: 

      INFO: Starting to record.
      INFO: Processing PHPUnit-3.x (default)
      INFO: [PHPUnit-3.x (default)] - 1 test report file(s) were found with the pattern 'build/logs/junit.xml' relative to '/var/lib/jenkins/jobs/app/workspace' for the testing framework 'PHPUnit-3.x (default)'.
      INFO: Processing PHPUnit-3.x (default)
      INFO: [PHPUnit-3.x (default)] - 1 test report file(s) were found with the pattern 'build/codecept/report.xml' relative to '/var/lib/jenkins/jobs/app/workspace' for the testing framework 'PHPUnit-3.x (default)'.
      INFO: Check 'Failed Tests' threshold.
      INFO: The total number of tests for this category exceeds the specified "failure" threshold value.
      INFO: Setting the build status to FAILURE
      INFO: Stopping recording.
      Build step 'Publish xUnit test result report' changed build result to FAILURE

      It is a freestyle ant build project and for this case I use cumulative test results.

      This might be related to JENKINS-47194.

        Attachments

          Activity

          dukecz Michal Kruzik created issue -
          Hide
          nfalco Nikolas Falco added a comment -

          I need more informations or it is impossible to reproduce.

          Job configuration, workspace folder structures, generated reports and so on.

          Show
          nfalco Nikolas Falco added a comment - I need more informations or it is impossible to reproduce. Job configuration, workspace folder structures, generated reports and so on.
          Hide
          benken_parasoft Joseph Benken added a comment -

          I have run into the exact same issue and had to downgrade to 2.0.2.  In the build configuration, I click the Add button twice, to publish two different types of XML reports, each with a different pattern.  Only the first report type I added is processed and the second one is totally ignored.  I see the same thing if I re-add them in a different order, only the first one is used.

          Show
          benken_parasoft Joseph Benken added a comment - I have run into the exact same issue and had to downgrade to 2.0.2.  In the build configuration, I click the Add button twice, to publish two different types of XML reports, each with a different pattern.  Only the first report type I added is processed and the second one is totally ignored.  I see the same thing if I re-add them in a different order, only the first one is used.
          Hide
          mathan2k Mathankumar Chinnaraj added a comment -

          I too faced the same issue in pipeline project. Everything worked fine with 2.0.2

          Show
          mathan2k Mathankumar Chinnaraj added a comment - I too faced the same issue in pipeline project. Everything worked fine with 2.0.2
          dukecz Michal Kruzik made changes -
          Field Original Value New Value
          Attachment config.xml [ 43281 ]
          Hide
          dukecz Michal Kruzik added a comment -

          Nikolas Falco Config file attached. xUnit configuration snippet from it:

              <xunit plugin="xunit@1.102">
                <types>
                  <PHPUnitJunitHudsonTestType>
                    <pattern>build/logs/junit.xml</pattern>
                    <skipNoTestFiles>false</skipNoTestFiles>
                    <failIfNotNew>true</failIfNotNew>
                    <deleteOutputFiles>true</deleteOutputFiles>
                    <stopProcessingIfError>true</stopProcessingIfError>
                  </PHPUnitJunitHudsonTestType>
                  <PHPUnitJunitHudsonTestType>
                    <pattern>build/codecept/report.xml</pattern>
                    <skipNoTestFiles>false</skipNoTestFiles>
                    <failIfNotNew>true</failIfNotNew>
                    <deleteOutputFiles>true</deleteOutputFiles>
                    <stopProcessingIfError>true</stopProcessingIfError>
                  </PHPUnitJunitHudsonTestType>
                </types>
                <thresholds>
                  <org.jenkinsci.plugins.xunit.threshold.FailedThreshold>
                    <unstableThreshold>0</unstableThreshold>
                    <unstableNewThreshold>0</unstableNewThreshold>
                    <failureThreshold>1</failureThreshold>
                    <failureNewThreshold>1</failureNewThreshold>
                  </org.jenkinsci.plugins.xunit.threshold.FailedThreshold>
                  <org.jenkinsci.plugins.xunit.threshold.SkippedThreshold>
                    <unstableThreshold>1</unstableThreshold>
                    <unstableNewThreshold>1</unstableNewThreshold>
                    <failureThreshold>2</failureThreshold>
                    <failureNewThreshold>2</failureNewThreshold>
                  </org.jenkinsci.plugins.xunit.threshold.SkippedThreshold>
                </thresholds>
                <thresholdMode>1</thresholdMode>
                <extraConfiguration>
                  <testTimeMargin>3000</testTimeMargin>
                </extraConfiguration>
              </xunit>
          Show
          dukecz Michal Kruzik added a comment - Nikolas Falco Config file attached. xUnit configuration snippet from it: <xunit plugin= "xunit@1.102" > <types> <PHPUnitJunitHudsonTestType> <pattern>build/logs/junit.xml</pattern> <skipNoTestFiles> false </skipNoTestFiles> <failIfNotNew> true </failIfNotNew> <deleteOutputFiles> true </deleteOutputFiles> <stopProcessingIfError> true </stopProcessingIfError> </PHPUnitJunitHudsonTestType> <PHPUnitJunitHudsonTestType> <pattern>build/codecept/report.xml</pattern> <skipNoTestFiles> false </skipNoTestFiles> <failIfNotNew> true </failIfNotNew> <deleteOutputFiles> true </deleteOutputFiles> <stopProcessingIfError> true </stopProcessingIfError> </PHPUnitJunitHudsonTestType> </types> <thresholds> <org.jenkinsci.plugins.xunit.threshold.FailedThreshold> <unstableThreshold>0</unstableThreshold> <unstableNewThreshold>0</unstableNewThreshold> <failureThreshold>1</failureThreshold> <failureNewThreshold>1</failureNewThreshold> </org.jenkinsci.plugins.xunit.threshold.FailedThreshold> <org.jenkinsci.plugins.xunit.threshold.SkippedThreshold> <unstableThreshold>1</unstableThreshold> <unstableNewThreshold>1</unstableNewThreshold> <failureThreshold>2</failureThreshold> <failureNewThreshold>2</failureNewThreshold> </org.jenkinsci.plugins.xunit.threshold.SkippedThreshold> </thresholds> <thresholdMode>1</thresholdMode> <extraConfiguration> <testTimeMargin>3000</testTimeMargin> </extraConfiguration> </xunit>
          nfalco Nikolas Falco made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          nfalco Nikolas Falco made changes -
          Summary xUnit processes only one test result in freestyle project xUnit processes only the first tool type
          nfalco Nikolas Falco made changes -
          Labels ant freestyle xunit regression xunit
          nfalco Nikolas Falco made changes -
          Priority Minor [ 4 ] Major [ 3 ]
          nfalco Nikolas Falco made changes -
          Summary xUnit processes only the first tool type xUnit in case of multiple tool type processes only the first one
          nfalco Nikolas Falco made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          dukecz Michal Kruzik made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              nfalco Nikolas Falco
              Reporter:
              dukecz Michal Kruzik
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: