Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58430

Publishing two fitnesse xml files with same test name confuses the results

    Details

    • Similar Issues:

      Description

      • In one pipeline job, run the same FitNesse tests and generate 2 xml files. So for instance:
        • FitNesse tests on Linux generates fitnesse_linux.xml
        • FitNesse tests on Windows generates fitnesse_windows.xml
          The way the tests are generated is as follows (same for windows)
          java -jar ./fitnesse-standalone.jar -c "MySuite?suite&format=xml&includehtml" -b fitnesse_windows.xml
          
      • Now in the pipeline job, the tests are published in this way:
        step([$class: 'FitnesseResultsRecorder', fitnessePathToXmlResultsIn: 'fitnesse_*.xml'])
      • Then let's assume that MySuite.TestOne fails on Linux, when I go to see the Details of the failure from the FitNesseReport link in Jenkins, while TestOne shows as failed, the html details I can see from there actually shows the details of the windows run.

      To reproduce this issue, use the following pipeline. Run it once and them manually copy the two xml files attached into the workspace. Run it again and it should publish the FitNesse results. My agent is a linux agent if that makes a difference.

      pipeline {
          agent {
              label 'your-agent'
          }
          stages {
              stage ('Repro') {
                  steps {
                      echo 'Make sure the files are in the Workspace'
                  }
              }
          }
          post {
              always {
                  step([$class: 'FitnesseResultsRecorder', fitnessePathToXmlResultsIn: 'fitnesse_*.xml'])
              }
          }
      }
      

      Once the results are published, you can see that the fitnesse_windows.xml has been successful while fitnesse_linux.xml has failed. You can see this in the FitNesse report that this is represented correctly:


      But if I click on the fitnesse_linux.xml and then on the Details link for test BuysideEnterprise.Ims.MySuite.TestOne then it actually shows me the details of the run in fitnesse_windows.xml instead of the linux one. It looks like the details have been overridden.

      Therefore, while I see that the test has failed, I cannot see why as it is showing me the incorrect run details:

        Attachments

          Activity

          Hide
          johannmoro Johann Moro added a comment -

          Note that this issue probably needs to be reproduced on Linux because JENKINS-58923 will prevent you from reproducing this on Windows.

          Show
          johannmoro Johann Moro added a comment - Note that this issue probably needs to be reproduced on Linux because JENKINS-58923 will prevent you from reproducing this on Windows.
          Hide
          jaruzafa jaruzafa added a comment -

          Fixed on fitnesse-plugin 1.30

          Show
          jaruzafa jaruzafa added a comment - Fixed on fitnesse-plugin 1.30

            People

            • Assignee:
              Unassigned
              Reporter:
              johannmoro Johann Moro
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: