Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58430

Publishing two fitnesse xml files with same test name confuses the results

XMLWordPrintable

      • In one pipeline job, run the same FitNesse tests and generate 2 xml files. So for instance:
        • FitNesse tests on Linux generates fitnesse_linux.xml
        • FitNesse tests on Windows generates fitnesse_windows.xml
          The way the tests are generated is as follows (same for windows)
          java -jar ./fitnesse-standalone.jar -c "MySuite?suite&format=xml&includehtml" -b fitnesse_windows.xml
          
      • Now in the pipeline job, the tests are published in this way:
        step([$class: 'FitnesseResultsRecorder', fitnessePathToXmlResultsIn: 'fitnesse_*.xml'])
      • Then let's assume that MySuite.TestOne fails on Linux, when I go to see the Details of the failure from the FitNesseReport link in Jenkins, while TestOne shows as failed, the html details I can see from there actually shows the details of the windows run.

      To reproduce this issue, use the following pipeline. Run it once and them manually copy the two xml files attached into the workspace. Run it again and it should publish the FitNesse results. My agent is a linux agent if that makes a difference.

      pipeline {
          agent {
              label 'your-agent'
          }
          stages {
              stage ('Repro') {
                  steps {
                      echo 'Make sure the files are in the Workspace'
                  }
              }
          }
          post {
              always {
                  step([$class: 'FitnesseResultsRecorder', fitnessePathToXmlResultsIn: 'fitnesse_*.xml'])
              }
          }
      }
      

      Once the results are published, you can see that the fitnesse_windows.xml has been successful while fitnesse_linux.xml has failed. You can see this in the FitNesse report that this is represented correctly:


      But if I click on the fitnesse_linux.xml and then on the Details link for test BuysideEnterprise.Ims.MySuite.TestOne then it actually shows me the details of the run in fitnesse_windows.xml instead of the linux one. It looks like the details have been overridden.

      Therefore, while I see that the test has failed, I cannot see why as it is showing me the incorrect run details:

        1. Details-of-test-not-representing-failure.png
          11 kB
          Johann Moro
        2. fitnesse_linux.xml
          5 kB
          Johann Moro
        3. fitnesse_windows.xml
          5 kB
          Johann Moro
        4. FitNesseReport.png
          8 kB
          Johann Moro

            Unassigned Unassigned
            johannmoro Johann Moro
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: