Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27395

distinguish test reports filed from different parts of workflow

    XMLWordPrintable

    Details

    • Sprint:
      Pipeline - July/August, Pipeline - December
    • Similar Issues:

      Description

      JUnit test result archiving can happen multiple times inside workflow, and currently there's no way to meaningfully distinguish multiple invocations.

        Attachments

          Issue Links

            Activity

            Hide
            sid_s Sid S added a comment - - edited

            Documenting for future users

            1. Switched from nunit3.x to mstest's .trx (xml) test output. In dotnet this simply meant swapping the logger to the mstest one even though we're still using nunit as the actual test framework i.e.

            dotnet test --logger "trx;LogFileName=resultfile.trx"

            2. Directly used the xsl file published by the mstest folks (here) to transform .trx to junit compatible .xml files. In powershell is this a simple

            $Xslt = New-Object System.Xml.Xsl.XslCompiledTransform
            $Xslt.Load($xsltFilePath)    
            $Xslt.Transform($xmlInputFilePath, $xmlOutputFilePath)
            

            3. Use a Jenkinsfile like the one below. In the traditional Jenkins UI, the stage names get prefixed, so tests with same namespace are now seen differently. BlueOcean UI still shows them as indistinguishable / identical duplicates

            pipeline {
              agent {
                node {
                  label 'master'
                }
              }
              stages {    
                stage('variant-1') {
                  steps {
                    // build steps
                  }
                  post {
                    always {
                        junit 'results/*.xml'
                    }
                  }
                }
                stage('variant-2') {
                  steps {
                    // build steps
                  }
                  post {
                    always {
                        junit 'results/*.xml'
                    }
                  }
                }
              }
            }
            
            Show
            sid_s Sid S added a comment - - edited Documenting for future users 1. Switched from nunit3.x to mstest's .trx (xml) test output. In dotnet this simply meant swapping the logger to the mstest one even though we're still using nunit as the actual test framework i.e. dotnet test --logger "trx;LogFileName=resultfile.trx" 2. Directly used the xsl file published by the mstest folks ( here ) to transform .trx to junit compatible .xml files. In powershell is this a simple $Xslt = New- Object System .Xml.Xsl.XslCompiledTransform $Xslt.Load($xsltFilePath) $Xslt.Transform($xmlInputFilePath, $xmlOutputFilePath) 3. Use a Jenkinsfile like the one below. In the traditional Jenkins UI, the stage names get prefixed, so tests with same namespace are now seen differently. BlueOcean UI still shows them as indistinguishable / identical duplicates pipeline { agent { node { label 'master' } } stages { stage( 'variant-1' ) { steps { // build steps } post { always { junit 'results/*.xml' } } } stage( 'variant-2' ) { steps { // build steps } post { always { junit 'results/*.xml' } } } } }
            Hide
            trejkaz trejkaz added a comment -

            Something doesn't quite add up here. Our build already has stage names, but we still don't get the tests distinguished between the different stages. The only workaround idea I had so far was to use XSLT to add the prefix into the actual XML files.

             

            Show
            trejkaz trejkaz added a comment - Something doesn't quite add up here. Our build already has stage names, but we still don't get the tests distinguished between the different stages. The only workaround idea I had so far was to use XSLT to add the prefix into the actual XML files.  
            Hide
            sid_s Sid S added a comment -

            trejkaz, could you share your XSLT file or link to it?

            Show
            sid_s Sid S added a comment - trejkaz , could you share your XSLT file or link to it?
            Hide
            trejkaz trejkaz added a comment -

            We don't use an XSLT yet because we're using Gradle's reports, which are already in the right format.

             

            Show
            trejkaz trejkaz added a comment - We don't use an XSLT yet because we're using Gradle's reports, which are already in the right format.  
            Hide
            ottago Nicholas Klopfer-Webber added a comment -

            We had the same problem, but managed to work it out.

            When we started using a Jenkinsfile we copied this example, however this bypasses the code that keeps each run of tests independent.

            step([$class : 'JUnitResultArchiver', testResults: '**/build/test-results/**/*.xml'])
            

            What we now use and works is:

            junit testResults: '**/build/test-results/**/*.xml'
            
            Show
            ottago Nicholas Klopfer-Webber added a comment - We had the same problem, but managed to work it out. When we started using a Jenkinsfile we copied this example, however this bypasses the code that keeps each run of tests independent. step([$class : 'JUnitResultArchiver' , testResults: '**/build/test-results /**/ *.xml' ]) What we now use and works is: junit testResults: '**/build/test-results /**/ *.xml'

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                kohsuke Kohsuke Kawaguchi
              • Votes:
                35 Vote for this issue
                Watchers:
                64 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: