Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7458

Support for "orderedtest" results

    XMLWordPrintable

    Details

    • Type: Patch
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Won't Fix
    • Component/s: mstest-plugin
    • Labels:
      None
    • Environment:
      Hudson 1.373, MsTest plugin 0.6.
    • Similar Issues:

      Description

      When using .orderedtest as test containers in mstest (VS 2008), the resulting trx files has a slightly different structure which causes the plugin to yield "no tests".
      The attached xsl will handle the trx output from both "regular" unittest containers and "orderedtest" containers.
      I have not verified with anything else than the previously indicated versions.

        Attachments

          Activity

          Hide
          marco Marco Medrano added a comment -

          Near to one Year has happened, and looks like no support for this bug.

          Show
          marco Marco Medrano added a comment - Near to one Year has happened, and looks like no support for this bug.
          Hide
          nilleb Ivo Bellin Salarin added a comment -

          Good evening,

          I'm willing to close this issue, for a couple of reasons:
          1) ordered tests in mstest, afaik, are exposed with a hierarchical structure. junit has no comparable feature.
          2) the current xsl already contains all the functinalities proposed in the attached xsl.
          3) I haven't a trx resulting from the execution of an ordered test. If somebody can provide it, he's welcome.

          As a final point, dear Marco Medrano, this is an open source project. If you are willing to help, you're welcome.

          Show
          nilleb Ivo Bellin Salarin added a comment - Good evening, I'm willing to close this issue, for a couple of reasons: 1) ordered tests in mstest, afaik, are exposed with a hierarchical structure. junit has no comparable feature. 2) the current xsl already contains all the functinalities proposed in the attached xsl. 3) I haven't a trx resulting from the execution of an ordered test. If somebody can provide it, he's welcome. As a final point, dear Marco Medrano, this is an open source project. If you are willing to help, you're welcome.
          Hide
          nicholasdgoodman Nicholas Goodman added a comment - - edited

          Ivo Bellin Salarin, I am attaching a sample ordered test result from VS2015 (same schema as VS2010) for your re-consideration on this issue.

          Additionally, I am attaching an XSL document, mstest-ordered-to-flat, which will "flatten" the TestEntries node into a non-hierarchal collection of TestEntry nodes.

          This XSL file will also delete the OrderedTest node from the TestDefinitions section. In doing so, the Jenkins plugin correctly reports the number of executed tests (because the definition entry is missing for the ordered test).

          As a bonus, if you apply this XSL transformation on a standard (non-ordered) TRX file, it will leave the file unchanged. I have tested both ordered and regular test results against version 0.19 of the MSTest plugin and it successfully processes the files.

          This technique is preferred over the attached mstest-to-junit XSL file, because it preserves much more of the data, such as console output.

          I am not a java-coder so cannot contribute directly, but perhaps the plugin could (optionally) apply this transformation on any or all TRX files before further processing.

          Show
          nicholasdgoodman Nicholas Goodman added a comment - - edited Ivo Bellin Salarin , I am attaching a sample ordered test result from VS2015 (same schema as VS2010) for your re-consideration on this issue. Additionally, I am attaching an XSL document, mstest-ordered-to-flat , which will "flatten" the TestEntries node into a non-hierarchal collection of TestEntry nodes. This XSL file will also delete the OrderedTest node from the TestDefinitions section. In doing so, the Jenkins plugin correctly reports the number of executed tests (because the definition entry is missing for the ordered test). As a bonus, if you apply this XSL transformation on a standard (non-ordered) TRX file, it will leave the file unchanged. I have tested both ordered and regular test results against version 0.19 of the MSTest plugin and it successfully processes the files. This technique is preferred over the attached mstest-to-junit XSL file, because it preserves much more of the data, such as console output. I am not a java-coder so cannot contribute directly, but perhaps the plugin could (optionally) apply this transformation on any or all TRX files before further processing.

            People

            • Assignee:
              nilleb Ivo Bellin Salarin
              Reporter:
              wibg wibg
            • Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: