Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10154

Allow the MSTest plugin to mark inconclusive tests as skipped

    Details

    • Type: Improvement
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Won't Fix
    • Component/s: mstest-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      I would like to suggest an update to the MSTest plugin that will allow inconclusive tests to be marked as skipped. It would also be helpful to see the message included with the inconclusive assertion when viewing individual tests. Currently, the plugin marks inconclusive tests as passing, which leads to inaccurate overall test results.

        Attachments

          Activity

          Hide
          nilleb Ivo Bellin Salarin added a comment -

          My source (http://windyroad.com.au/dl/Open%20Source/JUnit.xsd) confirms me that junit XML format is pretty essential. Much of the descriptive mstest/vstest format is lost during this translation.

          However, I have committed some change to the mstest-to-junit.xsl, so I would ask you to check the next release.

          Show
          nilleb Ivo Bellin Salarin added a comment - My source ( http://windyroad.com.au/dl/Open%20Source/JUnit.xsd ) confirms me that junit XML format is pretty essential. Much of the descriptive mstest/vstest format is lost during this translation. However, I have committed some change to the mstest-to-junit.xsl, so I would ask you to check the next release.
          Hide
          nilleb Ivo Bellin Salarin added a comment - - edited

          [comment dropped since another comment has been added by the issue resolution]

          Show
          nilleb Ivo Bellin Salarin added a comment - - edited [comment dropped since another comment has been added by the issue resolution]
          Hide
          nilleb Ivo Bellin Salarin added a comment -

          According to junit documentation, skipped tests correspond to ones decorated with @Ignore, in Java.

          According to uservoice, TRX files do not include information about [Ignored] tests, even if the console output does.

          I think that the attached XSL is wrong, since errors and other outcomes would be listed as "skipped". I will have a look to the JUnit xml format to see if there's some markup for the outcomes other than failed, error and total..

          e.g. Counters total="3" executed="3" error="3" failed="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" passed="0" completed="0" inProgress="0" pending="0"

          Show
          nilleb Ivo Bellin Salarin added a comment - According to junit documentation, skipped tests correspond to ones decorated with @Ignore, in Java. According to uservoice, TRX files do not include information about [Ignored] tests, even if the console output does. I think that the attached XSL is wrong, since errors and other outcomes would be listed as "skipped". I will have a look to the JUnit xml format to see if there's some markup for the outcomes other than failed, error and total.. e.g. Counters total="3" executed="3" error="3" failed="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" passed="0" completed="0" inProgress="0" pending="0"
          Hide
          clmbrdude Dag Lövenvald added a comment -

          I added rules for setting the skipped element.
          Only tested on Visual Studio 2010

          Show
          clmbrdude Dag Lövenvald added a comment - I added rules for setting the skipped element. Only tested on Visual Studio 2010

            People

            • Assignee:
              nilleb Ivo Bellin Salarin
              Reporter:
              lmanco lmanco
            • Votes:
              5 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: