-
Type:
Bug
-
Status: Resolved (View Workflow)
-
Priority:
Major
-
Resolution: Fixed
-
Component/s: xunit-plugin
-
Labels:None
-
Environment:Jenkins 1.455
xunit 1.40
-
Similar Issues:
We had an out of memory exception while running our MSTEST unit tests which caused all subsequent tests to be NotExecuted. Unfortunately those "NotExecuted" tests were counted as passed, so the test job succeeded instead of failing.
One example from the TRX file:
<UnitTestResult executionId="88518b81-226a-4fe9-9896-774a00c13e8e" testId="3509a64f-6214-eb24-6628-bd431f93997c" testName="TestcaseWcdmaTxIntermod_5_12__FDD9" computerName="1SP1-SLAVE2" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="NotExecuted" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="88518b81-226a-4fe9-9896-774a00c13e8e"> </UnitTestResult>
The transformation in the junitResult.xml file:
<case> <duration>NaN</duration> <className>ConformanceWcdmaCompleteTest.BandSpecificTests</className> <testName>TestcaseWcdmaTxIntermod_5_12__FDD9</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case>
Field | Original Value | New Value |
---|---|---|
Status | Open [ 1 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Closed [ 6 ] |
Resolution | Incomplete [ 4 ] |
Resolution | Incomplete [ 4 ] | |
Status | Closed [ 6 ] | Reopened [ 4 ] |
Workflow | JNJira [ 143574 ] | JNJira + In-Review [ 186047 ] |
Description |
We had an out of memory exception while running our MSTEST unit tests which caused all subsequent tests to be NotExecuted. Unfortunately those "NotExecuted" tests were counted as passed, so the test job succeeded instead of failing.
One example from the TRX file: <UnitTestResult executionId="88518b81-226a-4fe9-9896-774a00c13e8e" testId="3509a64f-6214-eb24-6628-bd431f93997c" testName="TestcaseWcdmaTxIntermod_5_12__FDD9" computerName="1SP1-SLAVE2" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="NotExecuted" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="88518b81-226a-4fe9-9896-774a00c13e8e"> </UnitTestResult> The transformation in the junitResult.xml file: <case> <duration>NaN</duration> <className>ConformanceWcdmaCompleteTest.BandSpecificTests</className> <testName>TestcaseWcdmaTxIntermod_5_12__FDD9</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> |
We had an out of memory exception while running our MSTEST unit tests which caused all subsequent tests to be NotExecuted. Unfortunately those "NotExecuted" tests were counted as passed, so the test job succeeded instead of failing.
One example from the TRX file: {code:xml} <UnitTestResult executionId="88518b81-226a-4fe9-9896-774a00c13e8e" testId="3509a64f-6214-eb24-6628-bd431f93997c" testName="TestcaseWcdmaTxIntermod_5_12__FDD9" computerName="1SP1-SLAVE2" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="NotExecuted" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="88518b81-226a-4fe9-9896-774a00c13e8e"> </UnitTestResult> {code} The transformation in the junitResult.xml file: {code:xml} <case> <duration>NaN</duration> <className>ConformanceWcdmaCompleteTest.BandSpecificTests</className> <testName>TestcaseWcdmaTxIntermod_5_12__FDD9</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> {code} |
Assignee | Gregory Boissinot [ gbois ] | Nikolas Falco [ nfalco ] |
Summary | Xunit Plugin detects MSTEST "NotExecuted" as "Passed" instead of "Skipped" | xUnit plugins detects MSTEST "NotExecuted" as successed instead of skipped |
Status | Reopened [ 4 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Resolved [ 5 ] |
Resolution | Fixed [ 1 ] |
Do you have the same issue with the independent MSTest Jenkins plugins?
Thanks