The HP Application Automation Tools plugin, specifically the com.hp.application.automation.tools.run.RunFromAlmBuilder build step, is reporting incorrect test run results that do not match with the HP ALM web client.

Snippet from impacted results xml:
</system-out>
</testcase>
<testcase name="Root\Deployment_Checks\LifeQuotingEAR\TST\Checkout2[1]WebEZ 2.0" type="QTP" time="98" classname="All-Tests.TST\Checkout2" status="pass">
<system-out>14/11/2016 11:04:01 Running: [1]WebEZ 2.0
14/11/2016 11:04:01 Running test: [1]WebEZ 2.0, Test id: 1167, Test instance id: 456
Test: [1]WebEZ 2.0, Id: 456, Execution status: Running
Test: [1]WebEZ 2.0, Id: 456, Execution status: Passed, Message: Test run passed. host: #####
Step: Start Test
Start Test WebEZ 2.0
Step: Start Action
Start Action Main [WebEZ_2.0_Mstr_Web]
Step: Agency Portal, Status: Failed
Agency Portal did not load.
Step: End Action
End Action Main [WebEZ_2.0_Mstr_Web]
Step: End Test
End Test WebEZ 2.0

Link:#######=IRun&EntityID=317516
14/11/2016 11:05:44 Test complete: [1]WebEZ 2.0, Run ID: 317516

Snippet from the job's console output:
Passed : Root\Deployment_Checks\LifeQuotingEAR\TST\Checkout2[1]WebEZ 2.0

As shown in the results.xml snippet, the second step failed. In HP ALM the overall test is marked as a failure, but the Jenkins plugin is marking it as a success and when that test is the only one running the overall Jenkins job is marked as a success instead of the expected failure.

I saw JENKINS-28691, but this seems like a different issue as it doesn't appear to be picking up old test results, it's just not failing the test after the failed test step similar to the expected behavior that I'm seeing directly within the HP ALM web client. I've attached a screenshot of the test results reported within ALM from the same test run that I pulled the above results.xml and console output snippets from.

I can replicate this problem with HP Application Automation Tools plugin versions 4.0.1 and 4.5.0.