Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-25573

Junit plugin always produces 100% health report

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: junit-plugin
    • Labels:
      None
    • Environment:
      version 1.589
      Junit version 1.2
    • Similar Issues:

      Description

      Junit is always reporting 100% health report, regardless of number of failing testcases. I have a job with 1 of 1 test failing and still getting a 100% health report.

      Hovering over the (sunny) weather icon for the job gives this popup (see attached screenshot):
      "Test Result: 1 test failing out of a total of 1 test. 100%

      This statement above indicates there is something wrong with the plugin. I'm not sure why it would report 100% health when all tests are failing.

      This may be related to adding the health scale amplification factor, but I'm not sure. I have tried different values for the healthScaleFactor, all produce 100% health.
      https://issues.jenkins-ci.org/browse/JENKINS-24006

      This is the XML I am testing with. Note this is a custom XML I'm creating to mimic a real Junit XML.
      <?xml version="1.0" encoding="UTF-8"?>
      <testsuites name="all_tests" errors="1" tests="1" failures="1" >
      <testsuite name="all_tests" errors="1" tests="1" failures="1" >
      <testcase classname="myblock" name="mytest " time="30.57" >
      <failure> message="testfailed" </failure>
      </testcase>
      </testsuite>
      </testsuites>

      It is possible I'm doing something wrong, so I also posted here:
      https://groups.google.com/forum/#!topic/jenkinsci-users/-StbbZ0s8jI/discussion

        Attachments

          Activity

          Hide
          msinclair Mark Sinclair added a comment -

          I think this is the issue:

          The health score is calculated in this file:
          junit-plugin / src / main / java / hudson / tasks / test / AbstractTestResultAction.java

          On Line 160-162, the health score is calculated. The lines are shown below:

          int score = (totalCount == 0)
          ? 100
          : (int) (100.0 * Math.max(1.0, Math.min(0.0, 1.0 - (scaleFactor * failCount) / totalCount)));

          The problem is the Math.max has a first argument of 1.0, which makes the second argument irrelevant as it is always less than 1. The first argument should be 0.0, as the intention is to never report negative health.

          Corrected code is below (just changed to 1.0 to 0.0 in the first arg of Math.max:

          int score = (totalCount == 0)
          ? 100
          : (int) (100.0 * Math.max(0.0, Math.min(0.0, 1.0 - (scaleFactor * failCount) / totalCount)));

          I'm pretty sure the above would fix the problem. This nested max, min functions seem redundant so the code could be further simplified - I think the following is more concise:

          int score = (totalCount == 0)
          ? 100
          : (int) (100.0 * Math.max(0.0, 1.0 - (scaleFactor * failCount) / totalCount));

          Show
          msinclair Mark Sinclair added a comment - I think this is the issue: The health score is calculated in this file: junit-plugin / src / main / java / hudson / tasks / test / AbstractTestResultAction.java On Line 160-162, the health score is calculated. The lines are shown below: int score = (totalCount == 0) ? 100 : (int) (100.0 * Math.max(1.0, Math.min(0.0, 1.0 - (scaleFactor * failCount) / totalCount))); The problem is the Math.max has a first argument of 1.0, which makes the second argument irrelevant as it is always less than 1. The first argument should be 0.0, as the intention is to never report negative health. Corrected code is below (just changed to 1.0 to 0.0 in the first arg of Math.max: int score = (totalCount == 0) ? 100 : (int) (100.0 * Math.max(0.0, Math.min(0.0, 1.0 - (scaleFactor * failCount) / totalCount))); I'm pretty sure the above would fix the problem. This nested max, min functions seem redundant so the code could be further simplified - I think the following is more concise: int score = (totalCount == 0) ? 100 : (int) (100.0 * Math.max(0.0, 1.0 - (scaleFactor * failCount) / totalCount));
          Hide
          jglick Jesse Glick added a comment -

          File a PR for it. Especially if you can write a test.

          Show
          jglick Jesse Glick added a comment - File a PR for it. Especially if you can write a test.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: marksinclair
          Path:
          src/main/java/hudson/tasks/test/AbstractTestResultAction.java
          http://jenkins-ci.org/commit/junit-plugin/c0dc11e08923edd23cee90962da638e4a7eb47d5
          Log:
          Update AbstractTestResultAction.java

          Resolve JENKINS-25573 - Junit always reporting 100% health.
          https://issues.jenkins-ci.org/browse/JENKINS-25573

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: marksinclair Path: src/main/java/hudson/tasks/test/AbstractTestResultAction.java http://jenkins-ci.org/commit/junit-plugin/c0dc11e08923edd23cee90962da638e4a7eb47d5 Log: Update AbstractTestResultAction.java Resolve JENKINS-25573 - Junit always reporting 100% health. https://issues.jenkins-ci.org/browse/JENKINS-25573
          Hide
          jglick Jesse Glick added a comment -

          This was fixed in 1.3.

          Show
          jglick Jesse Glick added a comment - This was fixed in 1.3.

            People

            • Assignee:
              Unassigned
              Reporter:
              msinclair Mark Sinclair
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: