Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24006

Customize number of Junit test failures that affect weather

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: junit-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      Currently if 20% or more of tests fail, the weather for a project will be downgraded (cloudy).

      Some users have a large number of tests and one test failure is a major issue.

      The request is to allow customization of the % of Junit test failures that will cause a weather downgrade. Users that specify 100% pass require all tests to pass to maintain sunny weather.

        Attachments

          Activity

          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/java/hudson/tasks/junit/JUnitResultArchiver.java
          src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly
          src/test/java/hudson/tasks/junit/JUnitResultArchiverTest.java
          src/test/resources/hudson/tasks/junit/JUnitResultArchiverTest/MockTestDataPublisher/config.jelly
          http://jenkins-ci.org/commit/junit-plugin/e6416e34c0987a839d3dc2a34938dd1d2ac0d035
          Log:
          [FIXED JENKINS-24006] Changes to healthScaleFactor were not actually being saved.
          Ripping out the archaic manual form binding code and replacing with the modern simplified scheme.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/tasks/junit/JUnitResultArchiver.java src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly src/test/java/hudson/tasks/junit/JUnitResultArchiverTest.java src/test/resources/hudson/tasks/junit/JUnitResultArchiverTest/MockTestDataPublisher/config.jelly http://jenkins-ci.org/commit/junit-plugin/e6416e34c0987a839d3dc2a34938dd1d2ac0d035 Log: [FIXED JENKINS-24006] Changes to healthScaleFactor were not actually being saved. Ripping out the archaic manual form binding code and replacing with the modern simplified scheme.
          Hide
          msinclair Mark Sinclair added a comment -

          It appears I'm not getting an accurate health report out of Junit.

          I am running Junit version 1.2 (the health amplification factor fix is in).

          The problem I am seeing is I don't see the weather report changing, it is always reporting 100% from Junit. I tried different settings the amplification factor, but get the same result, always 100% health. I created a job with only 1 test in it, Junit reports the test as failing, and it shows up in my trend graph. Everything else with Junit seems to be working.

          The weather is reported sunny with 1 test failing out of a total of 1 test 100% health. The build is reported as unstable, but the weather is sunny. I expect the health/weather to be 0% from Junit because 100% of the tests are failing.

          Show
          msinclair Mark Sinclair added a comment - It appears I'm not getting an accurate health report out of Junit. I am running Junit version 1.2 (the health amplification factor fix is in). The problem I am seeing is I don't see the weather report changing, it is always reporting 100% from Junit. I tried different settings the amplification factor, but get the same result, always 100% health. I created a job with only 1 test in it, Junit reports the test as failing, and it shows up in my trend graph. Everything else with Junit seems to be working. The weather is reported sunny with 1 test failing out of a total of 1 test 100% health. The build is reported as unstable, but the weather is sunny. I expect the health/weather to be 0% from Junit because 100% of the tests are failing.
          Hide
          jglick Jesse Glick added a comment -

          Mark Sinclair I would suggest you file a separate bug and mark it blocking this one using a JIRA link. I know nothing about the intended behavior of healthScaleFactory. I was merely fixing the problem in its configuration form.

          Show
          jglick Jesse Glick added a comment - Mark Sinclair I would suggest you file a separate bug and mark it blocking this one using a JIRA link. I know nothing about the intended behavior of healthScaleFactory . I was merely fixing the problem in its configuration form.
          Hide
          msinclair Mark Sinclair added a comment -

          Thanks Jesse,

          I also put up a question on google groups to see if I'm doing something wrong in my xml generation:
          https://groups.google.com/d/topic/jenkinsci-users/-StbbZ0s8jI/discussion

          Show
          msinclair Mark Sinclair added a comment - Thanks Jesse, I also put up a question on google groups to see if I'm doing something wrong in my xml generation: https://groups.google.com/d/topic/jenkinsci-users/-StbbZ0s8jI/discussion
          Hide
          msinclair Mark Sinclair added a comment -

          The always 100% health report issue is tracked by the following:
          https://issues.jenkins-ci.org/browse/JENKINS-25573

          Show
          msinclair Mark Sinclair added a comment - The always 100% health report issue is tracked by the following: https://issues.jenkins-ci.org/browse/JENKINS-25573

            People

            • Assignee:
              jglick Jesse Glick
              Reporter:
              msinclair Mark Sinclair
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: