JenkinsIcon indicating the project type

Jenkins

Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6446

Unit test rules should be symmetric

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Currently the failed test rule and the passed test rule are not symmetric.
      The failed test rule only gives points only for new failed test and the passed test rule only for new passed tests.
      I.e. skipped test are lost along the way.
      Example:
      Build n : SUCCESS, 610 passing tests
      Build n+1: UNSTABLE, 71 passing, 67 failed, 2963 skipped tests (TestNG has the strange behaviour that when the init of some test classes fails, the skipped count sky-rockets)
      Build n+2: SUCCESS, 610 passing tests

      Build n+1 gives 71 minus points
      Build n+2 gives 539 plus points

      So, by breaking the build and fixing it again you can 'unrightfully' get lots of points

        Attachments

          Activity

          Descending order - Click to sort in ascending order
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: simschla
          Path:
          pom.xml
          src/main/java/hudson/plugins/cigame/GameDescriptor.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractSkippedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractUnitTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingSkippedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingSkippedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/RemovedFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/RemovedPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/UnitTestingRuleSet.java
          src/main/resources/hudson/plugins/cigame/GamePublisher/global.jelly
          src/main/resources/hudson/plugins/cigame/GamePublisher/global.properties
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-failedTestDecreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-failedTestIncreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-passedTestDecreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-passedTestIncreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-skippedTestDecreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-skippedTestIncreasingPoints.html
          src/main/resources/hudson/plugins/cigame/rules/unittesting/Messages.properties
          src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRuleTest.java
          src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRuleTest.java
          src/test/java/hudson/plugins/cigame/rules/unittesting/MavenMultiModuleUnitTestsTest.java
          http://jenkins-ci.org/commit/ci-game-plugin/c66201634b4310a7e1bb157c36026631093b22c1
          Log:
          Merge pull request #11 from lkamal/master

          [FIXED JENKINS-6446] added global configurations for all the six types of unittest changes (increase or descrease of passed, failed and skipped tests)

          Compare: https://github.com/jenkinsci/ci-game-plugin/compare/321ce5b740db...c66201634b43

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: simschla Path: pom.xml src/main/java/hudson/plugins/cigame/GameDescriptor.java src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractSkippedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractUnitTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingSkippedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingSkippedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/RemovedFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/RemovedPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/UnitTestingRuleSet.java src/main/resources/hudson/plugins/cigame/GamePublisher/global.jelly src/main/resources/hudson/plugins/cigame/GamePublisher/global.properties src/main/resources/hudson/plugins/cigame/GamePublisher/help-failedTestDecreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-failedTestIncreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-passedTestDecreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-passedTestIncreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-skippedTestDecreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-skippedTestIncreasingPoints.html src/main/resources/hudson/plugins/cigame/rules/unittesting/Messages.properties src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRuleTest.java src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRuleTest.java src/test/java/hudson/plugins/cigame/rules/unittesting/MavenMultiModuleUnitTestsTest.java http://jenkins-ci.org/commit/ci-game-plugin/c66201634b4310a7e1bb157c36026631093b22c1 Log: Merge pull request #11 from lkamal/master [FIXED JENKINS-6446] added global configurations for all the six types of unittest changes (increase or descrease of passed, failed and skipped tests) Compare: https://github.com/jenkinsci/ci-game-plugin/compare/321ce5b740db...c66201634b43
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: lkamal
          Path:
          src/main/resources/hudson/plugins/cigame/GamePublisher/global.jelly
          http://jenkins-ci.org/commit/ci-game-plugin/57b43778dfffa7b1dec25dcb6123b40c2993e195
          Log:
          [FIXED JENKINS-6446] Modified the global config view of unit tests so that it will be displayed in a separate table rather than using a top level title

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: lkamal Path: src/main/resources/hudson/plugins/cigame/GamePublisher/global.jelly http://jenkins-ci.org/commit/ci-game-plugin/57b43778dfffa7b1dec25dcb6123b40c2993e195 Log: [FIXED JENKINS-6446] Modified the global config view of unit tests so that it will be displayed in a separate table rather than using a top level title
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: lkamal
          Path:
          pom.xml
          src/main/java/hudson/plugins/cigame/GameDescriptor.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingSkippedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingSkippedTestsRule.java
          src/main/resources/hudson/plugins/cigame/GamePublisher/global.jelly
          src/main/resources/hudson/plugins/cigame/GamePublisher/global.properties
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-failedTestDecreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-failedTestIncreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-passedTestDecreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-passedTestIncreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-skippedTestDecreasingPoints.html
          src/main/resources/hudson/plugins/cigame/GamePublisher/help-skippedTestIncreasingPoints.html
          src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRuleTest.java
          src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRuleTest.java
          src/test/java/hudson/plugins/cigame/rules/unittesting/MavenMultiModuleUnitTestsTest.java
          http://jenkins-ci.org/commit/ci-game-plugin/626af1ccd3ce3eee662f9a567b114c4cc095fdd3
          Log:
          [FIXED JENKINS-6446] Added global configurations for all the six types of unittest changes (increase or descrease of passed, failed and skipped tests). The configuration is available under 'Continuous integration game - Unit Tests' section in advanced mode. Default values are set to 1 for increasing passing tests, -1 for increasing failed tests and 0 for all other 4 scenarios

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: lkamal Path: pom.xml src/main/java/hudson/plugins/cigame/GameDescriptor.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingSkippedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingSkippedTestsRule.java src/main/resources/hudson/plugins/cigame/GamePublisher/global.jelly src/main/resources/hudson/plugins/cigame/GamePublisher/global.properties src/main/resources/hudson/plugins/cigame/GamePublisher/help-failedTestDecreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-failedTestIncreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-passedTestDecreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-passedTestIncreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-skippedTestDecreasingPoints.html src/main/resources/hudson/plugins/cigame/GamePublisher/help-skippedTestIncreasingPoints.html src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRuleTest.java src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRuleTest.java src/test/java/hudson/plugins/cigame/rules/unittesting/MavenMultiModuleUnitTestsTest.java http://jenkins-ci.org/commit/ci-game-plugin/626af1ccd3ce3eee662f9a567b114c4cc095fdd3 Log: [FIXED JENKINS-6446] Added global configurations for all the six types of unittest changes (increase or descrease of passed, failed and skipped tests). The configuration is available under 'Continuous integration game - Unit Tests' section in advanced mode. Default values are set to 1 for increasing passing tests, -1 for increasing failed tests and 0 for all other 4 scenarios
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: lkamal
          Path:
          src/main/java/hudson/plugins/cigame/rules/build/BuildResultRule.java
          http://jenkins-ci.org/commit/ci-game-plugin/e796a32831959f0e22490a5718eba71b11611e3f
          Log:
          [FIXED JENKINS-6446] Removing a TODO added for future references

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: lkamal Path: src/main/java/hudson/plugins/cigame/rules/build/BuildResultRule.java http://jenkins-ci.org/commit/ci-game-plugin/e796a32831959f0e22490a5718eba71b11611e3f Log: [FIXED JENKINS-6446] Removing a TODO added for future references
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: lkamal
          Path:
          src/main/java/hudson/plugins/cigame/rules/build/BuildResultRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractSkippedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractUnitTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingSkippedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingSkippedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/RemovedFailedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/RemovedPassedTestsRule.java
          src/main/java/hudson/plugins/cigame/rules/unittesting/UnitTestingRuleSet.java
          src/main/resources/hudson/plugins/cigame/rules/unittesting/Messages.properties
          src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRuleTest.java
          src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRuleTest.java
          src/test/java/hudson/plugins/cigame/rules/unittesting/MavenMultiModuleUnitTestsTest.java
          http://jenkins-ci.org/commit/ci-game-plugin/081f250e4d42cf5f2e6194f57f6e9d0cbf47d9c2
          Log:
          [FIXED JENKINS-6446] Modified the scoring mechanism as follows to avoid the inconsistency in granting marks.
          Passing test
          added: +1
          removed: -1
          Failing test
          added: -1
          removed: +1
          Skipped test (no marks given or reduced)
          added: 0
          removed: 0

          Following is the details of how all scenarios are handled.

          Passing Tests
          Passing test added = +1
          [Increases passing count = +1]
          Passing test removed = -1
          [Decreases passing count = -1]
          Passing an already failing test = +2
          [Decreases failing count = +1]
          [Increases passing count = +1]
          Passing an already skipped test = +1
          [Decreases skipped count = 0]
          [Increases passing count = +1]

          Failing tests
          Failing test added = -1
          [Increases failing count = -1]
          Failing test removed = +1
          [Decreases failing count = +1]
          Failing an already passing test = -2
          [Decreases passing count = -1]
          [Increases failing count = -1]
          Failing an already skipped test = -1
          [Decreases skipped count = 0]
          [Increases failing count = -1]

          Skipped tests
          Skipped test added = 0
          [Increases skipped count = 0]
          Skipped test removed = 0
          [Decreases skipped count = 0]
          Skipped an already passing test = -1
          [Decreases passing count = -1]
          [Increases skipped count = 0]
          Skipped an already failing test = +1
          [Decreases failing count = +1]
          [Increases skipped count = 0]

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: lkamal Path: src/main/java/hudson/plugins/cigame/rules/build/BuildResultRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractSkippedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/AbstractUnitTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/DecreasingSkippedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/IncreasingSkippedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/RemovedFailedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/RemovedPassedTestsRule.java src/main/java/hudson/plugins/cigame/rules/unittesting/UnitTestingRuleSet.java src/main/resources/hudson/plugins/cigame/rules/unittesting/Messages.properties src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingFailedTestsRuleTest.java src/test/java/hudson/plugins/cigame/rules/unittesting/IncreasingPassedTestsRuleTest.java src/test/java/hudson/plugins/cigame/rules/unittesting/MavenMultiModuleUnitTestsTest.java http://jenkins-ci.org/commit/ci-game-plugin/081f250e4d42cf5f2e6194f57f6e9d0cbf47d9c2 Log: [FIXED JENKINS-6446] Modified the scoring mechanism as follows to avoid the inconsistency in granting marks. Passing test added: +1 removed: -1 Failing test added: -1 removed: +1 Skipped test (no marks given or reduced) added: 0 removed: 0 Following is the details of how all scenarios are handled. Passing Tests Passing test added = +1 [Increases passing count = +1] Passing test removed = -1 [Decreases passing count = -1] Passing an already failing test = +2 [Decreases failing count = +1] [Increases passing count = +1] Passing an already skipped test = +1 [Decreases skipped count = 0] [Increases passing count = +1] Failing tests Failing test added = -1 [Increases failing count = -1] Failing test removed = +1 [Decreases failing count = +1] Failing an already passing test = -2 [Decreases passing count = -1] [Increases failing count = -1] Failing an already skipped test = -1 [Decreases skipped count = 0] [Increases failing count = -1] Skipped tests Skipped test added = 0 [Increases skipped count = 0] Skipped test removed = 0 [Decreases skipped count = 0] Skipped an already passing test = -1 [Decreases passing count = -1] [Increases skipped count = 0] Skipped an already failing test = +1 [Decreases failing count = +1] [Increases skipped count = 0]
          11 older comments

            People

            • Assignee:
              lkamal Kamal Mettananda
              Reporter:
              kutzi kutzi
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: