-
Bug
-
Resolution: Unresolved
-
Major
-
None
When building an incremental build maven job, tests of modules that are skipped are ignored:
Module A: 5 Tests
Moudle B: 3 Tests
Full Build: 8 Tests run
Incremental Build Module A: 5 Tests run (-3)
Full Build: 8 Tests run (3 new Tests)
Clearly, at least the 3 new Tests are plainly wrong.
This leads to misleading information in test-graphs as well as a wrong score in the CI-Game (initator of the second full build is awarded points for creating 3 new tests).
This issue is somewhat related to JENKINS-6122, it has already been reported in issue JENKINS-4243 (as an afterthought in the descripton), however this issue has been marked duplicate to an issue which only adressed the first part.
There would be 2 possible solutions:
1: simply count the results from the last actual run of the module
2: as 1, but include the count of the current build separately:
4 failures, Tests run (8 +-0, 2 from previous builds),
one could give them a different color in the test-bar (i.e. red: failed tests, blue: passed tests that actually did run in this build, grey: tests that passed in a previous build an were not rerun due to a skipped module)
This is only an issue for passed tests, since modules with failing tests are never skipped.