Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45014

Developer is able to see which tests are intermittent

XMLWordPrintable

      Feature request: 

      If tests in a test suite often flick between failure and pass, it would be nice to report on this right in the test results screen in blue ocean. 

       

      https://wiki.jenkins-ci.org/display/JENKINS/Flaky+Test+Handler+Plugin shows some of the functionality, but this is not about rerunning bad things, but just identifying that "often this test fails, but then passes in a subsequent run when there were no code changes. 

      A fairly simple heuristic could be used - doens't necessarily need to use stats over many builds. If any test failed, but then passed due a subsequent run when there were NO CODE changes, mark it as flaky. This could also be applicable across branches (ie use the default branch mainly). 

       

      This would mean when you see failures, you know that you can run it again, but also that you really should get your shit together and fix that test somehow. 

            Unassigned Unassigned
            michaelneale Michael Neale
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: