Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20985

TestNG plugin should has a possibility to use thresholds for failed tests

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Component/s: testng-plugin
    • Labels:
      None
    • Similar Issues:
      Show 5 results

      Description

      Current version of TestNG plugin does not have a possibility to use threshold values for failed and skipped tests. If some build has at least one failed test, then this build will be marked as unstable.
      Maybe it would be better to have some mechanism to mark build as successfull instead of unstable with using a threshold value for failed tests.

        Issue Links

          Activity

          Hide
          raksingh rakesh singh added a comment -

          Is there any update is available to mark build as Stable if some configuration test cases failed ?

          Show
          raksingh rakesh singh added a comment - Is there any update is available to mark build as Stable if some configuration test cases failed ?
          Show
          nullin Nalin Makar added a comment - https://github.com/jenkinsci/testng-plugin-plugin/commit/52a4637968ea1407ae09be257992e4c0ef4d3c41 merged pull #20.
          Hide
          cwcam Cam Spencer added a comment -

          Hey nalin_makar, I don't think the default threshold value should be 100%. We updated the plugin without noticing this feature, and this was causing tests that failed to show as stable.

          Show
          cwcam Cam Spencer added a comment - Hey nalin_makar , I don't think the default threshold value should be 100%. We updated the plugin without noticing this feature, and this was causing tests that failed to show as stable.
          Hide
          nullin Nalin Makar added a comment -

          Cam Spencer, can you please file a separate bug? And, it'll be great if you can give specific information about the properties you are referring to. Thanks!

          Show
          nullin Nalin Makar added a comment - Cam Spencer , can you please file a separate bug? And, it'll be great if you can give specific information about the properties you are referring to. Thanks!
          Hide
          chriseverling Chris Everling added a comment -

          Cam Spencer, nalin_makar, I don't think that is a bug. The default behavior before the update was to mark the build as UNSTABLE whenever there were test failures. The default failure threshold to FAIL the build is 100%, but the default failure threshold to mark the build as UNSTABLE is 0. The default behavior should not have changed with the update.

          Show
          chriseverling Chris Everling added a comment - Cam Spencer , nalin_makar , I don't think that is a bug. The default behavior before the update was to mark the build as UNSTABLE whenever there were test failures. The default failure threshold to FAIL the build is 100%, but the default failure threshold to mark the build as UNSTABLE is 0. The default behavior should not have changed with the update.

            People

            • Assignee:
              gavingray Gavin Gray
              Reporter:
              ndriuchatyi Nikolay Driuchatyi
            • Votes:
              3 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: