Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20985

TestNG plugin should has a possibility to use thresholds for failed tests

    Details

    • Type: New Feature
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: testng-plugin
    • Labels:
      None

      Description

      Current version of TestNG plugin does not have a possibility to use threshold values for failed and skipped tests. If some build has at least one failed test, then this build will be marked as unstable.
      Maybe it would be better to have some mechanism to mark build as successfull instead of unstable with using a threshold value for failed tests.

        Activity

        Hide
        ndriuchatyi Nikolay Driuchatyi added a comment -

        I have several jobs with different number of skipped and failed tests on each of them. I need to have a possibility to set an acceptable number of the failed and skipped tests. When number of skipped/failed tests is more than threshold value then this job should be set to unstable/failed. This is my point of view. Also would be good to have different strategies e.g. to have a possibility to set number or percentage of the skipped/failed tests depending on requirements.

        Show
        ndriuchatyi Nikolay Driuchatyi added a comment - I have several jobs with different number of skipped and failed tests on each of them. I need to have a possibility to set an acceptable number of the failed and skipped tests. When number of skipped/failed tests is more than threshold value then this job should be set to unstable/failed. This is my point of view. Also would be good to have different strategies e.g. to have a possibility to set number or percentage of the skipped/failed tests depending on requirements.
        Hide
        nullin Nalin Makar added a comment -

        ok. Please submit a pull request. Lets make sure that the configuration is similar to that in xUnit plugin. Thanks.

        Show
        nullin Nalin Makar added a comment - ok. Please submit a pull request. Lets make sure that the configuration is similar to that in xUnit plugin. Thanks.
        Hide
        ndriuchatyi Nikolay Driuchatyi added a comment -

        Hello guys, are there any updates?

        Show
        ndriuchatyi Nikolay Driuchatyi added a comment - Hello guys, are there any updates?
        Hide
        gatrot gatrot added a comment -

        I lost some steam after the back and forth in January. I'll pick it up again next week and send out a pull request.

        Show
        gatrot gatrot added a comment - I lost some steam after the back and forth in January. I'll pick it up again next week and send out a pull request.
        Hide
        navels Lee Nave added a comment -

        Any update to this?

        Show
        navels Lee Nave added a comment - Any update to this?

          People

          • Assignee:
            gavingray Gavin Gray
            Reporter:
            ndriuchatyi Nikolay Driuchatyi
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated: