Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51341

Suite name is not shown on the main page and method results page

    Details

    • Similar Issues:

      Description

      I use TestNG for integration testing, and I happen to use the same test class/method across multiple test suites.

      That is I have lots of xml files that have different parameter values.

      The problem is current results page lists package.Class.method multiple times, and it is extremely hard to tell which suites actually failed.

      I wish the results page shown not just package, class, and method names, but test suite names (and/or "test" names that come from <test tag).

       

      Note: currently the plugin just lists all the failed methods, so if I have 10 methods across 20 suites, it would show 200 rows with repeating test names.

       

      Frankly speaking, I would like to have those 200 items grouped into something like:

      The following tests are failed in "testsuite 1", "testsuite 2", and "testsuite 3":   "method1", "method2", "method3".

      That would make test results shorter, however it would "break" ordering. I'm not sure if current order "by the order in results xml file" is important or not.

       

      TL;DR:

      1) Either (preferred for me) group the tests that fail in multiple suites into a single record. For instance:

      The following tests are failed in "testsuite 1", "testsuite 2", and "testsuite 3":   "method1", "method2", "method3".

      2) Or add "testsuite name" as suite name changes. For instance:

      suite1
      test1
      test2
      suite2
      test1
      test2 

       

        Attachments

        1. 1of3.png
          1of3.png
          176 kB
        2. 2of3.png
          2of3.png
          184 kB
        3. 3of3.png
          3of3.png
          191 kB
        4. failed_configs.png
          failed_configs.png
          254 kB
        5. failed_configs.png
          failed_configs.png
          254 kB
        6. overall.png
          overall.png
          494 kB

          Activity

          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment -

          Nalin Makar, As far as I can see, plugin wiki page is marked with adopt-this-plugin. Is that up to date? Are you actively maintaining the plugin?

          I think I can implement the fix on my own, however it is not clear what would be the release procedure.

          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - Nalin Makar , As far as I can see, plugin wiki page is marked with adopt-this-plugin. Is that up to date? Are you actively maintaining the plugin? I think I can implement the fix on my own, however it is not clear what would be the release procedure.
          Hide
          nullin Nalin Makar added a comment - - edited

          You are right, this plugin is up for adoption. But, if you make a pull request, I can help release it. Please add tests for your changes.

          And, can you add a screenshot of the problem you are encountering? Reading from the description, I would lean towards solution 2, but it'll be useful to understand what does that mean as an overall experience for users.

          Show
          nullin Nalin Makar added a comment - - edited You are right, this plugin is up for adoption. But, if you make a pull request, I can help release it. Please add tests for your changes. And, can you add a screenshot of the problem you are encountering? Reading from the description, I would lean towards solution 2, but it'll be useful to understand what does that mean as an overall experience for users.
          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment -

          There are quite a few tests:
          And it turns out certain test is listed 3 times:
          , , and

          Note: those are different executions of the same test method (there are just multiple suite xml files that list the same method name with different suite parameters), so the result file is parsed properly. It is just representation that makes the whole thing hard to reason about.

          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - There are quite a few tests: And it turns out certain test is listed 3 times: , , and Note: those are different executions of the same test method (there are just multiple suite xml files that list the same method name with different suite parameters), so the result file is parsed properly. It is just representation that makes the whole thing hard to reason about.
          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment -

          A side note: I would move "Failed Configurations" to the top (right after "total tests"), as it is likely to be more important than "skipped/failed" tests. "Failed configurations" might cause tests to be skipped not the other way around.

           

          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - A side note: I would move "Failed Configurations" to the top (right after "total tests"), as it is likely to be more important than "skipped/failed" tests. "Failed configurations" might cause tests to be skipped not the other way around.  
          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment -
          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - Fixed in https://github.com/jenkinsci/testng-plugin-plugin/pull/36 Nalin Makar , would you please check that out?
          Hide
          nullin Nalin Makar added a comment -

          released in v1.15

          Show
          nullin Nalin Makar added a comment - released in v1.15

            People

            • Assignee:
              nullin Nalin Makar
              Reporter:
              vladimirsitnikov Vladimir Sitnikov
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: