Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26000

Invalid processing of suite-files in testng

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      There is a problem while retrieving results for testng when the testng XML file contains the constructions <suite-file path="./suite1.xml"/> with multiple suites.
      Here are the details.

      At first, this is the example that works OK:
      <suite name="TAM TestSuite">
      <test name="MRCT1100"><classes><class name="com.ingenico.tam.testsuite.merchants.MRCT1101"></class></classes></test>
      <test name="SCNWZD1100"><classes><class name="com.ingenico.tam.testsuite.scenariowizard.SCNWZD1100"></class></classes></test>
      </suite>
      Here there are two tests, the first one is successful, the second one fails. In this format everything is OK, see the screenshot jenkins-ok.png.
      I attach also the file testng-results.ok for this case.

      Next, I do the same tests java with the same confuguration of testlink and plugin. I replace only the testng XML file by three files (attached): All.xml, TAMMerchants.xml, TAMScenarioWizard.xml.
      The tests are the same, but they are organized in separate files:
      <suite name="TAM TestSuite" parallel="false" thread-count="1" preserve-order="true">
      <suite-files>
      <suite-file path="./TAMMerchants.xml"/>
      <suite-file path="./TAMScenarioWizard.xml"/>
      </suite-files>
      </suite>
      In this case both tests are executed by testng, but the problem appears when the plugin retrieves the result, see the screenshot jenkins-ko.PNG attached.
      The test from TAMScenarioWizard.xml is Not Run.
      However it was executed, see the file testng-results.ko attached.

      In TestLink also:

      • for the first case the test MRCT1100 is successful, and SCNWZD1100 failes
      • in the second case the test MRCT1100 is successful, and SCNWZD1100 Not Run

      It seems that there is a problem when processing testng-results.xml: in first case it contains only one suite ,and it works, and in the seconde case there are three suite tags, and only one result is retrieved.

      I join also the file config.xml

        Attachments

        1. All.xml
          0.3 kB
        2. config.xml
          4 kB
        3. jenkins-ko.PNG
          jenkins-ko.PNG
          25 kB
        4. jenkins-ok.PNG
          jenkins-ok.PNG
          24 kB
        5. ko62.xml
          18 kB
        6. ok63.xml
          21 kB
        7. TAMMerchants.xml
          0.8 kB
        8. TAMScenarioWizard.xml
          1.0 kB
        9. testng-results.ko
          3 kB
        10. testng-results.ok
          4 kB

          Activity

          Hide
          ab_ing A B added a comment - - edited

          The file ko62.xml is the XML RPC exchange between the plugin and testlink for build 62 where the second test is not run

          Show
          ab_ing A B added a comment - - edited The file ko62.xml is the XML RPC exchange between the plugin and testlink for build 62 where the second test is not run
          Hide
          ab_ing A B added a comment -

          The file ok63.xml is the XML RPC exchange between the plugin and testlink for build 63 where the second test is present

          Show
          ab_ing A B added a comment - The file ok63.xml is the XML RPC exchange between the plugin and testlink for build 63 where the second test is present
          Hide
          ab_ing A B added a comment -

          A added two more files: ko62.xml and ok63.xml containing the XML RPC messages exchanged between the plugin and testlink.
          The main difference is at the end:

          • ko62.xml: the plugin sends back only one test case result
          • ok63.xml: the plugin sends back two test case results

          the builds 62 and 63 use the same set of test cases, but for 62 they are structured into suites, and in 63 they are in the same suite.

          Show
          ab_ing A B added a comment - A added two more files: ko62.xml and ok63.xml containing the XML RPC messages exchanged between the plugin and testlink. The main difference is at the end: ko62.xml: the plugin sends back only one test case result ok63.xml: the plugin sends back two test case results the builds 62 and 63 use the same set of test cases, but for 62 they are structured into suites, and in 63 they are in the same suite.
          Hide
          neril aurelien dubois added a comment -

          Hello.

          I have exactly the same problem described below
          Jenkins Version : 1.613
          Plugin Version : 3.10

          A correction time is it planned?

          Show
          neril aurelien dubois added a comment - Hello. I have exactly the same problem described below Jenkins Version : 1.613 Plugin Version : 3.10 A correction time is it planned?

            People

            • Assignee:
              kinow Bruno P. Kinoshita
              Reporter:
              ab_ing A B
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: