Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18095

nose generated xunit reports are rejected by xunit 1.59

    Details

    • Similar Issues:

      Description

      After upgrading xunit to 1.59, all nosetests generated xunit reports are rejected.
      Seems xml validation got more strict, but the console output does not help explain what's wrong...

        Attachments

          Activity

          Hide
          simonjohansson Simon Johansson added a comment -

          I seem to have the same issue, I generate XML like so;
          "nosetests --with-xunit"

          Runnig post-build action "Publish JUnit test result report" over the file works fine, but using "Publish xUnit test result report" with JUnit patters I get the following error

          [xUnit] [INFO] - Starting to record.
          [xUnit] [INFO] - Processing JUnit
          [xUnit] [INFO] - [JUnit] - 1 test report file(s) were found with the pattern '**/nosetests.xml' relative to '/var/lib/jenkins/jenkins-data/jobs/Skuscraper/workspace' for the testing framework 'JUnit'.
          [xUnit] [ERROR] - The result file '/var/lib/jenkins/jenkins-data/jobs/Skuscraper/workspace/skuscraper/nosetests.xml' for the metric 'JUnit' is not valid. The result file has been skipped.
          [xUnit] [INFO] - Fail BUILD because 'set build failed if errors' option is activated.
          [xUnit] [INFO] - There are errors when processing test results.
          [xUnit] [INFO] - Skipping tests recording.
          [xUnit] [INFO] - Stop build.

          I have attached my nosetests.xml.

          Show
          simonjohansson Simon Johansson added a comment - I seem to have the same issue, I generate XML like so; "nosetests --with-xunit" Runnig post-build action "Publish JUnit test result report" over the file works fine, but using "Publish xUnit test result report" with JUnit patters I get the following error [xUnit] [INFO] - Starting to record. [xUnit] [INFO] - Processing JUnit [xUnit] [INFO] - [JUnit] - 1 test report file(s) were found with the pattern '**/nosetests.xml' relative to '/var/lib/jenkins/jenkins-data/jobs/Skuscraper/workspace' for the testing framework 'JUnit'. [xUnit] [ERROR] - The result file '/var/lib/jenkins/jenkins-data/jobs/Skuscraper/workspace/skuscraper/nosetests.xml' for the metric 'JUnit' is not valid. The result file has been skipped. [xUnit] [INFO] - Fail BUILD because 'set build failed if errors' option is activated. [xUnit] [INFO] - There are errors when processing test results. [xUnit] [INFO] - Skipping tests recording. [xUnit] [INFO] - Stop build. I have attached my nosetests.xml.
          Hide
          gmarchin George Marchin added a comment -

          See this:
          https://github.com/jenkinsci/xunit-plugin/compare/xunit-1.58...xunit-1.59

          I would appear that the JUnit model validator was silently changed from 4 to 5.

          JUnit 4 and JUnit 5 should be added as seperate xunit file types in the configuration.

          Show
          gmarchin George Marchin added a comment - See this: https://github.com/jenkinsci/xunit-plugin/compare/xunit-1.58...xunit-1.59 I would appear that the JUnit model validator was silently changed from 4 to 5. JUnit 4 and JUnit 5 should be added as seperate xunit file types in the configuration.
          Hide
          dshipp David Shipp added a comment -

          My Python project also has the same issue with nosetests output, which was working prior to a recent update.

          Show
          dshipp David Shipp added a comment - My Python project also has the same issue with nosetests output, which was working prior to a recent update.
          Hide
          mlua Magnus Lundin added a comment -

          Same problem with "mocha -R xunit". At least with 1.59, haven't tried 1.60 yet.

          Show
          mlua Magnus Lundin added a comment - Same problem with "mocha -R xunit". At least with 1.59, haven't tried 1.60 yet.
          Hide
          stengaard Brian Stengaard added a comment -

          Seen this one on 1.60 as well. Rolled back to 1.58 as a workaround.

          Show
          stengaard Brian Stengaard added a comment - Seen this one on 1.60 as well. Rolled back to 1.58 as a workaround.
          Hide
          rb2k Marc Seeger added a comment - - edited

          Same here. XML created by the CI Reporter gem (https://github.com/nicksieger/ci_reporter/)

          Show
          rb2k Marc Seeger added a comment - - edited Same here. XML created by the CI Reporter gem ( https://github.com/nicksieger/ci_reporter/ )
          Hide
          grandvizier Jeff K added a comment - - edited

          same problem here as well (using xunit-file module: https://github.com/peerigon/xunit-file)

          I noticed that a few of the things that are making it fail are the 'skip' attribute in the testsuite element, and 'message' attribute on the testcase element, and possibly the 'failure' child element to the testcase element (this last issue might be more specific to the format of the failure element and not just the failure element)

          this became an issue when upgrading to v1.60 (haven't tried rolling back to v1.58 yet)

          Show
          grandvizier Jeff K added a comment - - edited same problem here as well (using xunit-file module: https://github.com/peerigon/xunit-file ) I noticed that a few of the things that are making it fail are the 'skip' attribute in the testsuite element, and 'message' attribute on the testcase element, and possibly the 'failure' child element to the testcase element (this last issue might be more specific to the format of the failure element and not just the failure element) this became an issue when upgrading to v1.60 (haven't tried rolling back to v1.58 yet)
          Hide
          pitr Piotr Kral added a comment -

          Rollback from 1.60 to 1.58 fixed it for me, but it would be nice to have it working

          Show
          pitr Piotr Kral added a comment - Rollback from 1.60 to 1.58 fixed it for me, but it would be nice to have it working
          Hide
          kinow Bruno P. Kinoshita added a comment -

          Hi all, chiming in.

          The issue is due to a validation error. From what I understood, the plug-in uses DTKit framework and a set of XSD's (and maybe other validation schemas) to validate the input files.

          In the case of the nosetests.xml file, I used the plug-in version from GitHub, branch master. Debugging the plug-in looks like it uses junit-7.xsd [1] from DTKit.

          If you are curious, copy the contents of nosetests.xml attached to this issue, and the junit-7.xsd contents, and paste them at http://www.utilities-online.info/xsdvalidation

          The following error message is displayed:

          Not valid.
          Error - Line 1, 110: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 110; cvc-complex-type.3.2.2: Attribute 'skip' is not allowed to appear in element 'testsuite'.

          I'll take a look at the DTKit to see if it's possible to send a pull request to its repository.

          Cheers
          /Bruno

          [1] https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-7.xsd

          Show
          kinow Bruno P. Kinoshita added a comment - Hi all, chiming in. The issue is due to a validation error. From what I understood, the plug-in uses DTKit framework and a set of XSD's (and maybe other validation schemas) to validate the input files. In the case of the nosetests.xml file, I used the plug-in version from GitHub, branch master. Debugging the plug-in looks like it uses junit-7.xsd [1] from DTKit. If you are curious, copy the contents of nosetests.xml attached to this issue, and the junit-7.xsd contents, and paste them at http://www.utilities-online.info/xsdvalidation The following error message is displayed: Not valid. Error - Line 1, 110: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 110; cvc-complex-type.3.2.2: Attribute 'skip' is not allowed to appear in element 'testsuite'. I'll take a look at the DTKit to see if it's possible to send a pull request to its repository. Cheers /Bruno [1] https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-7.xsd
          Hide
          aubonbeurre Alexandre Parenteau added a comment - - edited

          This file 'com.adobe.aftereffects.fgSmoketest_V1.alpha.xml' also fails with xunit/JUnit, but the XSD validation mentioned in the thread works. The XML is hand written.

          Error message:

          [xUnit] [INFO] - [JUnit] - 5 test report file(s) were found with the pattern 'aeserver/pro/testing/*/Results//.xml' relative to 'd:\jenkins\pushjson' for the testing framework 'JUnit'.
          [xUnit] [ERROR] - The result file 'd:\jenkins\pushjson\aeserver\pro\testing\FootageGenerator\Results\JUnitLogs\com.adobe.aftereffects.fgSmoketest_V1.alpha.xml' for the metric 'JUnit' is not valid. The result file has been skipped.

          Show
          aubonbeurre Alexandre Parenteau added a comment - - edited This file 'com.adobe.aftereffects.fgSmoketest_V1.alpha.xml' also fails with xunit/JUnit, but the XSD validation mentioned in the thread works. The XML is hand written. Error message: [xUnit] [INFO] - [JUnit] - 5 test report file(s) were found with the pattern 'aeserver/pro/testing/* /Results/ / .xml' relative to 'd:\jenkins\pushjson' for the testing framework 'JUnit'. [xUnit] [ERROR] - The result file 'd:\jenkins\pushjson\aeserver\pro\testing\FootageGenerator\Results\JUnitLogs\com.adobe.aftereffects.fgSmoketest_V1.alpha.xml' for the metric 'JUnit' is not valid. The result file has been skipped.
          Hide
          kinow Bruno P. Kinoshita added a comment - - edited

          Hi @Alexandre,

          I'm using the latest code from GitHub, probably that's why your XML is working for me. See attached screenshot too. I believe your issue will be fixed when the code is released (unless you're not using the latest version). So far the issue continues to be the skip attribute in the suite element of the XML generated by nose.

          Started by user anonymous
          [xUnit] [INFO] - Starting to record.
          [xUnit] [INFO] - Processing JUnit
          [xUnit] [INFO] - [JUnit] - 1 test report file(s) were found with the pattern '*.xml' relative to '/home/kinow/java/jenkins/xunit-plugin/work/jobs/test-18095/workspace' for the testing framework 'JUnit'.
          [xUnit] [INFO] - Check 'Failed Tests' threshold.
          [xUnit] [INFO] - Check 'Skipped Tests' threshold.
          [xUnit] [INFO] - Setting the build status to SUCCESS
          [xUnit] [INFO] - Stopping recording.
          Finished: SUCCESS

          Show
          kinow Bruno P. Kinoshita added a comment - - edited Hi @Alexandre, I'm using the latest code from GitHub, probably that's why your XML is working for me. See attached screenshot too. I believe your issue will be fixed when the code is released (unless you're not using the latest version). So far the issue continues to be the skip attribute in the suite element of the XML generated by nose. Started by user anonymous [xUnit] [INFO] - Starting to record. [xUnit] [INFO] - Processing JUnit [xUnit] [INFO] - [JUnit] - 1 test report file(s) were found with the pattern '*.xml' relative to '/home/kinow/java/jenkins/xunit-plugin/work/jobs/test-18095/workspace' for the testing framework 'JUnit'. [xUnit] [INFO] - Check 'Failed Tests' threshold. [xUnit] [INFO] - Check 'Skipped Tests' threshold. [xUnit] [INFO] - Setting the build status to SUCCESS [xUnit] [INFO] - Stopping recording. Finished: SUCCESS
          Hide
          aubonbeurre Alexandre Parenteau added a comment -

          Thanks @Bruno for the quick answer! I was able to reproduce the problem with junit-6.xsd. So I guess we can revert to 1.58 xunit for now, or wait for 1.61.

          Show
          aubonbeurre Alexandre Parenteau added a comment - Thanks @Bruno for the quick answer! I was able to reproduce the problem with junit-6.xsd. So I guess we can revert to 1.58 xunit for now, or wait for 1.61.
          Hide
          dkinzer David Kinzer added a comment -

          I want to add that I am having this same issue but with JUnit xml files that are outputed by drush.

          Show
          dkinzer David Kinzer added a comment - I want to add that I am having this same issue but with JUnit xml files that are outputed by drush.
          Hide
          grandvizier Jeff K added a comment -

          has anyone tried v1.61?
          Based on the release log it doesn't look like it's been fixed, but I thought I'd ask: https://github.com/jenkinsci/xunit-plugin/compare/xunit-1.60...xunit-1.61

          Show
          grandvizier Jeff K added a comment - has anyone tried v1.61? Based on the release log it doesn't look like it's been fixed, but I thought I'd ask: https://github.com/jenkinsci/xunit-plugin/compare/xunit-1.60...xunit-1.61
          Hide
          dkinzer David Kinzer added a comment - - edited

          @Jeff K, I tried it on version 1.61. It's broken there too. I actually had to compile 1.58 from source because it wasn't available anymore. (Note if you compile from source, do not use Open JDK, use the Oracle JDK; the former method does not work).

          Show
          dkinzer David Kinzer added a comment - - edited @Jeff K, I tried it on version 1.61. It's broken there too. I actually had to compile 1.58 from source because it wasn't available anymore. (Note if you compile from source, do not use Open JDK, use the Oracle JDK; the former method does not work).
          Hide
          inbar_rose Inbar Rose added a comment -

          There is some form of "answer" as to why its happening here: https://issues.jenkins-ci.org/browse/JENKINS-17883 It seems it is hardcoded?

          Show
          inbar_rose Inbar Rose added a comment - There is some form of "answer" as to why its happening here: https://issues.jenkins-ci.org/browse/JENKINS-17883 It seems it is hardcoded?
          Hide
          pryce Adam Pryce added a comment -

          Note for others trying 1.58 or other versions: They can be downloaded from http://updates.jenkins-ci.org/download/plugins/xunit/

          Show
          pryce Adam Pryce added a comment - Note for others trying 1.58 or other versions: They can be downloaded from http://updates.jenkins-ci.org/download/plugins/xunit/
          Hide
          rb2k Marc Seeger added a comment -

          We're on 1.63 now. Any news on this?

          Show
          rb2k Marc Seeger added a comment - We're on 1.63 now. Any news on this?
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Gregory Boissinot
          Path:
          src/main/java/org/jenkinsci/plugins/xunit/types/JUnitInputMetric.java
          src/main/java/org/jenkinsci/plugins/xunit/types/model/JUnit8.java
          src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-8.xsd
          http://jenkins-ci.org/commit/xunit-plugin/15da098793691c4d5a6b1a934a83587e4594643f
          Log:
          Fix JENKINS-18095 and JENKINS-18575

          Compare: https://github.com/jenkinsci/xunit-plugin/compare/88672e87da9f...15da09879369

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Gregory Boissinot Path: src/main/java/org/jenkinsci/plugins/xunit/types/JUnitInputMetric.java src/main/java/org/jenkinsci/plugins/xunit/types/model/JUnit8.java src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-8.xsd http://jenkins-ci.org/commit/xunit-plugin/15da098793691c4d5a6b1a934a83587e4594643f Log: Fix JENKINS-18095 and JENKINS-18575 Compare: https://github.com/jenkinsci/xunit-plugin/compare/88672e87da9f...15da09879369
          Hide
          rdesgroppes Régis Desgroppes added a comment -

          Thank you for this one.

          Show
          rdesgroppes Régis Desgroppes added a comment - Thank you for this one.

            People

            • Assignee:
              gbois Gregory Boissinot
              Reporter:
              rdesgroppes Régis Desgroppes
            • Votes:
              13 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: