Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-31258

Jenkins Maven plug-in ignores JUnit-format test results from unknown Maven plug-ins

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: maven-plugin
    • Labels:
      None
    • Environment:
      Windows 8.1. JavaSE-1.8. Jenkins 1.635.
    • Similar Issues:

      Description

      The Jenkins Maven plug-in ignores JUnit format files in the test results directory (${project.build.directory}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain* test-capable MOJOs and attempts to support unknown MOJOs by making assumptions which may or may not be justified, depending on the MOJO in question:

      1. that the MOJO will target the 'test' goal or similar
      2. that the MOJO configuration will have a 'reportsDirectory' property

      * maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:

      • org.apache.maven.plugins:maven-surefire-plugin
      • net.kennychua:phantomjs-qunit-runner
      • org.apache.maven.plugins:maven-failsafe-plugin
      • com.sun.maven:maven-junit-plugin
      • org.sonatype.flexmojos:flexmojos-maven-plugin
      • org.sonatype.tycho:maven-osgi-test-plugin
      • org.eclipse.tycho:tycho-surefire-plugin
      • com.jayway.maven.plugins.android.generation2:maven-android-plugin
      • com.jayway.maven.plugins.android.generation2:android-maven-plugin
      • org.codehaus.mojo:gwt-maven-plugin
      • com.smartbear.soapui:soapui-maven-plugin
      • com.smartbear.soapui:soapui-pro-maven-plugin
      • com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
      • com.github.searls:jasmine-maven-plugin
      • org.terracotta.maven.plugins:toolkit-resolver-plugin
      • org.scalatest:scalatest-maven-plugin
      • org.nanoko.playframework:play2-maven-plugin

      Examples of unknown/unsupported Maven test-capable plug-ins:

      • com.github.eirslett:frontend-maven-plugin:karma
      • com.kelveden:maven-karma-plugin:start
      • com.github.greengerong:maven-ng-protractor:run
      • org.codehaus.mojo:exec-maven-plugin:exec (can launch any arbitrary testing process)
      • others, doubtless...

      In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

      Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

      I see three possible solutions:

      1. The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
        • PROs:
          • straightforward to configure
          • no Jenkins-specific POM changes required
          • only one set of changes required
        • CONs:
          • requires explicit configuration in every Jenkins Maven job
      2. Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
        • PROs:
          • should work OOTB in any up-to-date Jenkins installation
          • the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
        • CONs:
          • additional POM complexity purely to support Jenkins integration
          • requires changes to the Jenkins-Maven integration plus a new Maven plug-in
      3. Support a Maven property name format that can supply the equivalent information. Something like:
        • jenkins.<goal>/<group-id>:<artifact-id>[:<version>].reportsDirectory = <test-reports-path>
          • e.g. jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports (inexpressible in XML)
        • jenkins.<plugin-execution-id>.reportsDirectory = <test-reports-path>
          • e.g. jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports
        • PROs:
          • minimal changes required to implement
          • should work OOTB in any up-to-date Jenkins installation
        • CONs:
          • somewhat arcane syntax, not readily discoverable
          • sets a sub-optimal precedent for other such extension metadata

      I have prepared a simple implementation of Option 3 for which I am awaiting legal approval to submit a Pull Request.

        Attachments

          Issue Links

            Activity

            adrianp Adrian Price created issue -
            adrianp Adrian Price made changes -
            Field Original Value New Value
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory (${project.build.directory}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            The Jenkins Job configuration for Maven projects should provide some declarative means to identify additional test reports.
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            The Jenkins Job configuration for Maven projects should provide some declarative means to identify additional test reports.
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            The Jenkins Job configuration for Maven projects should provide some declarative means to identify additional test reports.
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            The Jenkins Job configuration for Maven projects should provide some declarative means to identify additional test reports.
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            The Jenkins Job configuration for Maven projects should provide some declarative means to identify additional test reports.
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            The Jenkins Job configuration for Maven projects should provide some declarative means to identify additional test reports.
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            The Jenkins Job configuration for Maven projects should provide some declarative means to identify additional test reports.
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see two possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see two possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see two possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see two possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. For example {monospaced}jenkins.<goal>/<groupId>:<artifactId>\[:<version>\].reportsDirectory = <pom-relative-test-reports-path>{monospaced}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. For example {monospaced}jenkins.<goal>/<groupId>:<artifactId>\[:<version>\].reportsDirectory = <pom-relative-test-reports-path>{monospaced}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- {{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <pom-relative-test-reports-path>}}
            --- e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}
            -- {{jenkins.#<execution-id>.reportsDirectory = <pom-relative-test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- {{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <pom-relative-test-reports-path>}}
            --- e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}
            -- {{jenkins.#<execution-id>.reportsDirectory = <pom-relative-test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- {{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}
            -- {{jenkins.#<execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- {{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}
            -- {{jenkins.#<execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- -{{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}-
            --- -e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}- (inexpressible in XML)
            -- {{jenkins.<plugin-execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- -{{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}-
            --- -e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}- (inexpressible in XML)
            -- {{jenkins.<plugin-execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- -{{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}-
            --- -e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}- (inexpressible in XML)
            -- {{jenkins.<plugin-execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            --- sets a sub-optimal precedent for other such extension metadata
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven testing plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- -{{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}-
            --- -e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}- (inexpressible in XML)
            -- {{jenkins.<plugin-execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- arcane syntax, not readily discoverable
            --- sets a sub-optimal precedent for other such extension metadata
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven test-capable plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * org.codehaus.mojo:exec-maven-plugin:exec (can launch any arbitrary testing process)
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- -{{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}-
            --- -e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}- (inexpressible in XML)
            -- {{jenkins.<plugin-execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- somewhat arcane syntax, not readily discoverable
            --- sets a sub-optimal precedent for other such extension metadata

            I have prepared a simple implementation of Option 3 for which I am awaiting legal approval to submit a Pull Request.
            adrianp Adrian Price made changes -
            Link This issue is related to JENKINS-13188 [ JENKINS-13188 ]
            adrianp Adrian Price made changes -
            Link This issue is related to JENKINS-11353 [ JENKINS-11353 ]
            adrianp Adrian Price made changes -
            Link This issue is related to JENKINS-6893 [ JENKINS-6893 ]
            adrianp Adrian Price made changes -
            Link This issue is related to JENKINS-31524 [ JENKINS-31524 ]
            adrianp Adrian Price made changes -
            Description The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making the following unjustifiable assumptions:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven test-capable plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * org.codehaus.mojo:exec-maven-plugin:exec (can launch any arbitrary testing process)
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- -{{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}-
            --- -e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}- (inexpressible in XML)
            -- {{jenkins.<plugin-execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- somewhat arcane syntax, not readily discoverable
            --- sets a sub-optimal precedent for other such extension metadata

            I have prepared a simple implementation of Option 3 for which I am awaiting legal approval to submit a Pull Request.
            The Jenkins Maven plug-in ignores JUnit format files in the test results directory ($\{project.build.directory\}/surefire-reports by default). Debugging revealed the cause to be that maven-plugin knows about certain\* test-capable MOJOs and attempts to support unknown MOJOs by making assumptions which may or may not be justified, depending on the MOJO in question:
            # that the MOJO will target the 'test' goal or similar
            # that the MOJO configuration will have a 'reportsDirectory' property

            \* maven testing plug-ins known to the Jenkins Maven plug-in / project type as of 2014-12-04 13:40:29:
            * org.apache.maven.plugins:maven-surefire-plugin
            * net.kennychua:phantomjs-qunit-runner
            * org.apache.maven.plugins:maven-failsafe-plugin
            * com.sun.maven:maven-junit-plugin
            * org.sonatype.flexmojos:flexmojos-maven-plugin
            * org.sonatype.tycho:maven-osgi-test-plugin
            * org.eclipse.tycho:tycho-surefire-plugin
            * com.jayway.maven.plugins.android.generation2:maven-android-plugin
            * com.jayway.maven.plugins.android.generation2:android-maven-plugin
            * org.codehaus.mojo:gwt-maven-plugin
            * com.smartbear.soapui:soapui-maven-plugin
            * com.smartbear.soapui:soapui-pro-maven-plugin
            * com.github.redfish4ktc.soapui:maven-soapui-extension-plugin
            * com.github.searls:jasmine-maven-plugin
            * org.terracotta.maven.plugins:toolkit-resolver-plugin
            * org.scalatest:scalatest-maven-plugin
            * org.nanoko.playframework:play2-maven-plugin

            Examples of unknown/unsupported Maven test-capable plug-ins:
            * com.github.eirslett:frontend-maven-plugin:karma
            * com.kelveden:maven-karma-plugin:start
            * com.github.greengerong:maven-ng-protractor:run
            * org.codehaus.mojo:exec-maven-plugin:exec (can launch any arbitrary testing process)
            * others, doubtless...

            In the general 'unknown Maven plug-in' case neither of these assumptions are likely to be valid e.g., com.github.eirslett:frontend-maven-plugin running Karma tests, the goal is 'karma' (phase = 'test' or 'integration-test', typically) and the test output directory is configured in a separate karma.conf.js file.

            Overall, this means there is no general solution for reporting standard JUnit-format results that have been placed in the Maven-standard location for such results, other than to use only plug-ins that the Jenkins Maven plugin actually knows about! This is inflexible and an unreasonable limitation for a general-purpose CI tool and a highly extensible build tool.

            I see three possible solutions:
            # The Jenkins Job configuration for Maven projects provides some declarative means to identify additional test reports.
            -- PROs:
            --- straightforward to configure
            --- no Jenkins-specific POM changes required
            --- only one set of changes required
            -- CONs:
            --- requires explicit configuration in every Jenkins Maven job
            # Develop a Maven plug-in that could be incorporated into a POM's pluginManagement section purely as the means to indicate that a given plug-in execution (specified as the groupdId, artifactId, version, goal and phase of the otherwise unknown-to-Jenkins Maven test plug-in) produces test results in the specified directory).
            -- PROs:
            --- should work OOTB in any up-to-date Jenkins installation
            --- the Maven plug-in could be extended to cover additional capabilities where Jenkins' built-in Maven integration is inadquate
            -- CONs:
            --- additional POM complexity purely to support Jenkins integration
            --- requires changes to the Jenkins-Maven integration plus a new Maven plug-in
            # Support a Maven property name format that can supply the equivalent information. Something like:
            -- -{{jenkins.<goal>/<group-id>:<artifact-id>\[:<version>\].reportsDirectory = <test-reports-path>}}-
            --- -e.g. {{jenkins.integration-test/org.codehaus.mojo:exec-maven-plugin:1.4.0.reportsDirectory = target/surefire-reports}}- (inexpressible in XML)
            -- {{jenkins.<plugin-execution-id>.reportsDirectory = <test-reports-path>}}
            --- e.g. {{jenkins.run-protractor-tests.reportsDirectory = target/surefire-reports}}
            -- PROs:
            --- minimal changes required to implement
            --- should work OOTB in any up-to-date Jenkins installation
            -- CONs:
            --- somewhat arcane syntax, not readily discoverable
            --- sets a sub-optimal precedent for other such extension metadata

            I have prepared a simple implementation of Option 3 for which I am awaiting legal approval to submit a Pull Request.
            scm_issue_link SCM/JIRA link daemon made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 166508 ] JNJira + In-Review [ 198006 ]

              People

              • Assignee:
                Unassigned
                Reporter:
                adrianp Adrian Price
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: