Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47684

test failures are ignored as part of pipeline run

    Details

    • Similar Issues:
    • Epic Link:
    • Sprint:
      Blue Ocean 1.4 - beta 2

      Description

      Eg take a look at: https://ci.blueocean.io/job/blueocean/job/debug%2Ffailures/2/consoleFull - search for 

       

      "Test suite failed to run"

       

      It should be picked up by having a failing junit.xml file that marks the build as unstable, but for some reason it isn't (those tests are ignored in the results, we see this on master too). 

      However, https://github.com/jenkinsci/blueocean-plugin/pull/1505 picks up some failures (but not all maybe)... 

      this only recently started happening. 

       

      failures like: 

       

      [ERROR] 
      [ERROR] Summary of all failing tests
      [ERROR]  FAIL  src/test/js/pipeline-graph-converter-spec.js
      [ERROR]   ● Test suite failed to run
      [ERROR] 
      [ERROR]     Cannot find module '@jenkins-cd/blueocean-core-js' from 'PipelineRunGraph.jsx'
      [ERROR]       
      [ERROR]       at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:169:17)
      [ERROR]       at Object.<anonymous> (src/main/js/components/PipelineRunGraph.jsx:96:315)
      [ERROR] 
      [ERROR]  FAIL  src/test/js/push-events-actions-spec.js
      [ERROR]   ● Test suite failed to run
      [ERROR] 
      [ERROR]     Cannot find module '@jenkins-cd/blueocean-core-js' from 'smart-fetch.js'
      [ERROR]       
      [ERROR]       at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:169:17)
      [ERROR]       at Object.<anonymous> (src/main/js/util/smart-fetch.js:193:225)
      [ERROR] 
      [ERROR]  FAIL  src/test/js/testResult-spec.js
      [ERROR]   ● Test suite failed to run
      [ERROR] 
      [ERROR]     Cannot find module '@jenkins-cd/blueocean-core-js' from 'TestSection.jsx'
      [ERROR]       
      [ERROR]       at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:169:17)
      [ERROR]       at Object.<anonymous> (src/main/js/components/testing/TestSection.jsx:3:50)
      [ERROR] 

       

       Also, you will see in the log, which is immediately after these failures... I wonder if this is related at all? 

       

      [INFO] [07:21:23] tests failed or errored!
      [ERROR] 
      [ERROR] npm ERR! Linux 4.4.0-1035-aws
      [ERROR] npm ERR! argv "/home/jenkins/slave/workspace/blueocean_PR-1505-VP6JSVRIVLGISITB26EPZFJK7AT7NCIZS2YD7XHO6CUEU2DLXEJA/blueocean-dashboard/node/node" "/home/jenkins/slave/workspace/blueocean_PR-1505-VP6JSVRIVLGISITB26EPZFJK7AT7NCIZS2YD7XHO6CUEU2DLXEJA/blueocean-dashboard/node/node_modules/npm/bin/npm-cli.js" "run" "mvntest"
      [ERROR] npm ERR! node v6.4.0
      [ERROR] npm ERR! npm  v3.10.3
      [ERROR] npm ERR! code ELIFECYCLE
      [ERROR] npm ERR! blueocean-dashboard@0.0.1 mvntest: `gulp test`
      [ERROR] npm ERR! Exit status 1
      [ERROR] npm ERR! 
      [ERROR] npm ERR! Failed at the blueocean-dashboard@0.0.1 mvntest script 'gulp test'.
      [ERROR] npm ERR! Make sure you have the latest version of node.js and npm installed.
      [ERROR] npm ERR! If you do, this is most likely a problem with the blueocean-dashboard package,
      [ERROR] npm ERR! not with npm itself.
      [ERROR] npm ERR! Tell the author that this fails on your system:
      [ERROR] npm ERR!     gulp test
      [ERROR] npm ERR! You can get information on how to open an issue for this project with:
      [ERROR] npm ERR!     npm bugs blueocean-dashboard
      [ERROR] npm ERR! Or if that isn't available, you can get their info via:
      [ERROR] npm ERR!     npm owner ls blueocean-dashboard
      [ERROR] npm ERR! There is likely additional logging output above.
      [ERROR] 
      [ERROR] npm ERR! Please include the following file with any support request:
      [ERROR] npm ERR!     /home/jenkins/slave/workspace/blueocean_PR-1505-VP6JSVRIVLGISITB26EPZFJK7AT7NCIZS2YD7XHO6CUEU2DLXEJA/blueocean-dashboard/npm-debug.log
      [ERROR] There are test failures.
      Failed to run task: 'npm run mvntest' failed.
      com.github.eirslett.maven.plugins.frontend.lib.TaskRunnerException: 'npm run mvntest' failed.
      	at com.github.eirslett.maven.plugins.frontend.lib.NodeTaskExecutor.execute(NodeTaskExecutor.java:63)
      	at com.github.eirslett.maven.plugins.frontend.mojo.NpmMojo.execute(NpmMojo.java:62)
      	at com.github.eirslett.maven.plugins.frontend.mojo.AbstractFrontendMojo.execute(AbstractFrontendMojo.java:94)
      	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
      	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
      	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
      	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
      	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
      	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
      	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
      	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
      	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
      	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
      	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
      	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
      	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
      	at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
      	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
      	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
      	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
      Caused by: com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutionException: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
      	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.execute(ProcessExecutor.java:82)
      	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.executeAndRedirectOutput(ProcessExecutor.java:64)
      	at com.github.eirslett.maven.plugins.frontend.lib.NodeExecutor.executeAndRedirectOutput(NodeExecutor.java:29)
      	at com.github.eirslett.maven.plugins.frontend.lib.NodeTaskExecutor.execute(NodeTaskExecutor.java:58)
      	... 24 more
      Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
      	at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
      	at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
      	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.execute(ProcessExecutor.java:74)
      	... 27 more

        Attachments

          Activity

          Hide
          michaelneale Michael Neale added a comment -

          Note, I took a "green" run and grabbed the junit.xml: https://gist.githubusercontent.com/michaelneale/28f09a8f734499ef385eaffd3368a8cb/raw/a54ef4fa71c237f8e5084591c23a95dacf4a520b/gistfile1.txt

          You can see this is all good - so that means that the test runner thought things were ok, there are no failures logged, so this is NOT a pipeline or Jenkins problem, but something to do with jest/runners

           

          the log for that run: https://ci.blueocean.io/job/blueocean/job/debug%2Ffailures/3/consoleFull clearly shows failures if you search for them (like above) but the dashboard clearly showed up as green in the xml. 

          Perhaps keiths fork that shows test failures is just lucky that it is picking them up, and perhaps some other ones are still being lost. Something if not right with running tests and reporting the results. 

          Show
          michaelneale Michael Neale added a comment - Note, I took a "green" run and grabbed the junit.xml: https://gist.githubusercontent.com/michaelneale/28f09a8f734499ef385eaffd3368a8cb/raw/a54ef4fa71c237f8e5084591c23a95dacf4a520b/gistfile1.txt You can see this is all good - so that means that the test runner thought things were ok, there are no failures logged, so this is NOT a pipeline or Jenkins problem, but something to do with jest/runners   the log for that run: https://ci.blueocean.io/job/blueocean/job/debug%2Ffailures/3/consoleFull  clearly shows failures if you search for them (like above) but the dashboard clearly showed up as green in the xml.  Perhaps keiths fork that shows test failures is just lucky that it is picking them up, and perhaps some other ones are still being lost. Something if not right with running tests and reporting the results. 

            People

            • Assignee:
              kzantow Keith Zantow
              Reporter:
              michaelneale Michael Neale
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: