Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45728

UI not updating with current status. HTTP 500 on nodes/

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Labels:
      None
    • Environment:
      Windows Server 2012R2
      Jenkins 2.60.1
      Java 8u111
      Jenkins installed as Windows Service
      Blue Ocean 1.1.2
    • Sprint:
      Blue Ocean 1.2-beta4, Blue Ocean 1.2
    • Similar Issues:

      Description

      Recently we started having some issues on a couple jobs where the details are no longer being shown within Blue Ocean; we just get greeted by a 'Waiting for run to start' despite seeing it running within the console.

      Pulling up developer tools, I see that we're getting an HTTP 500 against the nodes/ endpoint on the job.

      https://jenkins.contoso.com/blue/rest/organizations/jenkins/pipelines/<job_name>/runs/19/nodes/

      Stack trace

      java.lang.IllegalStateException: Could not load matching start node: java.io.IOException: java.util.concurrent.ExecutionException: java.nio.file.NoSuchFileException: D:\Jenkins-Windows\jobs\Quartermaster_COLO_Pipeline-nonprod\builds\17\workflow\3.xml at org.jenkinsci.plugins.workflow.graph.BlockEndNode.getStartNode(BlockEndNode.java:68) at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.getNodeType(ForkScanner.java:190) at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.next(ForkScanner.java:592) at org.jenkinsci.plugins.workflow.graphanalysis.AbstractFlowScanner.next(AbstractFlowScanner.java:212) at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.next(ForkScanner.java:564) at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.visitSimpleChunks(ForkScanner.java:768) at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.visitSimpleChunks(ForkScanner.java:633) at io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeGraphVisitor.<init>(PipelineNodeGraphVisitor.java:88) at io.jenkins.blueocean.rest.impl.pipeline.NodeGraphBuilder$NodeGraphBuilderFactory.getInstance(NodeGraphBuilder.java:39) at io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeContainerImpl.<init>(PipelineNodeContainerImpl.java:39) at io.jenkins.blueocean.rest.impl.pipeline.PipelineRunImpl.getNodes(PipelineRunImpl.java:150) at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627) at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:343) at org.kohsuke.stapler.MetaClass$3.doDispatch(MetaClass.java:209) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) Caused: javax.servlet.ServletException at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:765) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$3.doDispatch(MetaClass.java:209) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$3.doDispatch(MetaClass.java:209) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:686) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135) at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:225) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:51) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:230) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:202) at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:85) at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:112) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:138) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:92) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

       

        Attachments

          Activity

          Hide
          michaelneale Michael Neale added a comment -

          Going to need more information - a pipeline that reproduces this? 

          Show
          michaelneale Michael Neale added a comment - Going to need more information - a pipeline that reproduces this? 
          Hide
          imeredith Ivan Meredith added a comment -

          Agree about a sample pipeline. 

          Looking at the code this seems like the issue is in the pipeline plugin. Unless we are calling the api wrong.

          As for 500 vs 404, unsure if the error is an expected error state or unexpected, but im leaning towards it being unexpected.

          Show
          imeredith Ivan Meredith added a comment - Agree about a sample pipeline.  Looking at the code this seems like the issue is in the pipeline plugin. Unless we are calling the api wrong. As for 500 vs 404, unsure if the error is an expected error state or unexpected, but im leaning towards it being unexpected.
          Hide
          jamesdumay James Dumay added a comment -

          Assigning to self for investigation

          Show
          jamesdumay James Dumay added a comment - Assigning to self for investigation
          Hide
          jakauppila Jared Kauppila added a comment -

          I'll have to see if I can recreate the problem with a pipeline that I can share.

          Is there any additional debug output I could collect to better dig into this?

          Show
          jakauppila Jared Kauppila added a comment - I'll have to see if I can recreate the problem with a pipeline that I can share. Is there any additional debug output I could collect to better dig into this?
          Hide
          jamesdumay James Dumay added a comment -

          Andrew Bayer Sam Van Oort this looks weird - seen anything like this?

          Show
          jamesdumay James Dumay added a comment - Andrew Bayer Sam Van Oort this looks weird - seen anything like this?
          Hide
          michaelneale Michael Neale added a comment -

          Jared Kauppila a sample pipeline woudl be enough to start with. A "har file" and perhaps support core bundle later on (do not attach a har file to the ticket though - we can get an email to send that to). But really, if there is some generically runnable jenkinsfile, that would show where to look next (ie if there is something obvious, or some plugin in your instance affecting it). 

          Show
          michaelneale Michael Neale added a comment - Jared Kauppila a sample pipeline woudl be enough to start with. A "har file" and perhaps support core bundle later on (do not attach a har file to the ticket though - we can get an email to send that to). But really, if there is some generically runnable jenkinsfile, that would show where to look next (ie if there is something obvious, or some plugin in your instance affecting it). 
          Hide
          jamesdumay James Dumay added a comment -

          Jared Kauppila if you can provide reproduction instructions please reopen this.

          Show
          jamesdumay James Dumay added a comment - Jared Kauppila if you can provide reproduction instructions please reopen this.

            People

            • Assignee:
              jamesdumay James Dumay
              Reporter:
              jakauppila Jared Kauppila
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: