Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57010

Build agent offline with "FATAL: java.nio.channels.ClosedChannelException"

    Details

    • Similar Issues:

      Description

      Agents (swarm-client in use) go offline during job execution

      00:00:00.002 [EnvInject] - Loading node environment variables.
      00:08:49.244 FATAL: java.nio.channels.ClosedChannelException
      00:08:49.249 java.nio.channels.ClosedChannelException
      00:08:49.249 Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ************/************
      00:08:49.249 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1737)
      00:08:49.249 at hudson.remoting.Request.call(Request.java:197)
      00:08:49.249 at hudson.remoting.Channel.call(Channel.java:951)
      00:08:49.249 at hudson.FilePath.act(FilePath.java:1104)
      00:08:49.249 at org.jenkinsci.plugins.envinject.service.EnvironmentVariablesNodeLoader.gatherEnvVarsForNode(EnvironmentVariablesNodeLoader.java:64)
      00:08:49.249 at org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:80)
      00:08:49.249 at org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:44)
      00:08:49.249 at hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:542)
      00:08:49.249 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:462)
      00:08:49.249 at hudson.model.Run.execute(Run.java:1727)
      00:08:49.249 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
      00:08:49.249 at hudson.model.ResourceController.execute(ResourceController.java:97)
      00:08:49.249 at hudson.model.Executor.run(Executor.java:429)
      00:08:49.249 Caused: hudson.remoting.RequestAbortedException
      00:08:49.249 at hudson.remoting.Request.abort(Request.java:335)
      00:08:49.249 at hudson.remoting.Channel.terminate(Channel.java:1034)
      00:08:49.249 at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208)
      00:08:49.249 at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
      00:08:49.249 at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
      00:08:49.249 at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
      00:08:49.250 at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
      00:08:49.250 at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
      00:08:49.250 at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
      00:08:49.250 at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:179)
      00:08:49.250 at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:789)
      00:08:49.250 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      00:08:49.250 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      00:08:49.250 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      00:08:49.250 at java.lang.Thread.run(Thread.java:748)
      00:08:49.254 ERROR: Step ?Delete workspace when build is done? failed: no workspace for docker/***************-build #30
      00:08:49.254 Finished: FAILURE

      Java version: 8

      Jenkins and build agents running in docker

        Attachments

          Activity

          Hide
          jthompson Jeff Thompson added a comment -

          Ewelina Wilkosz, does it commonly fail at that some point, with `EnvironmentVariablesNodeLoader`? Or something similar?

          Generally a `ClosedChannelException` is caused by system, network, or environmental issues. Something outside the agent / remoting connection causes the TCP connection to drop. The messages in Jenkins indicate that has happened but do not have any information about the external cause of the disconnection. A heavily loaded master or other resources can cause network connections to drop. I recommend investigating whether the failures share any common characteristics, including stack trace, system type, load, or anything else.

          Show
          jthompson Jeff Thompson added a comment - Ewelina Wilkosz , does it commonly fail at that some point, with `EnvironmentVariablesNodeLoader`? Or something similar? Generally a `ClosedChannelException` is caused by system, network, or environmental issues. Something outside the agent / remoting connection causes the TCP connection to drop. The messages in Jenkins indicate that has happened but do not have any information about the external cause of the disconnection. A heavily loaded master or other resources can cause network connections to drop. I recommend investigating whether the failures share any common characteristics, including stack trace, system type, load, or anything else.
          Hide
          ewel Ewelina Wilkosz added a comment -

          this seems to be a common characteristic:

          00:00:00.002 [EnvInject] - Loading node environment variables. 

           

          Show
          ewel Ewelina Wilkosz added a comment - this seems to be a common characteristic: 00:00:00.002 [EnvInject] - Loading node environment variables.   
          Hide
          jthompson Jeff Thompson added a comment -

          Ewelina Wilkosz, could you clarify – is this a log from the agent from the master?

          Show
          jthompson Jeff Thompson added a comment - Ewelina Wilkosz , could you clarify – is this a log from the agent from the master?
          Hide
          jthompson Jeff Thompson added a comment -

          Since there has been a lack of response for quite a while on this, I'm going to close this issue. If you have additional information to help move this along, feel free to provide it and re-open as needed.

          Show
          jthompson Jeff Thompson added a comment - Since there has been a lack of response for quite a while on this, I'm going to close this issue. If you have additional information to help move this along, feel free to provide it and re-open as needed.

            People

            • Assignee:
              jthompson Jeff Thompson
              Reporter:
              ewel Ewelina Wilkosz
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: