Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59885

"Agent went offline" or "Channel alread closed" during the build

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: component
    • Environment:
    • Similar Issues:

      Description

      The main job is started, which is starting a some other (sub)jobs. Mostly at the same sub-job, after the first four jobs has been executed without any problem, a job is started. When that job is started or if this job runs one or three minutes we get the issue "Agent went offline". When it runs like say 4 min. than it runs to the end without any issue! That job runs for 1,5 hour or even more!

      After it went wrong with "Agent went offliner" or with "Channel already closed" we start the main job again and than it works always without a issue!!!!????? WHY?????

      Analyzing this:

      • Windows event viewer (see screenshot), I can see that the Jenkinsslave-C_Jenkins is "normally" stopped. But by whom? The Job is not doing this or at least so far as I can see the Job is not doing this and should not do this. See my screenshot. In this screenshot it's a few seconds later the service is stopped. But I have other circumstances and then it was in the exact same second!
      • Jenkins logging "Jenkins-slave.err". There I see issue's with JNLP connection but on this moment I don't know what to do about it! What do I have to change or were can I find documentation. 

      INFO: Trying protocol: JNLP3-connect
      Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
      Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP2-connect
      Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Oct 21, 2019 7:12:21 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Terminated
      Oct 21, 2019 7:12:31 PM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect
      INFO: Restarting agent via jenkins.slaves.restarter.WinswSlaveRestarter@48eae634
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main createEngine
      INFO: Setting up slave: nodename
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener <init>
      INFO: Jenkins agent is running in headless mode.
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Locating server among http://<ip adres>/jenkins/
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Handshaking
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP3-connect
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP2-connect
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main createEngine
      INFO: Setting up slave: nodename
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener <init>
      INFO: Jenkins agent is running in headless mode.
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Locating server among http://<ip adress>jenkins/
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Handshaking
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP3-connect
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP2-connect
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Failed to load native POSIX impl; falling back on Java impl. Unsupported OS.
      Oct 21, 2019 10:51:41 PM hudson.plugins.tmpcleaner.TmpCleanTask call
      INFO: end TmpCleanTask
      Oct 22, 2019 4:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
      INFO: end TmpCleanTask
      Oct 22, 2019 10:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
      INFO: end TmpCleanTask

      • The console log Agent went offline at the same time the JNLP is TERMINATED in Jenkins-slave.err file!:

      19:12:10 2019-10-21
      19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
      19:12:10 2019-10-21 19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
      19:12:10 2019-10-21 19:12:10.3153|INFO|Deploy.Program|File upload successfully done: yyy.json
      19:12:21 Agent went offline during the build19:12:21 ERROR: Connection was broken: java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@2838043[name=<node_name>]
      19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
      19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:629)19:12:21 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      19:12:21 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)19:12:21 at java.util.concurrent.FutureTask.run(FutureTask.java:266)19:12:21 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      19:12:21 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      19:12:21 at java.lang.Thread.run(Thread.java:745)
      19:12:21 Caused by: java.io.IOException: Connection timed out19:12:21 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
      19:12:21 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
      19:12:21 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
      19:12:21 at sun.nio.ch.IOUtil.read(IOUtil.java:197)19:12:21 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
      19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:137)
      19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:310)
      19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
      19:12:21 ... 6 more19:12:21 

      • Screenshot configuring the node

       

      I hope that someone can help me out this issue. When go DuckDuckGo than I find out I'm not the only one who have this issue but I can't find a solution. 

      Regards,

      Nico

        Attachments

          Activity

          Hide
          nkamp01 Nico van de Kamp added a comment -

          Ok, I understand now that, to take these  JVM options into account by the slave I do have to remove the slave first on the Windows server and then install that slave again. Than these options are taken into account by slave in the JNLP file. I was expecting that what I have filled in here, when I stop/start the Jenkins service on the slave, that these options are taken into account by the slave!

          I have not set up the environment, so this is new to me. I have created a jobs, and a lot of scripting but not configured the master and slaves. And I hope that we go asap to the latest Jenkins version but till than, I have this issue. And I don't know what to do than try a lot of things.

          Show
          nkamp01 Nico van de Kamp added a comment - Ok, I understand now that, to take these  JVM options into account by the slave I do have to remove the slave first on the Windows server and then install that slave again. Than these options are taken into account by slave in the JNLP file. I was expecting that what I have filled in here, when I stop/start the Jenkins service on the slave, that these options are taken into account by the slave! I have not set up the environment, so this is new to me. I have created a jobs, and a lot of scripting but not configured the master and slaves. And I hope that we go asap to the latest Jenkins version but till than, I have this issue. And I don't know what to do than try a lot of things.

            People

            • Assignee:
              praqma Praqma Support
              Reporter:
              nkamp01 Nico van de Kamp
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: