-
Bug
-
Resolution: Not A Defect
-
Major
-
None
-
Jenkins: 2.105
Kubernetes-plugin: 1.2
I've set up Jenkins to run on a k8s cluster.
I'm using the kubernetes plugin to create agent in the cluster.
I have a discovery service available at jenkins-discovery.jenkins.svc.cluster.local:50000.
I've set up the tunnel property to jenkins-discovery.jenkins.svc.cluster.local:50000
When I start a job, the agent seems to be able to connect but then fails for some unknown reasons.
The agent logs:
INFO: Locating server among [http://jenkins-internal.jenkins.svc.cluster.local:8080/] Feb 12, 2018 4:08:37 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping] Feb 12, 2018 4:08:37 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Agent discovery successful Agent address: jenkins-discovery.jenkins.svc.cluster.local Agent port: 50000 Identity: a1:e6:e0:d2:ad:30:2e:11:47:4f:0f:95:41:d3:d0:a4 Feb 12, 2018 4:08:37 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Feb 12, 2018 4:08:37 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to jenkins-discovery.jenkins.svc.cluster.local:50000 Feb 12, 2018 4:08:37 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Trying protocol: JNLP4-connect Feb 12, 2018 4:08:37 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Remote identity confirmed: a1:e6:e0:d2:ad:30:2e:11:47:4f:0f:95:41:d3:d0:a4 Feb 12, 2018 4:08:39 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connected Feb 12, 2018 4:08:41 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Terminated
It repeats this 4 times and then I see
WARNING: Failed to send back a reply to the request hudson.remoting.Request$2@71b21b06 hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@4d5ba314:JNLP4-connect connection to jenkins-discovery.jenkins.svc.cluster.local/100.71.204.63:50000": channel is already closed at hudson.remoting.Channel.send(Channel.java:672) at hudson.remoting.Request$2.run(Request.java:371) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1$1.run(Engine.java:94) at java.lang.Thread.run(Thread.java:748) Caused by: hudson.remoting.Channel$OrderlyShutdown: Command close created at at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1226) at hudson.remoting.Channel$1.handle(Channel.java:561) at hudson.remoting.AbstractByteBufferCommandTransport.processCommand(AbstractByteBufferCommandTransport.java:202) at hudson.remoting.AbstractByteBufferCommandTransport.receive(AbstractByteBufferCommandTransport.java:188) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onRead(ChannelApplicationLayer.java:186) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecv(ApplicationLayer.java:207) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$2200(BIONetworkLayer.java:48) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:283) ... 4 more Caused by: Command close created at at hudson.remoting.Command.<init>(Command.java:62) at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1220) at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1218) at hudson.remoting.Channel.close(Channel.java:1391) at hudson.remoting.Channel.close(Channel.java:1358) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:745) at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:98) at hudson.slaves.SlaveComputer$2.onClosed(SlaveComputer.java:533) at hudson.remoting.Channel.terminate(Channel.java:1014) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:313) at hudson.remoting.Channel.close(Channel.java:1405) at hudson.remoting.Channel.close(Channel.java:1358) at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:603) at jenkins.slaves.DefaultJnlpSlaveReceiver.afterChannel(DefaultJnlpSlaveReceiver.java:168) at org.jenkinsci.remoting.engine.JnlpConnectionState$4.invoke(JnlpConnectionState.java:421) at org.jenkinsci.remoting.engine.JnlpConnectionState.fire(JnlpConnectionState.java:312) at org.jenkinsci.remoting.engine.JnlpConnectionState.fireAfterChannel(JnlpConnectionState.java:418) at org.jenkinsci.remoting.engine.JnlpProtocol4Handler$Handler$1.run(JnlpProtocol4Handler.java:334) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more
In the master I see
INFO: Created Pod: slave-with-docker-g2k77 in namespace jenkins Feb 12, 2018 4:08:36 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch INFO: Waiting for Pod to be scheduled (0/100): slave-with-docker-g2k77 Feb 12, 2018 4:08:37 PM hudson.TcpSlaveAgentListener$ConnectionHandler run INFO: Accepted JNLP4-connect connection #482 from /100.96.10.76:41026 Feb 12, 2018 4:08:38 PM hudson.TcpSlaveAgentListener$ConnectionHandler run INFO: Accepted JNLP4-connect connection #483 from /100.96.10.76:41036 Feb 12, 2018 4:08:41 PM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed WARNING: Computer.threadPoolForRemoting [#553] for slave-with-docker-g2k77 terminated java.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:313) at hudson.remoting.Channel.close(Channel.java:1405) at hudson.remoting.Channel.close(Channel.java:1358) at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:603) at jenkins.slaves.DefaultJnlpSlaveReceiver.afterChannel(DefaultJnlpSlaveReceiver.java:168) at org.jenkinsci.remoting.engine.JnlpConnectionState$4.invoke(JnlpConnectionState.java:421) at org.jenkinsci.remoting.engine.JnlpConnectionState.fire(JnlpConnectionState.java:312) at org.jenkinsci.remoting.engine.JnlpConnectionState.fireAfterChannel(JnlpConnectionState.java:418) at org.jenkinsci.remoting.engine.JnlpProtocol4Handler$Handler$1.run(JnlpProtocol4Handler.java:334) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
I can see no meaningful errors in the logs. It really confuses me that it says the connection is successful but then it fails
- is blocking
-
-
- Closed
-
- is related to
-
-
- Closed
-
