Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45910

Template namespace is ignored when deleting pod

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • kubernetes-plugin, remoting
    • None
    • Jenkins 2.60.2
      Kubernetes plugin 0.12
      Kubernetes 1.7.2

      We are dynamically creating and destroying build slaves and they have now started to exit in a Error state.

      Although the build job completes successfully it seems that the slave tries to reconnect.

      Log from slave:

      Aug 01, 2017 1:37:46 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Terminated
Aug 01, 2017 1:37:56 PM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect
INFO: Restarting agent via jenkins.slaves.restarter.UnixSlaveRestarter@863f8a3
Aug 01, 2017 1:37:57 PM hudson.remoting.jnlp.Main createEngine
INFO: Setting up slave: jenkins-build-slave-qvdpn
Aug 01, 2017 1:37:57 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://jenkins-service]
Aug 01, 2017 1:37:58 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, JNLP-connect, Ping, JNLP2-connect]
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
  Agent address: jenkins-service
  Agent port:    50000
  Identity:      51:ac:fd:f7:69:94:53:62:f9:ac:43:6b:72:ff:e7:c2
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-service:50000
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 51:ac:fd:f7:69:94:53:62:f9:ac:43:6b:72:ff:e7:c2
Aug 01, 2017 1:37:58 PM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection to jenkins-service/10.251.248.121:50000] Local headers refused by remote: Unknown client name: jenkins-build-slave-qvdpn
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkins-build-slave-qvdpn
	at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
	at hudson.remoting.Engine.innerRun(Engine.java:385)
	at hudson.remoting.Engine.run(Engine.java:287)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkins-build-slave-qvdpn
	at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:375)
	at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:432)
	at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
	at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
	at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
	at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
	at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
	at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)
	at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at hudson.remoting.Engine$1$1.run(Engine.java:94)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: java.nio.channels.ClosedChannelException
		... 7 more

Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-service:50000
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP3-connect is not enabled, skipping
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP2-connect
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP2-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Server didn't accept the handshake: 
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at hudson.remoting.Engine.innerRun(Engine.java:385)
	at hudson.remoting.Engine.run(Engine.java:287)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Server didn't accept the handshake: 
	at org.jenkinsci.remoting.engine.JnlpProtocol2Handler.sendHandshake(JnlpProtocol2Handler.java:134)
	at org.jenkinsci.remoting.engine.LegacyJnlpProtocolHandler$2.call(LegacyJnlpProtocolHandler.java:162)
	at org.jenkinsci.remoting.engine.LegacyJnlpProtocolHandler$2.call(LegacyJnlpProtocolHandler.java:158)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at hudson.remoting.Engine$1$1.run(Engine.java:94)
	at java.lang.Thread.run(Thread.java:748)

Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-service:50000
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP-connect
Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Server didn't accept the handshake: 
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at hudson.remoting.Engine.innerRun(Engine.java:385)
	at hudson.remoting.Engine.run(Engine.java:287)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Server didn't accept the handshake: 
	at org.jenkinsci.remoting.engine.JnlpProtocol1Handler.sendHandshake(JnlpProtocol1Handler.java:121)
	at org.jenkinsci.remoting.engine.LegacyJnlpProtocolHandler$2.call(LegacyJnlpProtocolHandler.java:162)
	at org.jenkinsci.remoting.engine.LegacyJnlpProtocolHandler$2.call(LegacyJnlpProtocolHandler.java:158)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at hudson.remoting.Engine$1$1.run(Engine.java:94)
	at java.lang.Thread.run(Thread.java:748)

Aug 01, 2017 1:37:58 PM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
	at hudson.remoting.Engine.onConnectionRejected(Engine.java:484)
	at hudson.remoting.Engine.innerRun(Engine.java:448)
	at hudson.remoting.Engine.run(Engine.java:287)

      Log from master:

      Aug 01, 2017 3:37:46 PM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate

Terminating Kubernetes instance for slave jenkins-build-slave-qvdpn

Aug 01, 2017 3:37:46 PM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate

Terminated Kubernetes instance for slave jenkins-build-slave-qvdpn

Aug 01, 2017 3:37:46 PM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate

Disconnected computer jenkins-build-slave-qvdpn

Aug 01, 2017 3:37:46 PM WARNING jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed

Computer.threadPoolForRemoting [#324] for jenkins-build-slave-qvdpn terminated
java.nio.channels.ClosedChannelException
	at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208)
	at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
	at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
	at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
	at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
	at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
	at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
	at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
	at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
	at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
	at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)
	at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
	at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311)
	at hudson.remoting.Channel.close(Channel.java:1295)
	at hudson.remoting.Channel.close(Channel.java:1263)
	at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:708)
	at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96)
	at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:626)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Aug 01, 2017 3:37:46 PM INFO org.jenkinsci.plugins.workflow.job.WorkflowRun finish

api-gateway #269 completed: SUCCESS

Aug 01, 2017 3:37:58 PM INFO hudson.TcpSlaveAgentListener$ConnectionHandler run

Accepted JNLP4-connect connection #145 from 10.248.2.34/10.248.2.34:34644

Aug 01, 2017 3:37:58 PM INFO org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv

[JNLP4-connect connection from 10.248.2.34/10.248.2.34:34644] Refusing headers from remote: Unknown client name: jenkins-build-slave-qvdpn

Aug 01, 2017 3:37:58 PM INFO hudson.TcpSlaveAgentListener$ConnectionHandler run

Accepted JNLP2-connect connection #146 from /10.248.2.34:34646

Aug 01, 2017 3:37:58 PM INFO hudson.TcpSlaveAgentListener$ConnectionHandler run

Accepted JNLP-connect connection #147 from /10.248.2.34:34648

      From kubectl describe

      State:		Terminated
      Reason:		Error
      Exit Code:	255

        1. first-slave-describe
          4 kB
        2. first-slave-log
          7 kB
        3. kubernetes-plugin.log
          17 kB
        4. kubernetes-plugin.log
          7 kB
        5. okhttp3-log-recorder.png
          okhttp3-log-recorder.png
          25 kB
        6. second-slave-describe
          4 kB
        7. second-slave-log
          8 kB

            csanchez Carlos Sanchez
            vikgun Viktor Gunnarson
            Votes:
            10 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: