Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53297

Pod name is detected as `localhost` in Bluemix IKS

    Details

    • Similar Issues:

      Description

      I've Jenkins running on Kubernetes Cluster (Bluemix IKS).

      I am using the kubernetes plugin to create dynamic agents.

      The jenkins connection with agent is made in the standard way:

      • Jenkins URL: jenkins:8080
      • Jenkins Tunnel: jenkins-agent:50000

       

      When i start a job, the agent connect with Jenkins master successfully. But he keeps long time and not execute pipeline steps.

       

      Agent logs: 

      Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorWarning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorAug 29, 2018 12:07:02 AM hudson.remoting.jnlp.Main createEngineINFO: Setting up agent: jenkins-slave-n39mq-9zgzzAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener <init>INFO: Jenkins agent is running in headless mode.Aug 29, 2018 12:07:03 AM hudson.remoting.Engine startEngineINFO: Using Remoting version: 3.23Aug 29, 2018 12:07:03 AM hudson.remoting.Engine startEngineWARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jarsAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Locating server among [http://jenkins:8080/]Aug 29, 2018 12:07:03 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]Aug 29, 2018 12:07:03 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability checkAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Agent discovery successful  Agent address: jenkins-agent  Agent port:    50000  Identity:      bb:d3:5f:09:02:5c:56:70:04:7f:8f:72:f5:bd:23:79Aug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: HandshakingAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Connecting to jenkins-agent:50000Aug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Trying protocol: JNLP4-connectAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Remote identity confirmed: bb:d3:5f:09:02:5c:56:70:04:7f:8f:72:f5:bd:23:79Aug 29, 2018 12:07:04 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: ConnectedAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave$SlaveDisconnector callINFO: Disabled slave engine reconnects.Aug 29, 2018 12:12:09 AM hudson.util.ProcessTree getWARNING: Error while determining if vetoers existhudson.remoting.RequestAbortedException: hudson.remoting.ChannelClosedException: Channel "unknown": Protocol stack cannot write data anymore. ChannelApplicationLayer reports that the NIO Channel is closed at hudson.remoting.Request.abort(Request.java:340) at hudson.remoting.Channel.terminate(Channel.java:1038) at hudson.remoting.Channel.close(Channel.java:1439) at hudson.remoting.Channel.close(Channel.java:1403) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1270) at hudson.remoting.Channel$1.handle(Channel.java:565) at hudson.remoting.AbstractByteBufferCommandTransport.processCommand(AbstractByteBufferCommandTransport.java:203) at hudson.remoting.AbstractByteBufferCommandTransport.receive(AbstractByteBufferCommandTransport.java:189) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onRead(ChannelApplicationLayer.java:187) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecv(ApplicationLayer.java:207) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$2200(BIONetworkLayer.java:48) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:283) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to jenkins-agent/172.21.160.215:50000 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.Request.call(Request.java:202) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286) at com.sun.proxy.$Proxy5.fetch3(Unknown Source) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:209) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.lang.ClassLoader.defineClass(ClassLoader.java:642) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:383) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at hudson.util.ProcessTree.get(ProcessTree.java:399) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 4 moreCaused by: hudson.remoting.ChannelClosedException: Channel "unknown": Protocol stack cannot write data anymore. ChannelApplicationLayer reports that the NIO Channel is closed at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.write(ChannelApplicationLayer.java:302) at hudson.remoting.AbstractByteBufferCommandTransport.write(AbstractByteBufferCommandTransport.java:304) at hudson.remoting.Channel.send(Channel.java:721) at hudson.remoting.Channel.close(Channel.java:1436) ... 18 moreCaused by: java.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doSend(ProtocolStack.java:694) at org.jenkinsci.remoting.protocol.ApplicationLayer.write(ApplicationLayer.java:157) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.write(ChannelApplicationLayer.java:299) ... 21 more
      Aug 29, 2018 12:12:09 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: TerminatedAug 29, 2018 12:12:09 AM hudson.util.ProcessTree getWARNING: Failed to load winp. Reverting to the defaultjava.lang.NoClassDefFoundError: hudson/util/ProcessTree$Linux$LinuxProcess at hudson.util.ProcessTree.get(ProcessTree.java:415) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748)Caused by: java.lang.ClassNotFoundException: hudson.util.ProcessTree$Linux$LinuxProcess at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:171) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 12 more
      

       

      Jenkins Master logs

      Aug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionAug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionINFO: Excess workload after pending Kubernetes agents: 1Aug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionINFO: Template: Kubernetes Pod TemplateAug 29, 2018 12:06:51 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:06:51 AM hudson.slaves.NodeProvisioner$StandardStrategyImpl applyINFO: Started provisioning Kubernetes Pod Template from devops-k8s with 1 executors. Remaining excess workload: 0Aug 29, 2018 12:07:01 AM hudson.slaves.NodeProvisioner$2 runINFO: Kubernetes Pod Template provisioning successfully completed. We have now 2 computer(s)Aug 29, 2018 12:07:01 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:07:01 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launchINFO: Created Pod: jenkins-slave-n39mq-9zgzz in namespace jenkinsAug 29, 2018 12:07:01 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launchINFO: Waiting for Pod to be scheduled (0/100): jenkins-slave-n39mq-9zgzzAug 29, 2018 12:07:03 AM hudson.TcpSlaveAgentListener$ConnectionHandler runINFO: Accepted JNLP4-connect connection #3 from /172.30.89.194:44930Aug 29, 2018 12:07:08 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:12:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminateINFO: Terminating Kubernetes instance for agent jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:12:09 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosedWARNING: Computer.threadPoolForRemoting [#88] for jenkins-slave-n39mq-9zgzz terminatedjava.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:314) at hudson.remoting.Channel.close(Channel.java:1450) at hudson.remoting.Channel.close(Channel.java:1403) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:799) at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:103) at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:715) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Terminated Kubernetes instance for agent jenkins/jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePodINFO: Terminated Kubernetes instance for agent jenkins/jenkins-slave-n39mq-9zgzzDisconnected computer jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminateINFO: Disconnected computer jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution$PodTemplateCallback finishedINFO: Removing pod template jenkins-slave-n39mq from cloud devops-k8sAug 29, 2018 12:12:10 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finishINFO: teste-deploy #45 completed: FAILURE
      

       

       

      Console Output 

      Started by user Vinicius Xavier Running in Durability level: MAX_SURVIVABILITY [Pipeline] podTemplate [Pipeline] { [Pipeline] node Still waiting to schedule task There are no nodes with the label ‘jenkins-pipeline’ Agent jenkins-slave-n39mq-9zgzz is provisioned from template Kubernetes Pod Template Agent specification [Kubernetes Pod Template] (jenkins-pipeline): * [helm] lachlanevenson/k8s-helm Running on jenkins-slave-n39mq-9zgzz in /home/jenkins/workspace/teste-deploy [Pipeline] { [Pipeline] container [Pipeline] { [Pipeline] sh [teste-deploy] Running shell script [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:417) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:449) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20) Caused: java.io.IOException: Failed to execute shell script inside container [helm] of pod [localhost]. Timed out waiting for container to become ready! at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:438) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:449) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20) at WorkflowScript.run(WorkflowScript:10) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor375.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE
      

       

      Jenkinsfile

      podTemplate(label: 'jenkins-pipeline', cloud: 'devops-k8s', 
        containers: [
          containerTemplate(name: 'helm', image: 'lachlanevenson/k8s-helm', command: 'cat', ttyEnabled: true)], 
        volumes:[ 
          hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')]
      ){ 
      node ('jenkins-pipeline') { 
        container('helm') { 
          sh 'helm init --client-only' sh 'helm ls --namespace default' 
        } 
       } 
      }
      

       

      Obs.: This happened after update Kubernetes 1.10 to 1.11. IKS changed RuntimeContainer from Docker to Containerd
       

        Attachments

          Issue Links

            Activity

            vxavier Vinicius Xavier created issue -
            Hide
            vxavier Vinicius Xavier added a comment -

            I actived the ALL log level and caught this:

            <-- 404 Not Found https://kubernetes.default/api/v1/namespaces/jenkins/pods/localhost (3ms)Aug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logContent-Type: application/jsonAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logDate: Wed, 29 Aug 2018 18:32:10 GMTAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logContent-Length: 186Aug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform log{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"localhost\" not found","reason":"NotFound","details":{"name":"localhost","kind":"pods"},"code":404}
            

            Why jenkins using 'localhost' with name of the pod???

            Show
            vxavier Vinicius Xavier added a comment - I actived the ALL log level and caught this: <-- 404 Not Found https: //kubernetes. default /api/v1/namespaces/jenkins/pods/localhost (3ms)Aug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logContent-Type: application/jsonAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logDate: Wed, 29 Aug 2018 18:32:10 GMTAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logContent-Length: 186Aug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform log{ "kind" : "Status" , "apiVersion" : "v1" , "metadata" :{}, "status" : "Failure" , "message" : "pods \" localhost\ " not found" , "reason" : "NotFound" , "details" :{ "name" : "localhost" , "kind" : "pods" }, "code" :404} Why jenkins using 'localhost' with name of the pod???
            Hide
            csanchez Carlos Sanchez added a comment -

            what are the log lines before that one

            Show
            csanchez Carlos Sanchez added a comment - what are the log lines before that one
            Hide
            vxavier Vinicius Xavier added a comment -

            I noticed that when using for example: 'echo hi', the pipeline runs successfully, jenkins GET pods/<slave name> endpoint.
            But, when using any command 'sh "cat /etc/hosts" ', jenkins try GET pods/localhost endpoint

            --> GET https://kubernetes.default/api/v1/namespaces/jenkins/pods/localhost http/1.1
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJqZW5raW5zIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImplbmtpbnMtdG9rZW4tZnFsZmMiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiamVua2lucyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjM3NzU1YTY1LTk1YmEtMTFlOC04MzlkLTJhZGJhODJkZmMxMSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpqZW5raW5zOmplbmtpbnMifQ.mfHpadNaF6JhwS7YL171TBg1J_RxOQJwfcyAYUHDjPnc3VI_E6_LwXZ9CRJIIXxtcN7Ny8EWwL09EonaFq20LfZNK5TQaYbu6FIyipMNGaZ1JxA-vPZ-ZYghxOJcbK36JK_ap71g9Jiun7qUVbqGo39yuF6Ga-ELcDil30BOX3FMrZiAH3D6yTy0FUKMLt81KnCzwUEdyKSOO86Z6nNzzZTQ8qw_lVJOJbUPjicbMcOLz1yp19UW8sFYr2smY-BSv6oeHkoXCfbccKu7UsRFSxFKnFOK-5bv8TBwuZm8Wkm93RVUg9dNhrGnYUI5RtzJ1i2ntjqyBSBdYDeinYz5VPPTRhK7zG_k8bmIsw1OAiUdKEQZIsON1EaNYxFEwJ2yd3byQ5N-BUvkfEbOXwM6pWbKDM_RjEfZcFcSXr2XOC_1q6vbwaWBLKtEyD_xvz_ZLq7dbIU5N-FYm8NwAtSYcyxssnHleOqpUfUcDvVepkXJySMs__aarzMjdQYqLhpoCDQtiAZUq6LyFRtW3AezOClj1vweUoUKVx5KCyHm3meEKFc38TUUn9BkYY3lMlz4Q579TLzKzlTmQSFy9sdY1eSvQlikzLx-VSCb5aSc9D6xrINbky0tDmFH1uM7eT7oNRE9ZphctWADz3kZrP_cU-EWW4Ja798jpln93YgrFpY
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            Host: kubernetes.default
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            Connection: Keep-Alive
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            Accept-Encoding: gzip
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            User-Agent: okhttp/3.9.1
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            --> END GET
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            <-- 404 Not Found https://kubernetes.default/api/v1/namespaces/jenkins/pods/localhost (3ms)
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            Content-Type: application/json
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            Date: Wed, 29 Aug 2018 19:28:34 GMT
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            Content-Length: 186
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"localhost\" not found","reason":"NotFound","details":{"name":"localhost","kind":"pods"},"code":404}
            
            Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
            <-- END HTTP (186-byte body)
            
            Show
            vxavier Vinicius Xavier added a comment - I noticed that when using for example: 'echo hi', the pipeline runs successfully, jenkins GET pods/<slave name> endpoint. But, when using any command 'sh "cat /etc/hosts" ', jenkins try GET pods/localhost endpoint --> GET https: //kubernetes. default /api/v1/namespaces/jenkins/pods/localhost http/1.1 Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJqZW5raW5zIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImplbmtpbnMtdG9rZW4tZnFsZmMiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiamVua2lucyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjM3NzU1YTY1LTk1YmEtMTFlOC04MzlkLTJhZGJhODJkZmMxMSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpqZW5raW5zOmplbmtpbnMifQ.mfHpadNaF6JhwS7YL171TBg1J_RxOQJwfcyAYUHDjPnc3VI_E6_LwXZ9CRJIIXxtcN7Ny8EWwL09EonaFq20LfZNK5TQaYbu6FIyipMNGaZ1JxA-vPZ-ZYghxOJcbK36JK_ap71g9Jiun7qUVbqGo39yuF6Ga-ELcDil30BOX3FMrZiAH3D6yTy0FUKMLt81KnCzwUEdyKSOO86Z6nNzzZTQ8qw_lVJOJbUPjicbMcOLz1yp19UW8sFYr2smY-BSv6oeHkoXCfbccKu7UsRFSxFKnFOK-5bv8TBwuZm8Wkm93RVUg9dNhrGnYUI5RtzJ1i2ntjqyBSBdYDeinYz5VPPTRhK7zG_k8bmIsw1OAiUdKEQZIsON1EaNYxFEwJ2yd3byQ5N-BUvkfEbOXwM6pWbKDM_RjEfZcFcSXr2XOC_1q6vbwaWBLKtEyD_xvz_ZLq7dbIU5N-FYm8NwAtSYcyxssnHleOqpUfUcDvVepkXJySMs__aarzMjdQYqLhpoCDQtiAZUq6LyFRtW3AezOClj1vweUoUKVx5KCyHm3meEKFc38TUUn9BkYY3lMlz4Q579TLzKzlTmQSFy9sdY1eSvQlikzLx-VSCb5aSc9D6xrINbky0tDmFH1uM7eT7oNRE9ZphctWADz3kZrP_cU-EWW4Ja798jpln93YgrFpY Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Host: kubernetes. default Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Connection: Keep-Alive Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Accept-Encoding: gzip Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log User-Agent: okhttp/3.9.1 Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log --> END GET Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log <-- 404 Not Found https: //kubernetes. default /api/v1/namespaces/jenkins/pods/localhost (3ms) Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Content-Type: application/json Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Date: Wed, 29 Aug 2018 19:28:34 GMT Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Content-Length: 186 Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log { "kind" : "Status" , "apiVersion" : "v1" , "metadata" :{}, "status" : "Failure" , "message" : "pods \" localhost\ " not found" , "reason" : "NotFound" , "details" :{ "name" : "localhost" , "kind" : "pods" }, "code" :404} Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log <-- END HTTP (186- byte body)
            vxavier Vinicius Xavier made changes -
            Field Original Value New Value
            Attachment jenkins-master.log [ 43949 ]
            Hide
            vxavier Vinicius Xavier added a comment -

            I attached a log, from the beginning of the job to the first 404 in the endpoint pods / localhost.

            Show
            vxavier Vinicius Xavier added a comment - I attached a log, from the beginning of the job to the first 404 in the endpoint pods / localhost.
            Hide
            leonbadcam leon gregori added a comment - - edited

            Hello,

            I have also been using the Jenkins Kubernetes Plugin 1.12.4 on IBM Cloud.
            It also stopped working over the Weekend and im currently in investigation.

            Job output is this, but i will provide more soon. As soon as I can identify the issue and see if it is another root cause, I will create a seperate issue.

            java.io.IOException: container [java] does not exist in pod [jenkins-slave-sm0j8-666ng]
             at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:435)
             at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
             at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
             at hudson.Launcher$ProcStarter.start(Launcher.java:454)
             at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186)
             at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86)
             at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182)
             at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229)
             at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153)
             at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
             at sun.reflect.GeneratedMethodAccessor906.invoke(Unknown Source)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
             at java.lang.reflect.Method.invoke(Method.java:498)
             at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
             at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
             at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
             at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
             at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
             at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
             at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
             at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
             at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
             at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:133)
             at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
             at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
             at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
             at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
             at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
             at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
             at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
             at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
             at WorkflowScript.run(WorkflowScript:44)
             at __cps.transform__(Native Method)
             at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
             at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
             at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
             at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
             at java.lang.reflect.Method.invoke(Method.java:498)
             at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
             at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:103)
             at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
             at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
             at java.lang.reflect.Method.invoke(Method.java:498)
             at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
             at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:60)
             at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
             at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
             at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
             at java.lang.reflect.Method.invoke(Method.java:498)
             at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
             at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
             at com.cloudbees.groovy.cps.Next.step(Next.java:83)
             at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
             at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
             at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
             at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
             at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
             at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
             at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
             at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
             at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
             at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
             at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
             at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
             at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
             at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
             at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
             at java.util.concurrent.FutureTask.run(FutureTask.java:266)
             at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
             at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
             at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
             at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
             at java.util.concurrent.FutureTask.run(FutureTask.java:266)
             at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
             at java.lang.Thread.run(Thread.java:748)
             Finished: FAILURE
            Show
            leonbadcam leon gregori added a comment - - edited Hello, I have also been using the Jenkins Kubernetes Plugin 1.12.4 on IBM Cloud. It also stopped working over the Weekend and im currently in investigation. Job output is this, but i will provide more soon. As soon as I can identify the issue and see if it is another root cause, I will create a seperate issue. java.io.IOException: container [java] does not exist in pod [jenkins-slave-sm0j8-666ng] at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:435) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor906.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:133) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:44) at __cps.transform__(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:103) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:60) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE
            Hide
            alokrajiv Alok Rajiv added a comment - - edited

            Hi ,

            So, I think what everyone is facing here is an issue stemming from the runtime dependency change. Thanks to @rtheis from IBM Cloud Network Dev Team on Slack who reminded me of the change.

            IKS on K8S v1.11 now uses containerd runtime instead of docker runtime. I fear this is why our container state cannot be ready by the master. I experimented on 2 new fresh clusters with 1.10 and 1.11 on IBM Cloud and do confirm that 1.10 works just fine and replicated the above issue on 1.11 (The docker mount itself is failing on 1.11)

             IBM migration doc link: https://console.bluemix.net/docs/containers/cs_versions.html#containerd

             

            Do correct me if anyone has another side to this story.

            Show
            alokrajiv Alok Rajiv added a comment - - edited Hi , So, I think what everyone is facing here is an issue stemming from the runtime dependency change. Thanks to @rtheis from IBM Cloud Network Dev Team on Slack who reminded me of the change. IKS on K8S v1.11 now uses containerd   runtime instead of docker runtime. I fear this is why our container state cannot be ready by the master. I experimented on 2 new fresh clusters with 1.10 and 1.11 on IBM Cloud and do confirm that 1.10 works just fine and replicated the above issue on 1.11 (The docker mount itself is failing on 1.11)  IBM migration doc link:  https://console.bluemix.net/docs/containers/cs_versions.html#containerd   Do correct me if anyone has another side to this story.
            Hide
            leonbadcam leon gregori added a comment -

            Alok Rajiv yes that is the root cause if using IKS 1.11.

            In Jenkins Pipeline it should be just fine to mount containerd socket instead of docker socket.
            I could not yet test this as our cluster is not updated yet to containerd.

            I will keep you updated here.

            Currently (Working on old IBM IKS)

                volumes: [  hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: '/etc/docker/certs.d'),
                            hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')]
            

            Possible fix (Working on new containerd IBM IKS)

                volumes: [  hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: 'Certificates'),
                            hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: 'SOCKET FOR CONTAINERD')]

             

            Show
            leonbadcam leon gregori added a comment - Alok Rajiv yes that is the root cause if using IKS 1.11. In Jenkins Pipeline it should be just fine to mount containerd socket instead of docker socket. I could not yet test this as our cluster is not updated yet to containerd. I will keep you updated here. Currently (Working on old IBM IKS) volumes: [ hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: '/etc/docker/certs.d' ), hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: '/ var /run/docker.sock' )] Possible fix (Working on new containerd IBM IKS) volumes: [ hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: 'Certificates' ), hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: 'SOCKET FOR CONTAINERD' )]  
            leonbadcam leon gregori made changes -
            Environment Jenkins 2.121.3
            Kubernetes IKS 1.11 (Bluemix)
            Kubernetes Plugin 1.12.4
            Jenkins 2.121.3, 2.141
            Kubernetes IKS 1.11 (Bluemix)
            Kubernetes Plugin 1.12.4
            Hide
            leonbadcam leon gregori added a comment -

            Still no solution until now.
            1.11 Kubernetes on IBM Cloud uses containerd runtime and we dont know how to use the correct socket. I think same is applicable for kubernetes plugin in jenkins.

            See also:
            https://stackoverflow.com/questions/52179148/jenkins-kubernetes-plugin-with-containerd?noredirect=1#comment92152693_52179148

             

            Show
            leonbadcam leon gregori added a comment - Still no solution until now. 1.11 Kubernetes on IBM Cloud uses containerd runtime and we dont know how to use the correct socket. I think same is applicable for kubernetes plugin in jenkins. See also: https://stackoverflow.com/questions/52179148/jenkins-kubernetes-plugin-with-containerd?noredirect=1#comment92152693_52179148  
            Hide
            dcurrie David Currie added a comment -

            Is that Jenkinsfile just an example or what you're actually trying to achieve? You shouldn't need a Docker socket in order to use helm. That is not to say that this isn't a problem in other scenarios e.g. when trying to use the daemon on the Kubernetes node to perform Docker builds. In that case, you could look at using kaniko or fall back to Docker-in-Docker.

            Show
            dcurrie David Currie added a comment - Is that Jenkinsfile just an example or what you're actually trying to achieve? You shouldn't need a Docker socket in order to use helm . That is not to say that this isn't a problem in other scenarios e.g. when trying to use the daemon on the Kubernetes node to perform Docker builds. In that case, you could look at using kaniko or fall back to Docker-in-Docker.
            Hide
            alokrajiv Alok Rajiv added a comment -

            Hi David, 

            About the docker builds I was thinking on the same grounds. But, before we reach that step the issue is the builder containers spawned up for the job itself is the issue.

            Currently, jenkins master doesn't see the slave builder container ready status. My error is  `Timed out waiting for container to become ready!`

            Is the Kubernetes Pod Template, using docker daemon underneath? 

            Show
            alokrajiv Alok Rajiv added a comment - Hi David,  About the docker builds I was thinking on the same grounds. But, before we reach that step the issue is the builder containers spawned up for the job itself is the issue. Currently, jenkins master doesn't see the slave builder container ready status. My error is  `Timed out waiting for container to become ready!` Is the Kubernetes Pod Template, using docker daemon underneath? 
            dcurrie David Currie made changes -
            Description I've Jenkins running on Kubernetes Cluster (Bluemix IKS).

            I am using the kubernetes plugin to create dynamic agents.

            The jenkins connection with agent is made in the standard way:

            - Jenkins URL: jenkins:8080

            - Jenkins Tunnel: jenkins-agent:50000

             

            When i start a job, the agent connect with Jenkins master successfully. But he keeps long time and not execute pipeline steps.

             

            Agent logs: 
            {code:java}
            Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorWarning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorAug 29, 2018 12:07:02 AM hudson.remoting.jnlp.Main createEngineINFO: Setting up agent: jenkins-slave-n39mq-9zgzzAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener <init>INFO: Jenkins agent is running in headless mode.Aug 29, 2018 12:07:03 AM hudson.remoting.Engine startEngineINFO: Using Remoting version: 3.23Aug 29, 2018 12:07:03 AM hudson.remoting.Engine startEngineWARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jarsAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Locating server among [http://jenkins:8080/]Aug 29, 2018 12:07:03 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]Aug 29, 2018 12:07:03 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability checkAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Agent discovery successful  Agent address: jenkins-agent  Agent port:    50000  Identity:      bb:d3:5f:09:02:5c:56:70:04:7f:8f:72:f5:bd:23:79Aug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: HandshakingAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Connecting to jenkins-agent:50000Aug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Trying protocol: JNLP4-connectAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Remote identity confirmed: bb:d3:5f:09:02:5c:56:70:04:7f:8f:72:f5:bd:23:79Aug 29, 2018 12:07:04 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: ConnectedAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave$SlaveDisconnector callINFO: Disabled slave engine reconnects.Aug 29, 2018 12:12:09 AM hudson.util.ProcessTree getWARNING: Error while determining if vetoers existhudson.remoting.RequestAbortedException: hudson.remoting.ChannelClosedException: Channel "unknown": Protocol stack cannot write data anymore. ChannelApplicationLayer reports that the NIO Channel is closed at hudson.remoting.Request.abort(Request.java:340) at hudson.remoting.Channel.terminate(Channel.java:1038) at hudson.remoting.Channel.close(Channel.java:1439) at hudson.remoting.Channel.close(Channel.java:1403) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1270) at hudson.remoting.Channel$1.handle(Channel.java:565) at hudson.remoting.AbstractByteBufferCommandTransport.processCommand(AbstractByteBufferCommandTransport.java:203) at hudson.remoting.AbstractByteBufferCommandTransport.receive(AbstractByteBufferCommandTransport.java:189) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onRead(ChannelApplicationLayer.java:187) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecv(ApplicationLayer.java:207) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$2200(BIONetworkLayer.java:48) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:283) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to jenkins-agent/172.21.160.215:50000 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.Request.call(Request.java:202) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286) at com.sun.proxy.$Proxy5.fetch3(Unknown Source) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:209) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.lang.ClassLoader.defineClass(ClassLoader.java:642) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:383) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at hudson.util.ProcessTree.get(ProcessTree.java:399) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 4 moreCaused by: hudson.remoting.ChannelClosedException: Channel "unknown": Protocol stack cannot write data anymore. ChannelApplicationLayer reports that the NIO Channel is closed at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.write(ChannelApplicationLayer.java:302) at hudson.remoting.AbstractByteBufferCommandTransport.write(AbstractByteBufferCommandTransport.java:304) at hudson.remoting.Channel.send(Channel.java:721) at hudson.remoting.Channel.close(Channel.java:1436) ... 18 moreCaused by: java.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doSend(ProtocolStack.java:694) at org.jenkinsci.remoting.protocol.ApplicationLayer.write(ApplicationLayer.java:157) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.write(ChannelApplicationLayer.java:299) ... 21 more
            Aug 29, 2018 12:12:09 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: TerminatedAug 29, 2018 12:12:09 AM hudson.util.ProcessTree getWARNING: Failed to load winp. Reverting to the defaultjava.lang.NoClassDefFoundError: hudson/util/ProcessTree$Linux$LinuxProcess at hudson.util.ProcessTree.get(ProcessTree.java:415) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748)Caused by: java.lang.ClassNotFoundException: hudson.util.ProcessTree$Linux$LinuxProcess at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:171) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 12 more
            {code}
             

            Jenkins Master logs
            {code:java}
            Aug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionAug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionINFO: Excess workload after pending Kubernetes agents: 1Aug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionINFO: Template: Kubernetes Pod TemplateAug 29, 2018 12:06:51 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:06:51 AM hudson.slaves.NodeProvisioner$StandardStrategyImpl applyINFO: Started provisioning Kubernetes Pod Template from devops-k8s with 1 executors. Remaining excess workload: 0Aug 29, 2018 12:07:01 AM hudson.slaves.NodeProvisioner$2 runINFO: Kubernetes Pod Template provisioning successfully completed. We have now 2 computer(s)Aug 29, 2018 12:07:01 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:07:01 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launchINFO: Created Pod: jenkins-slave-n39mq-9zgzz in namespace jenkinsAug 29, 2018 12:07:01 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launchINFO: Waiting for Pod to be scheduled (0/100): jenkins-slave-n39mq-9zgzzAug 29, 2018 12:07:03 AM hudson.TcpSlaveAgentListener$ConnectionHandler runINFO: Accepted JNLP4-connect connection #3 from /172.30.89.194:44930Aug 29, 2018 12:07:08 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:12:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminateINFO: Terminating Kubernetes instance for agent jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:12:09 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosedWARNING: Computer.threadPoolForRemoting [#88] for jenkins-slave-n39mq-9zgzz terminatedjava.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:314) at hudson.remoting.Channel.close(Channel.java:1450) at hudson.remoting.Channel.close(Channel.java:1403) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:799) at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:103) at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:715) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Terminated Kubernetes instance for agent jenkins/jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePodINFO: Terminated Kubernetes instance for agent jenkins/jenkins-slave-n39mq-9zgzzDisconnected computer jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminateINFO: Disconnected computer jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution$PodTemplateCallback finishedINFO: Removing pod template jenkins-slave-n39mq from cloud devops-k8sAug 29, 2018 12:12:10 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finishINFO: teste-deploy #45 completed: FAILURE
            {code}
             

             

            Console Output 
            {code:java}
            Started by user Vinicius Xavier Running in Durability level: MAX_SURVIVABILITY [Pipeline] podTemplate [Pipeline] { [Pipeline] node Still waiting to schedule task There are no nodes with the label ‘jenkins-pipeline’ Agent jenkins-slave-n39mq-9zgzz is provisioned from template Kubernetes Pod Template Agent specification [Kubernetes Pod Template] (jenkins-pipeline): * [helm] lachlanevenson/k8s-helm Running on jenkins-slave-n39mq-9zgzz in /home/jenkins/workspace/teste-deploy [Pipeline] { [Pipeline] container [Pipeline] { [Pipeline] sh [teste-deploy] Running shell script [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:417) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:449) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20) Caused: java.io.IOException: Failed to execute shell script inside container [helm] of pod [localhost]. Timed out waiting for container to become ready! at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:438) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:449) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20) at WorkflowScript.run(WorkflowScript:10) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor375.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE
            {code}
             

            Jenkinsfile
            {code:java}
            podTemplate(label: 'jenkins-pipeline', cloud: 'devops-k8s',
              containers: [
                containerTemplate(name: 'helm', image: 'lachlanevenson/k8s-helm', command: 'cat', ttyEnabled: true)],
              volumes:[
                hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')]
            ){
            node ('jenkins-pipeline') {
              container('helm') {
                sh 'helm init --client-only' sh 'helm ls --namespace default'
              }
             }
            }
            {code}
             

            Obs.: This happened after update Kubernetes 1.10 to 1.11. IKS changed RuntimeContainer from Docker to Containerd
              
            I've Jenkins running on Kubernetes Cluster (Bluemix IKS).

            I am using the kubernetes plugin to create dynamic agents.

            The jenkins connection with agent is made in the standard way:
             - Jenkins URL: jenkins:8080

             - Jenkins Tunnel: jenkins-agent:50000

             

            When i start a job, the agent connect with Jenkins master successfully. But he keeps long time and not execute pipeline steps.

             

            Agent logs: 
            {code:java}
            Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorWarning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorAug 29, 2018 12:07:02 AM hudson.remoting.jnlp.Main createEngineINFO: Setting up agent: jenkins-slave-n39mq-9zgzzAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener <init>INFO: Jenkins agent is running in headless mode.Aug 29, 2018 12:07:03 AM hudson.remoting.Engine startEngineINFO: Using Remoting version: 3.23Aug 29, 2018 12:07:03 AM hudson.remoting.Engine startEngineWARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jarsAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Locating server among [http://jenkins:8080/]Aug 29, 2018 12:07:03 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]Aug 29, 2018 12:07:03 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability checkAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Agent discovery successful  Agent address: jenkins-agent  Agent port:    50000  Identity:      bb:d3:5f:09:02:5c:56:70:04:7f:8f:72:f5:bd:23:79Aug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: HandshakingAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Connecting to jenkins-agent:50000Aug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Trying protocol: JNLP4-connectAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Remote identity confirmed: bb:d3:5f:09:02:5c:56:70:04:7f:8f:72:f5:bd:23:79Aug 29, 2018 12:07:04 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: ConnectedAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave$SlaveDisconnector callINFO: Disabled slave engine reconnects.Aug 29, 2018 12:12:09 AM hudson.util.ProcessTree getWARNING: Error while determining if vetoers existhudson.remoting.RequestAbortedException: hudson.remoting.ChannelClosedException: Channel "unknown": Protocol stack cannot write data anymore. ChannelApplicationLayer reports that the NIO Channel is closed at hudson.remoting.Request.abort(Request.java:340) at hudson.remoting.Channel.terminate(Channel.java:1038) at hudson.remoting.Channel.close(Channel.java:1439) at hudson.remoting.Channel.close(Channel.java:1403) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1270) at hudson.remoting.Channel$1.handle(Channel.java:565) at hudson.remoting.AbstractByteBufferCommandTransport.processCommand(AbstractByteBufferCommandTransport.java:203) at hudson.remoting.AbstractByteBufferCommandTransport.receive(AbstractByteBufferCommandTransport.java:189) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onRead(ChannelApplicationLayer.java:187) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecv(ApplicationLayer.java:207) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$2200(BIONetworkLayer.java:48) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:283) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to jenkins-agent/172.21.160.215:50000 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.Request.call(Request.java:202) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286) at com.sun.proxy.$Proxy5.fetch3(Unknown Source) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:209) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.lang.ClassLoader.defineClass(ClassLoader.java:642) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:383) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at hudson.util.ProcessTree.get(ProcessTree.java:399) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 4 moreCaused by: hudson.remoting.ChannelClosedException: Channel "unknown": Protocol stack cannot write data anymore. ChannelApplicationLayer reports that the NIO Channel is closed at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.write(ChannelApplicationLayer.java:302) at hudson.remoting.AbstractByteBufferCommandTransport.write(AbstractByteBufferCommandTransport.java:304) at hudson.remoting.Channel.send(Channel.java:721) at hudson.remoting.Channel.close(Channel.java:1436) ... 18 moreCaused by: java.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doSend(ProtocolStack.java:694) at org.jenkinsci.remoting.protocol.ApplicationLayer.write(ApplicationLayer.java:157) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.write(ChannelApplicationLayer.java:299) ... 21 more
            Aug 29, 2018 12:12:09 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: TerminatedAug 29, 2018 12:12:09 AM hudson.util.ProcessTree getWARNING: Failed to load winp. Reverting to the defaultjava.lang.NoClassDefFoundError: hudson/util/ProcessTree$Linux$LinuxProcess at hudson.util.ProcessTree.get(ProcessTree.java:415) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748)Caused by: java.lang.ClassNotFoundException: hudson.util.ProcessTree$Linux$LinuxProcess at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:171) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 12 more
            {code}
             

            Jenkins Master logs
            {code:java}
            Aug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionAug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionINFO: Excess workload after pending Kubernetes agents: 1Aug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionINFO: Template: Kubernetes Pod TemplateAug 29, 2018 12:06:51 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:06:51 AM hudson.slaves.NodeProvisioner$StandardStrategyImpl applyINFO: Started provisioning Kubernetes Pod Template from devops-k8s with 1 executors. Remaining excess workload: 0Aug 29, 2018 12:07:01 AM hudson.slaves.NodeProvisioner$2 runINFO: Kubernetes Pod Template provisioning successfully completed. We have now 2 computer(s)Aug 29, 2018 12:07:01 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:07:01 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launchINFO: Created Pod: jenkins-slave-n39mq-9zgzz in namespace jenkinsAug 29, 2018 12:07:01 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launchINFO: Waiting for Pod to be scheduled (0/100): jenkins-slave-n39mq-9zgzzAug 29, 2018 12:07:03 AM hudson.TcpSlaveAgentListener$ConnectionHandler runINFO: Accepted JNLP4-connect connection #3 from /172.30.89.194:44930Aug 29, 2018 12:07:08 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:12:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminateINFO: Terminating Kubernetes instance for agent jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:12:09 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosedWARNING: Computer.threadPoolForRemoting [#88] for jenkins-slave-n39mq-9zgzz terminatedjava.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:314) at hudson.remoting.Channel.close(Channel.java:1450) at hudson.remoting.Channel.close(Channel.java:1403) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:799) at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:103) at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:715) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Terminated Kubernetes instance for agent jenkins/jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePodINFO: Terminated Kubernetes instance for agent jenkins/jenkins-slave-n39mq-9zgzzDisconnected computer jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminateINFO: Disconnected computer jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution$PodTemplateCallback finishedINFO: Removing pod template jenkins-slave-n39mq from cloud devops-k8sAug 29, 2018 12:12:10 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finishINFO: teste-deploy #45 completed: FAILURE
            {code}
             

             

            Console Output 
            {code:java}
            Started by user Vinicius Xavier Running in Durability level: MAX_SURVIVABILITY [Pipeline] podTemplate [Pipeline] { [Pipeline] node Still waiting to schedule task There are no nodes with the label ‘jenkins-pipeline’ Agent jenkins-slave-n39mq-9zgzz is provisioned from template Kubernetes Pod Template Agent specification [Kubernetes Pod Template] (jenkins-pipeline): * [helm] lachlanevenson/k8s-helm Running on jenkins-slave-n39mq-9zgzz in /home/jenkins/workspace/teste-deploy [Pipeline] { [Pipeline] container [Pipeline] { [Pipeline] sh [teste-deploy] Running shell script [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:417) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:449) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20) Caused: java.io.IOException: Failed to execute shell script inside container [helm] of pod [localhost]. Timed out waiting for container to become ready! at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:438) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:449) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20) at WorkflowScript.run(WorkflowScript:10) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor375.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE
            {code}
             

            Jenkinsfile
            {code:java}
            podTemplate(label: 'jenkins-pipeline', cloud: 'devops-k8s',
              containers: [
                containerTemplate(name: 'helm', image: 'lachlanevenson/k8s-helm', command: 'cat', ttyEnabled: true)],
              volumes:[
                hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')]
            ){
            node ('jenkins-pipeline') {
              container('helm') {
                sh 'helm init --client-only' sh 'helm ls --namespace default'
              }
             }
            }
            {code}
             

            Obs.: This happened after update Kubernetes 1.10 to 1.11. IKS changed RuntimeContainer from Docker to Containerd
             
            Hide
            dcurrie David Currie added a comment - - edited

            Is that even if you remove the hostPathVolume from the podTemplate? My assumption was that the slave pod was failing to start because it couldn't satisfy the requested mount.

            Show
            dcurrie David Currie added a comment - - edited Is that even if you remove the hostPathVolume from the podTemplate ? My assumption was that the slave pod was failing to start because it couldn't satisfy the requested mount.
            Hide
            alokrajiv Alok Rajiv added a comment -

            I'll need to get back on this. Will need to spin up a new cluster in 1.11 to test again tomorrow.
            Could just have been the bad mount. Thanks very much for the leads  !

            Show
            alokrajiv Alok Rajiv added a comment - I'll need to get back on this. Will need to spin up a new cluster in 1.11 to test again tomorrow. Could just have been the bad mount. Thanks very much for the leads  !
            Hide
            leonbadcam leon gregori added a comment -

            docker-sock-volume:
            Type: HostPath (bare host directory volume)
            Path: /var/run/containerd/containerd.sock
            HostPathType: File

            Seems to work. So within a docker container from pipeline this can be mounted to execute eg. "docker build ."
            But it gives new error

            Error: failed to create containerd task: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"docker pull jenkins\": executable file not found in $PATH": unknown
            Warning BackOff 0s (x2 over 2s) kubelet, 10.144.181.244 Back-off restarting failed container

            Show
            leonbadcam leon gregori added a comment - docker-sock-volume: Type: HostPath (bare host directory volume) Path: /var/run/containerd/containerd.sock HostPathType: File Seems to work. So within a docker container from pipeline this can be mounted to execute eg. "docker build ." But it gives new error Error: failed to create containerd task: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"docker pull jenkins\": executable file not found in $PATH": unknown Warning BackOff 0s (x2 over 2s) kubelet, 10.144.181.244 Back-off restarting failed container
            Hide
            dcurrie David Currie added a comment -

            You can't expect to be able to just swap out the Docker socket for a containerd socket and have that work. Even setting that aside, containerd doesn't even have the concept of building images. I've been out of the loop a bit with IKS but I suspect the path of least resistance here is to have Jenkins use the IBM Container Registry to build your Docker images i.e. via ibmcloud cr build.

            Show
            dcurrie David Currie added a comment - You can't expect to be able to just swap out the Docker socket for a containerd socket and have that work. Even setting that aside, containerd doesn't even have the concept of building images. I've been out of the loop a bit with IKS but I suspect the path of least resistance here is to have Jenkins use the IBM Container Registry to build your Docker images i.e. via  ibmcloud cr build .
            Hide
            leonbadcam leon gregori added a comment - - edited

            David Currie IBM container registry "cr build" command uses local docker deamon. I just tried it and it build locally.

            So i think there are only 2 workarounds:
            1. Having docker build Slave on some VM outside of K8
            2. Using some Java based building service like kaniko

             

            See also

             https://stackoverflow.com/questions/52179148/jenkins-kubernetes-plugin-with-containerd/52871199#52871199

            Show
            leonbadcam leon gregori added a comment - - edited David Currie IBM container registry "cr build" command uses local docker deamon. I just tried it and it build locally. So i think there are only 2 workarounds: 1. Having docker build Slave on some VM outside of K8 2. Using some Java based building service like kaniko   See also   https://stackoverflow.com/questions/52179148/jenkins-kubernetes-plugin-with-containerd/52871199#52871199
            Hide
            leonbadcam leon gregori added a comment - - edited

            So i can only say that the issue in this ticket was now reproduced by me:
            (IKS 1.11, Plugin 1.12.4 or 1.12.9, happens on both)

            [testingSimple] Running shell script
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] }
            [Pipeline] // container
            [Pipeline] }
            [Pipeline] // node
            [Pipeline] }
            [Pipeline] // podTemplate
            [Pipeline] End of Pipeline
            java.lang.InterruptedException: sleep interrupted
            	at java.lang.Thread.sleep(Native Method)
            	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959)
            	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219)
            	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:417)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
            	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
            	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
            	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
            	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
            	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
            	at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
            	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
            	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
            	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
            	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
            Caused: java.io.IOException: Failed to execute shell script inside container [java] of pod [localhost]. Timed out waiting for container to become ready!
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:438)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
            	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
            	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
            	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
            	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
            	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
            	at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
            	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
            	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
            	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
            	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
            	at WorkflowScript.run(WorkflowScript:43)
            	at ___cps.transform___(Native Method)
            	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
            	at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
            	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
            	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
            	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
            	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
            	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
            	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
            	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
            	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
            	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
            	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            	at java.lang.Thread.run(Thread.java:748)
            Finished: FAILURE
            

            So in the end the plugin is not capable of running sh inside the slaves

            Show
            leonbadcam leon gregori added a comment - - edited So i can only say that the issue in this ticket was now reproduced by me: (IKS 1.11, Plugin 1.12.4 or 1.12.9, happens on both) [testingSimple] Running shell script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline java.lang.InterruptedException: sleep interrupted at java.lang. Thread .sleep(Native Method) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:417) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) Caused: java.io.IOException: Failed to execute shell script inside container [java] of pod [localhost]. Timed out waiting for container to become ready! at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:438) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:43) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE So in the end the plugin is not capable of running sh inside the slaves
            Hide
            csanchez Carlos Sanchez added a comment -

            Can you try building the JENKINS-53297 branch in PR-385 and see what is the error now?

            Show
            csanchez Carlos Sanchez added a comment - Can you try building the  JENKINS-53297 branch in PR-385 and see what is the error now?
            csanchez Carlos Sanchez made changes -
            Remote Link This issue links to "PR-385 (Web Link)" [ 21934 ]
            Hide
            leonbadcam leon gregori added a comment - - edited

            @Carlos Sanchez will try tomorrow. Appreciate your commits 

            Show
            leonbadcam leon gregori added a comment - - edited @Carlos Sanchez will try tomorrow. Appreciate your commits 
            Hide
            leonbadcam leon gregori added a comment - - edited

            Carlos Sanchez I tested the JENKINS-53297 plugin

            Manage Plugins:
            Kubernetes plugin
            This plugin integrates Jenkins with Kubernetes
            1.12.10-SNAPSHOT (private-d0dc3394-blwdev)

            I tested your Commit with the following Jenkinsfile:

            // Simple Jenkinsfile for testing Kubernetes Plugin
            // Developers input is required here.
            def appname = "test"
            def jarname = "test.jar"// Jenkins Deployment Variables and Credentials
            def cloud = env.CLOUD ?: "kubernetes"
            def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id"
            def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id"
            def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id"
            def serviceAccount = env.SERVICE_ACCOUNT ?: "default"
            def sonarCredsID = "sonarID"
            appname = appname.toLowerCase()
            
            // Pod Environment Variables
            def kubenamespace = "default"
            def regnamespace = "blw-msa"
            def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net"podTemplate(label: 'mypod', cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [
                    envVar(key: 'NAMESPACE', value: kubenamespace),
                    envVar(key: 'REGNAMESPACE', value: regnamespace),
            	envVar(key: 'JARNAME', value: jarname),
                    envVar(key: 'APPNAME', value: appname),
                    envVar(key: 'REGISTRY', value: registry)
                ],
                volumes: [
                //hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: '/etc/docker/certs.d'),
                //hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')
                //hostPathVolume(hostPath: '/tmp/gradle', mountPath: '/root/.gradle/caches/')
                ],
                containers: [
                    containerTemplate(name: 'java', image: 'openjdk:8-jdk', ttyEnabled: true, command: 'cat')
              ]) {    node('mypod') {
                    stage('Checkout') {
            		echo "test"
            		}
                    container('java') {
            		stage('Static Code Analysis') {
                                		sh "java -version"}
                            }
            }
            }
            

            This unfortunately has not helped, the issue is still the same (Providing Console Output of above Jenkinsfile) :

            Started by user admin
            Running in Durability level: MAX_SURVIVABILITY
            [Pipeline] podTemplate
            [Pipeline] {
            [Pipeline] node
            Still waiting to schedule task
            Waiting for next available executor
            Agent jenkins-slave-brxph-v8fff is provisioned from template Kubernetes Pod Template
            Agent specification [Kubernetes Pod Template] (mypod): 
            * [java] openjdk:8-jdkRunning on jenkins-slave-brxph-v8fff in /home/jenkins/workspace/testingSimple
            [Pipeline] {
            [Pipeline] stage
            [Pipeline] { (Checkout)
            [Pipeline] echo
            test
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] container
            [Pipeline] {
            [Pipeline] stage
            [Pipeline] { (Static Code Analysis)[Pipeline] sh
            [testingSimple] Running shell script
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] }
            [Pipeline] // container
            [Pipeline] }
            [Pipeline] // node
            [Pipeline] }
            [Pipeline] // podTemplate
            [Pipeline] End of Pipeline
            java.lang.InterruptedException: sleep interrupted
            	at java.lang.Thread.sleep(Native Method)
            	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959)
            	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219)
            	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:420)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
            	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
            	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
            	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
            	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
            	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
            	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
            	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
            	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
            	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
            	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
            Caused: java.io.IOException: Failed to execute shell script inside container [java] of pod [localhost]. Timed out (299 seconds) waiting for container to become ready!
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:447)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
            	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
            	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
            	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
            	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
            	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
            	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
            	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
            	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
            	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
            	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
            	at WorkflowScript.run(WorkflowScript:44)
            	at ___cps.transform___(Native Method)
            	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
            	at sun.reflect.GeneratedMethodAccessor283.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
            	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
            	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
            	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
            	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
            	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
            	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
            	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
            	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
            	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
            	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            	at java.lang.Thread.run(Thread.java:748)
            Finished: FAILURE
            
            

            If you like, and can also provide you with credentials of my public jenkins testing instance, its running on a free ibm cluster for the next 20 days.

             

            Show
            leonbadcam leon gregori added a comment - - edited Carlos Sanchez I tested the JENKINS-53297 plugin Manage Plugins: Kubernetes plugin This plugin integrates Jenkins with Kubernetes 1.12.10-SNAPSHOT (private-d0dc3394-blwdev) I tested your Commit with the following Jenkinsfile: // Simple Jenkinsfile for testing Kubernetes Plugin // Developers input is required here. def appname = "test" def jarname = "test.jar" // Jenkins Deployment Variables and Credentials def cloud = env.CLOUD ?: "kubernetes" def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id" def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id" def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id" def serviceAccount = env.SERVICE_ACCOUNT ?: " default " def sonarCredsID = "sonarID" appname = appname.toLowerCase() // Pod Environment Variables def kubenamespace = " default " def regnamespace = "blw-msa" def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net" podTemplate(label: 'mypod' , cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [ envVar(key: 'NAMESPACE' , value: kubenamespace), envVar(key: 'REGNAMESPACE' , value: regnamespace), envVar(key: 'JARNAME' , value: jarname), envVar(key: 'APPNAME' , value: appname), envVar(key: 'REGISTRY' , value: registry) ], volumes: [ //hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: '/etc/docker/certs.d' ), //hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: '/ var /run/docker.sock' ) //hostPathVolume(hostPath: '/tmp/gradle' , mountPath: '/root/.gradle/caches/' ) ], containers: [ containerTemplate(name: 'java' , image: 'openjdk:8-jdk' , ttyEnabled: true , command: 'cat' ) ]) { node( 'mypod' ) { stage( 'Checkout' ) { echo "test" } container( 'java' ) { stage( 'Static Code Analysis' ) { sh "java -version" } } } } This unfortunately has not helped, the issue is still the same (Providing Console Output of above Jenkinsfile) : Started by user admin Running in Durability level: MAX_SURVIVABILITY [Pipeline] podTemplate [Pipeline] { [Pipeline] node Still waiting to schedule task Waiting for next available executor Agent jenkins-slave-brxph-v8fff is provisioned from template Kubernetes Pod Template Agent specification [Kubernetes Pod Template] (mypod): * [java] openjdk:8-jdkRunning on jenkins-slave-brxph-v8fff in /home/jenkins/workspace/testingSimple [Pipeline] { [Pipeline] stage [Pipeline] { (Checkout) [Pipeline] echo test [Pipeline] } [Pipeline] // stage [Pipeline] container [Pipeline] { [Pipeline] stage [Pipeline] { (Static Code Analysis)[Pipeline] sh [testingSimple] Running shell script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline java.lang.InterruptedException: sleep interrupted at java.lang. Thread .sleep(Native Method) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:420) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) Caused: java.io.IOException: Failed to execute shell script inside container [java] of pod [localhost]. Timed out (299 seconds) waiting for container to become ready! at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:447) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:44) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor283.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE If you like, and can also provide you with credentials of my public jenkins testing instance, its running on a free ibm cluster for the next 20 days.  
            Hide
            csanchez Carlos Sanchez added a comment -

            The plugin gets the pod name from the /etc/hostname file and looks like IKS just sets hostname to localhost instead of the pod name 

            https://github.com/jenkinsci/kubernetes-plugin/blob/2e10cd29fa7bd1bca86e3716f04af61af818639e/src/main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/KubernetesNodeContext.java#L42

             

            Only solution I can think of is the use of the downward API 

            https://stackoverflow.com/questions/37253068/programmatically-get-the-name-of-the-pod-that-a-container-belongs-to-in-kubernet/37261036

             

            Please try with the latest commit in the branch and adding this to your pod definition

             

                  env:
                    - name: POD_NAME
                      valueFrom:
                        fieldRef:
                          fieldPath: metadata.name 
            Show
            csanchez Carlos Sanchez added a comment - The plugin gets the pod name from the /etc/hostname file and looks like IKS just sets hostname to localhost instead of the pod name  https://github.com/jenkinsci/kubernetes-plugin/blob/2e10cd29fa7bd1bca86e3716f04af61af818639e/src/main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/KubernetesNodeContext.java#L42   Only solution I can think of is the use of the downward API  https://stackoverflow.com/questions/37253068/programmatically-get-the-name-of-the-pod-that-a-container-belongs-to-in-kubernet/37261036   Please try with the latest commit in the branch and adding this to your pod definition   env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name
            Hide
            leonbadcam leon gregori added a comment - - edited

            Im currently fighting with the pod template definition. The variable is available in the pod but empty / not defined, need some more try and error until i make it i guess.

             

            // Simple Jenkinsfile for testing Kubernetes Plugin
            // Developers input is required here.
            def appname = "test"
            def jarname = "test.jar"// Jenkins Deployment Variables and Credentials
            def cloud = env.CLOUD ?: "kubernetes"
            def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id"
            def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id"
            def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id"
            def serviceAccount = env.SERVICE_ACCOUNT ?: "default"
            def sonarCredsID = "sonarID"
            appname = appname.toLowerCase()// Pod Environment Variables
            def kubenamespace = "default"
            def regnamespace = "test"
            def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net"podTemplate(label: 'mypod', cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [
                    envVar(key: 'NAMESPACE', value: kubenamespace),
                    envVar(key: 'REGNAMESPACE', value: regnamespace),
            	envVar(key: 'JARNAME', value: jarname),
                    envVar(key: 'APPNAME', value: appname),
                    envVar(key: 'POD_NAME', valuefrom: { fieldref: { fieldPath: metadata.name }}),
                    envVar(key: 'REGISTRY', value: registry)
                ],
                volumes: [
                //hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: '/etc/docker/certs.d'),
                //hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')
            	//hostPathVolume(hostPath: '/tmp/gradle', mountPath: '/root/.gradle/caches/')
                ],
                containers: [
                    // In Theory, java container is not needed as the normal jenkins jnlp slave would be capable of executing gradle.
            	// But as we mount the cache from host, we need a root user running inside the build container, otherwise gradle wont be able to write to .gradle
                    containerTemplate(name: 'java', image: 'openjdk:8-jdk', ttyEnabled: true, command: 'cat')
              ]) {    node('mypod') {
                    stage('Checkout') {
            		echo "test"
            		}
                    container('java') {
            		stage('Static Code Analysis') {
                                		sh "java -version"}
                            }
            }
            }
            
            $ kubectl get pod jenkins-slave-hcwsg-c9ntw -o yaml
            apiVersion: v1
            kind: Pod
            metadata:
             annotations:
             kubernetes.io/psp: ibm-privileged-psp
             creationTimestamp: 2018-10-19T11:11:45Z
             deletionGracePeriodSeconds: 30
             deletionTimestamp: 2018-10-19T11:12:28Z
             labels:
             jenkins: slave
             jenkins/mypod: "true"
             name: jenkins-slave-hcwsg-c9ntw
             namespace: default
             resourceVersion: "20068"
             selfLink: /api/v1/namespaces/default/pods/jenkins-slave-hcwsg-c9ntw
             uid: c3ad40dc-d38f-11e8-a534-0623e2fdc71b
            spec:
             containers:
             - command:
             - cat
             env:
             - name: NAMESPACE
             value: default
             - name: JENKINS_SECRET
             value: -
             - name: JENKINS_TUNNEL
             value: my-release-jenkins-agent:50000
             - name: JENKINS_AGENT_NAME
             value: jenkins-slave-hcwsg-c9ntw
             - name: REGNAMESPACE
             value: blw-msa
             - name: JARNAME
             value: AddressManagement.jar
             - name: POD_NAME
             - name: APPNAME
             value: addressmanagement
             - name: JENKINS_NAME
             value: jenkins-slave-hcwsg-c9ntw
             - name: JENKINS_URL
             value: http://my-release-jenkins:8080/
             - name: HOME
             value: /home/jenkins
             - name: REGISTRY
             value: registry.eu-de.bluemix.net
             image: openjdk:8-jdk
             imagePullPolicy: IfNotPresent
             name: java
             resources: {}
             securityContext:
             privileged: false
             terminationMessagePath: /dev/termination-log
             terminationMessagePolicy: File
             tty: true
             volumeMounts:
             - mountPath: /home/jenkins
             name: workspace-volume
             - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
             name: default-token-9hntr
             readOnly: true
             workingDir: /home/jenkins
             - env:
             - name: NAMESPACE
             value: default
             - name: JENKINS_SECRET
             value: -
             - name: JENKINS_TUNNEL
             value: my-release-jenkins-agent:50000
             - name: JENKINS_AGENT_NAME
             value: jenkins-slave-hcwsg-c9ntw
             - name: REGNAMESPACE
             value: blw-msa
             - name: JARNAME
             value: AddressManagement.jar
             - name: POD_NAME
             - name: APPNAME
             value: addressmanagement
             - name: JENKINS_NAME
             value: jenkins-slave-hcwsg-c9ntw
             - name: JENKINS_URL
             value: http://my-release-jenkins:8080/
             - name: HOME
             value: /home/jenkins
             - name: REGISTRY
             value: registry.eu-de.bluemix.net
             image: jenkins/jnlp-slave:alpine
             imagePullPolicy: IfNotPresent
             name: jnlp
             resources: {}
             terminationMessagePath: /dev/termination-log
             terminationMessagePolicy: File
             volumeMounts:
             - mountPath: /home/jenkins
             name: workspace-volume
             - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
             name: default-token-9hntr
             readOnly: true
             dnsPolicy: ClusterFirst
             imagePullSecrets:
             - name: bluemix-default-secret
             - name: bluemix-default-secret-regional
             - name: bluemix-default-secret-international
             nodeName: 10.144.180.238
             priority: 0
             restartPolicy: Never
             schedulerName: default-scheduler
             securityContext: {}
             serviceAccount: default
             serviceAccountName: default
             terminationGracePeriodSeconds: 30
             tolerations:
             - effect: NoExecute
             key: node.kubernetes.io/not-ready
             operator: Exists
             tolerationSeconds: 300
             - effect: NoExecute
             key: node.kubernetes.io/unreachable
             operator: Exists
             tolerationSeconds: 300
             volumes:
             - emptyDir: {}
             name: workspace-volume
             - name: default-token-9hntr
             secret:
             defaultMode: 420
             secretName: default-token-9hntr
            status:
             conditions:
             - lastProbeTime: null
             lastTransitionTime: 2018-10-19T11:11:45Z
             status: "True"
             type: Initialized
             - lastProbeTime: null
             lastTransitionTime: 2018-10-19T11:11:47Z
             status: "True"
             type: Ready
             - lastProbeTime: null
             lastTransitionTime: null
             status: "True"
             type: ContainersReady
             - lastProbeTime: null
             lastTransitionTime: 2018-10-19T11:11:45Z
             status: "True"
             type: PodScheduled
             containerStatuses:
             - containerID: containerd://e8ce755b312e1d14201905eae9d09acd5b21e214f4df08a4959d6576a1f140ff
             image: docker.io/library/openjdk:8-jdk
             imageID: docker.io/library/openjdk@sha256:dfc720be416bdc7b5df39fb8b237a3bca88dcdf39ad00568f8c3cd3558ff1bc2
             lastState: {}
             name: java
             ready: true
             restartCount: 0
             state:
             running:
             startedAt: 2018-10-19T11:11:46Z
             - containerID: containerd://a5694b45c0a9bb88e54e24d66a37440a1fbe695cb8236187c1f37c92b9f3b416
             image: docker.io/jenkins/jnlp-slave:alpine
             imageID: docker.io/jenkins/jnlp-slave@sha256:7a826ce43cb7eeb59b0dc5000f89e00060ba31e51524d87ea811ba23453185ed
             lastState: {}
             name: jnlp
             ready: true
             restartCount: 0
             state:
             running:
             startedAt: 2018-10-19T11:11:46Z
             hostIP: 10.144.180.238
             phase: Running
             podIP: 172.30.202.87
             qosClass: BestEffort
             startTime: 2018-10-19T11:11:45Z
            
            

             

            Job Console Output

             

            java.lang.IllegalArgumentException: Name must be provided.
            	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withName(BaseOperation.java:234)
            	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withName(BaseOperation.java:49)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:419)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
            	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
            	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
            	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
            	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
            	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
            	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
            	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
            	at sun.reflect.GeneratedMethodAccessor262.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
            	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
            	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
            	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
            	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
            	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
            	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
            	at WorkflowScript.run(WorkflowScript:45)
            	at ___cps.transform___(Native Method)
            	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
            	at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
            	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
            	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
            	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
            	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
            	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
            	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
            	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
            	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
            	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
            	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            	at java.lang.Thread.run(Thread.java:748)
            Finished: FAILURE
            
            Show
            leonbadcam leon gregori added a comment - - edited Im currently fighting with the pod template definition. The variable is available in the pod but empty / not defined, need some more try and error until i make it i guess.   // Simple Jenkinsfile for testing Kubernetes Plugin // Developers input is required here. def appname = "test" def jarname = "test.jar" // Jenkins Deployment Variables and Credentials def cloud = env.CLOUD ?: "kubernetes" def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id" def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id" def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id" def serviceAccount = env.SERVICE_ACCOUNT ?: " default " def sonarCredsID = "sonarID" appname = appname.toLowerCase() // Pod Environment Variables def kubenamespace = " default " def regnamespace = "test" def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net" podTemplate(label: 'mypod' , cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [ envVar(key: 'NAMESPACE' , value: kubenamespace), envVar(key: 'REGNAMESPACE' , value: regnamespace), envVar(key: 'JARNAME' , value: jarname), envVar(key: 'APPNAME' , value: appname), envVar(key: 'POD_NAME' , valuefrom: { fieldref: { fieldPath: metadata.name }}), envVar(key: 'REGISTRY' , value: registry) ], volumes: [ //hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: '/etc/docker/certs.d' ), //hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: '/ var /run/docker.sock' ) //hostPathVolume(hostPath: '/tmp/gradle' , mountPath: '/root/.gradle/caches/' ) ], containers: [ // In Theory, java container is not needed as the normal jenkins jnlp slave would be capable of executing gradle. // But as we mount the cache from host, we need a root user running inside the build container, otherwise gradle wont be able to write to .gradle containerTemplate(name: 'java' , image: 'openjdk:8-jdk' , ttyEnabled: true , command: 'cat' ) ]) { node( 'mypod' ) { stage( 'Checkout' ) { echo "test" } container( 'java' ) { stage( 'Static Code Analysis' ) { sh "java -version" } } } } $ kubectl get pod jenkins-slave-hcwsg-c9ntw -o yaml apiVersion: v1 kind: Pod metadata: annotations: kubernetes.io/psp: ibm-privileged-psp creationTimestamp: 2018-10-19T11:11:45Z deletionGracePeriodSeconds: 30 deletionTimestamp: 2018-10-19T11:12:28Z labels: jenkins: slave jenkins/mypod: " true " name: jenkins-slave-hcwsg-c9ntw namespace: default resourceVersion: "20068" selfLink: /api/v1/namespaces/ default /pods/jenkins-slave-hcwsg-c9ntw uid: c3ad40dc-d38f-11e8-a534-0623e2fdc71b spec: containers: - command: - cat env: - name: NAMESPACE value: default - name: JENKINS_SECRET value: - - name: JENKINS_TUNNEL value: my-release-jenkins-agent:50000 - name: JENKINS_AGENT_NAME value: jenkins-slave-hcwsg-c9ntw - name: REGNAMESPACE value: blw-msa - name: JARNAME value: AddressManagement.jar - name: POD_NAME - name: APPNAME value: addressmanagement - name: JENKINS_NAME value: jenkins-slave-hcwsg-c9ntw - name: JENKINS_URL value: http: //my-release-jenkins:8080/ - name: HOME value: /home/jenkins - name: REGISTRY value: registry.eu-de.bluemix.net image: openjdk:8-jdk imagePullPolicy: IfNotPresent name: java resources: {} securityContext: privileged: false terminationMessagePath: /dev/termination-log terminationMessagePolicy: File tty: true volumeMounts: - mountPath: /home/jenkins name: workspace-volume - mountPath: / var /run/secrets/kubernetes.io/serviceaccount name: default -token-9hntr readOnly: true workingDir: /home/jenkins - env: - name: NAMESPACE value: default - name: JENKINS_SECRET value: - - name: JENKINS_TUNNEL value: my-release-jenkins-agent:50000 - name: JENKINS_AGENT_NAME value: jenkins-slave-hcwsg-c9ntw - name: REGNAMESPACE value: blw-msa - name: JARNAME value: AddressManagement.jar - name: POD_NAME - name: APPNAME value: addressmanagement - name: JENKINS_NAME value: jenkins-slave-hcwsg-c9ntw - name: JENKINS_URL value: http: //my-release-jenkins:8080/ - name: HOME value: /home/jenkins - name: REGISTRY value: registry.eu-de.bluemix.net image: jenkins/jnlp-slave:alpine imagePullPolicy: IfNotPresent name: jnlp resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /home/jenkins name: workspace-volume - mountPath: / var /run/secrets/kubernetes.io/serviceaccount name: default -token-9hntr readOnly: true dnsPolicy: ClusterFirst imagePullSecrets: - name: bluemix- default -secret - name: bluemix- default -secret-regional - name: bluemix- default -secret-international nodeName: 10.144.180.238 priority: 0 restartPolicy: Never schedulerName: default -scheduler securityContext: {} serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations: - effect: NoExecute key: node.kubernetes.io/not-ready operator : Exists tolerationSeconds: 300 - effect: NoExecute key: node.kubernetes.io/unreachable operator : Exists tolerationSeconds: 300 volumes: - emptyDir: {} name: workspace-volume - name: default -token-9hntr secret: defaultMode: 420 secretName: default -token-9hntr status: conditions: - lastProbeTime: null lastTransitionTime: 2018-10-19T11:11:45Z status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: 2018-10-19T11:11:47Z status: "True" type: Ready - lastProbeTime: null lastTransitionTime: null status: "True" type: ContainersReady - lastProbeTime: null lastTransitionTime: 2018-10-19T11:11:45Z status: "True" type: PodScheduled containerStatuses: - containerID: containerd: //e8ce755b312e1d14201905eae9d09acd5b21e214f4df08a4959d6576a1f140ff image: docker.io/library/openjdk:8-jdk imageID: docker.io/library/openjdk@sha256:dfc720be416bdc7b5df39fb8b237a3bca88dcdf39ad00568f8c3cd3558ff1bc2 lastState: {} name: java ready: true restartCount: 0 state: running: startedAt: 2018-10-19T11:11:46Z - containerID: containerd: //a5694b45c0a9bb88e54e24d66a37440a1fbe695cb8236187c1f37c92b9f3b416 image: docker.io/jenkins/jnlp-slave:alpine imageID: docker.io/jenkins/jnlp-slave@sha256:7a826ce43cb7eeb59b0dc5000f89e00060ba31e51524d87ea811ba23453185ed lastState: {} name: jnlp ready: true restartCount: 0 state: running: startedAt: 2018-10-19T11:11:46Z hostIP: 10.144.180.238 phase: Running podIP: 172.30.202.87 qosClass: BestEffort startTime: 2018-10-19T11:11:45Z   Job Console Output   java.lang.IllegalArgumentException: Name must be provided. at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withName(BaseOperation.java:234) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withName(BaseOperation.java:49) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:419) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor262.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:45) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE
            Hide
            csanchez Carlos Sanchez added a comment -

            you would need to use the yaml syntax, I don't think valuefrom will work

            Show
            csanchez Carlos Sanchez added a comment - you would need to use the yaml syntax, I don't think valuefrom will work
            Hide
            leonbadcam leon gregori added a comment - - edited

            Carlos Sanchez I used the yaml syntax now for jnlp container, but the yaml is not used properly:

            Show
            leonbadcam leon gregori added a comment - - edited Carlos Sanchez I used the yaml syntax now for jnlp container, but the yaml is not used properly:
            Hide
            csanchez Carlos Sanchez added a comment -

            don't name the container jenkins-slave, name it jnlp

            Show
            csanchez Carlos Sanchez added a comment - don't name the container jenkins-slave , name it jnlp
            leonbadcam leon gregori made changes -
            Attachment image-2018-10-22-09-40-49-930.png [ 44823 ]
            leonbadcam leon gregori made changes -
            Attachment image-2018-10-22-09-47-09-082.png [ 44824 ]
            Hide
            leonbadcam leon gregori added a comment -

            Okay, i named it jnlp, only 1 container is created now. But im still missing the variable pod_name

            $ kubectl describe pod test-2jrwj
            Name:               test-2jrwj
            Namespace:          default
            Priority:           0
            PriorityClassName:  <none>
            Node:               10.144.180.238/10.144.180.238
            Start Time:         Mon, 22 Oct 2018 09:44:34 +0200
            Labels:             jenkins=slave
                                jenkins/test=true
            Annotations:        kubernetes.io/psp=ibm-privileged-psp
            Status:             Running
            IP:                 172.30.202.66
            Containers:
              jnlp:
                Container ID:  containerd://60f5b2ad8849852ac09e5afb018d6b625a25710e9618b4d20910387eb763d7c7
                Image:         jenkinsci/jnlp-slave
                Image ID:      docker.io/jenkinsci/jnlp-slave@sha256:1b30012786011b62417a9ee9b612280c62bb9deac0ffd939a8dd31b6633cc0aa
                Port:          <none>
                Host Port:     <none>
                Command:
                  /bin/sh
                  -c
                Args:
                  cat
                State:          Running
                  Started:      Mon, 22 Oct 2018 09:44:36 +0200
                Ready:          True
                Restart Count:  0
                Environment:
                  JENKINS_SECRET:      46c5400d9757f360eeb132758bee5ea444954d558bb8543cd715e09c21a180c9
                  JENKINS_TUNNEL:      my-release-jenkins-agent:50000
                  JENKINS_AGENT_NAME:  test-2jrwj
                  JENKINS_NAME:        test-2jrwj
                  JENKINS_URL:         http://my-release-jenkins:8080/
                  HOME:                /home/jenkins
                Mounts:
                  /home/jenkins from workspace-volume (rw)
                  /var/run/secrets/kubernetes.io/serviceaccount from default-token-9hntr (ro)
            Conditions:
              Type              Status
              Initialized       True
              Ready             True
              ContainersReady   True
              PodScheduled      True
            Volumes:
              workspace-volume:
                Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
                Medium:
              default-token-9hntr:
                Type:        Secret (a volume populated by a Secret)
                SecretName:  default-token-9hntr
                Optional:    false
            QoS Class:       BestEffort
            Node-Selectors:  <none>
            Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                             node.kubernetes.io/unreachable:NoExecute for 300s
            Events:
              Type    Reason     Age   From                     Message
              ----    ------     ----  ----                     -------
              Normal  Scheduled  22s   default-scheduler        Successfully assigned default/test-2jrwj to 10.144.180.238
              Normal  Pulled     21s   kubelet, 10.144.180.238  Container image "jenkinsci/jnlp-slave" already present on machine
              Normal  Created    21s   kubelet, 10.144.180.238  Created container
              Normal  Started    20s   kubelet, 10.144.180.238  Started container
            
            

            Definition:

            Show
            leonbadcam leon gregori added a comment - Okay, i named it jnlp, only 1 container is created now. But im still missing the variable pod_name $ kubectl describe pod test-2jrwj Name: test-2jrwj Namespace: default Priority: 0 PriorityClassName: <none> Node: 10.144.180.238/10.144.180.238 Start Time: Mon, 22 Oct 2018 09:44:34 +0200 Labels: jenkins=slave jenkins/test= true Annotations: kubernetes.io/psp=ibm-privileged-psp Status: Running IP: 172.30.202.66 Containers: jnlp: Container ID: containerd: //60f5b2ad8849852ac09e5afb018d6b625a25710e9618b4d20910387eb763d7c7 Image: jenkinsci/jnlp-slave Image ID: docker.io/jenkinsci/jnlp-slave@sha256:1b30012786011b62417a9ee9b612280c62bb9deac0ffd939a8dd31b6633cc0aa Port: <none> Host Port: <none> Command: /bin/sh -c Args: cat State: Running Started: Mon, 22 Oct 2018 09:44:36 +0200 Ready: True Restart Count: 0 Environment: JENKINS_SECRET: 46c5400d9757f360eeb132758bee5ea444954d558bb8543cd715e09c21a180c9 JENKINS_TUNNEL: my-release-jenkins-agent:50000 JENKINS_AGENT_NAME: test-2jrwj JENKINS_NAME: test-2jrwj JENKINS_URL: http: //my-release-jenkins:8080/ HOME: /home/jenkins Mounts: /home/jenkins from workspace-volume (rw) / var /run/secrets/kubernetes.io/serviceaccount from default -token-9hntr (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: workspace-volume: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: default -token-9hntr: Type: Secret (a volume populated by a Secret) SecretName: default -token-9hntr Optional: false QoS Class : BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 22s default -scheduler Successfully assigned default /test-2jrwj to 10.144.180.238 Normal Pulled 21s kubelet, 10.144.180.238 Container image "jenkinsci/jnlp-slave" already present on machine Normal Created 21s kubelet, 10.144.180.238 Created container Normal Started 20s kubelet, 10.144.180.238 Started container Definition:
            Hide
            csanchez Carlos Sanchez added a comment -

            do you have more pod templates defined and is picking a different one ?

            Show
            csanchez Carlos Sanchez added a comment - do you have more pod templates defined and is picking a different one ?
            Hide
            leonbadcam leon gregori added a comment - - edited

            No its only this one. I will try to do some try and error testing to get the variable somehow into the pod. Will update as soon as i have news

            Show
            leonbadcam leon gregori added a comment - - edited No its only this one. I will try to do some try and error testing to get the variable somehow into the pod. Will update as soon as i have news
            csanchez Carlos Sanchez made changes -
            Remote Link This issue links to "PR-388 (Web Link)" [ 21950 ]
            Hide
            csanchez Carlos Sanchez added a comment -
            Show
            csanchez Carlos Sanchez added a comment - see https://github.com/jenkinsci/kubernetes-plugin/pull/388  for a proper fix
            Hide
            leonbadcam leon gregori added a comment -

            Carlos Sanchez Thanks for this information. I will try it as soon as the pull builds successfull.
            So with this fix in place, pod template out of pipeline can still be used?

            Show
            leonbadcam leon gregori added a comment - Carlos Sanchez Thanks for this information. I will try it as soon as the pull builds successfull. So with this fix in place, pod template out of pipeline can still be used?
            Hide
            csanchez Carlos Sanchez added a comment -

            yes, that fix should work in all cases

            Show
            csanchez Carlos Sanchez added a comment - yes, that fix should work in all cases
            csanchez Carlos Sanchez made changes -
            Summary Kubernetes v1.11 - ChannelClosed after job delaying aborted Pod name is detected as `localhost` in Bluemix IKS
            Hide
            leonbadcam leon gregori added a comment - - edited

            I have build the pull request, it works!
            finally able to execute commands in a container with a simple jenkinsfile like:

            // Developers input is required here.
            def appname = "test"
            def jarname = "test.jar"// Jenkins Deployment Variables and Credentials
            def cloud = env.CLOUD ?: "kubernetes"
            def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id"
            def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id"
            def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id"
            def serviceAccount = env.SERVICE_ACCOUNT ?: "default"
            def sonarCredsID = "sonarID"
            appname = appname.toLowerCase()
            
            // Pod Environment Variables
            def kubenamespace = "default"
            def regnamespace = "blw-msa"
            def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net"podTemplate(label: 'mypod', cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [
                    envVar(key: 'NAMESPACE', value: kubenamespace),
                    envVar(key: 'REGNAMESPACE', value: regnamespace),
            	envVar(key: 'JARNAME', value: jarname),
                    envVar(key: 'APPNAME', value: appname),
                    envVar(key: 'REGISTRY', value: registry)
                ],
                volumes: [
                //hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: '/etc/docker/certs.d'),
                //hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')
            	//hostPathVolume(hostPath: '/tmp/gradle', mountPath: '/root/.gradle/caches/')
                ],
                containers: [
                    // In Theory, java container is not needed as the normal jenkins jnlp slave would be capable of executing gradle.
            	// But as we mount the cache from host, we need a root user running inside the build container, otherwise gradle wont be able to write to .gradle
                    containerTemplate(name: 'java', image: 'openjdk:8-jdk', ttyEnabled: true, command: 'cat')
              ]) {    node('mypod') {
                    stage('Checkout') {
            		echo "test"
            		}
                    container('java') {
            		stage('Static Code Analysis') {
                                		sh "java -version"}
                            }
            }
            }

             

            Show
            leonbadcam leon gregori added a comment - - edited I have build the pull request, it works! finally able to execute commands in a container with a simple jenkinsfile like: // Developers input is required here. def appname = "test" def jarname = "test.jar" // Jenkins Deployment Variables and Credentials def cloud = env.CLOUD ?: "kubernetes" def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id" def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id" def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id" def serviceAccount = env.SERVICE_ACCOUNT ?: " default " def sonarCredsID = "sonarID" appname = appname.toLowerCase() // Pod Environment Variables def kubenamespace = " default " def regnamespace = "blw-msa" def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net" podTemplate(label: 'mypod' , cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [ envVar(key: 'NAMESPACE' , value: kubenamespace), envVar(key: 'REGNAMESPACE' , value: regnamespace), envVar(key: 'JARNAME' , value: jarname), envVar(key: 'APPNAME' , value: appname), envVar(key: 'REGISTRY' , value: registry) ], volumes: [ //hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: '/etc/docker/certs.d' ), //hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: '/ var /run/docker.sock' ) //hostPathVolume(hostPath: '/tmp/gradle' , mountPath: '/root/.gradle/caches/' ) ], containers: [ // In Theory, java container is not needed as the normal jenkins jnlp slave would be capable of executing gradle. // But as we mount the cache from host, we need a root user running inside the build container, otherwise gradle wont be able to write to .gradle containerTemplate(name: 'java' , image: 'openjdk:8-jdk' , ttyEnabled: true , command: 'cat' ) ]) { node( 'mypod' ) { stage( 'Checkout' ) { echo "test" } container( 'java' ) { stage( 'Static Code Analysis' ) { sh "java -version" } } } }  
            Hide
            leonbadcam leon gregori added a comment - - edited

            I can confirm that this issue is fixed now with PR 388
            Our Shared library fully able to run!

            Special Thanks to Carlos Sanchez Vincent Latombe

            Show
            leonbadcam leon gregori added a comment - - edited I can confirm that this issue is fixed now with PR 388 Our Shared library fully able to run! Special Thanks to Carlos Sanchez Vincent Latombe
            csanchez Carlos Sanchez made changes -
            Assignee Carlos Sanchez [ csanchez ] Vincent Latombe [ vlatombe ]
            csanchez Carlos Sanchez made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]

              People

              • Assignee:
                vlatombe Vincent Latombe
                Reporter:
                vxavier Vinicius Xavier
              • Votes:
                2 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: