Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60105

Kubernetes plugin fails with EKS

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • kubernetes-plugin
    • None
    • kubernetes 1.24.0

      I'm using a Jenkins installation which is run on a standard ec2 instance. This Jenkins instance is supposed to connect to an EKS cluster. EKS cluster auth is done via aws-iam-authenticator which is correctly configured on the machine. I've put all the config in the .kube/config for the jenkins user.
      The kubernetes plugin will pick up the k8s config just fine and succeed through all connection tests. It will also run builds just fine for about 15 minutes. At that point, the token issues from aws iam authenticator will be expired and no more pods can be created for my builds. Configuration tests do still work after that, so I guess the plugin creates a new client for those.

      The plugin will print the following error message:

       2019-11-08 11:32:50.552+0000 [id=24] WARNING o.c.j.p.k.KubernetesCloud#provision: Failed to count the # of live instances on Kubernetes2019-11-08 11:32:50.552+0000 [id=24] WARNING o.c.j.p.k.KubernetesCloud#provision: Failed to count the # of live instances on Kubernetesio.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: [https://....eu-west-1.eks.amazonaws.com/api/v1/namespaces/jenkins/pods?labelSelector=app.kubernetes.io%2Fpart-of%3Djenkins%2Capp.kubernetes.io%2Fcomponent%3Dslave]. Message: Unauthorized! Token may have expired! Please log-in again. Unauthorized. at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:510) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:447) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:413) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:372) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:354) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:153) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:620) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:69) at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.getActiveSlavePods(KubernetesCloud.java:615) at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.addProvisionedSlave(KubernetesCloud.java:590) at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.provision(KubernetesCloud.java:542) at hudson.slaves.NodeProvisioner$StandardStrategyImpl.apply(NodeProvisioner.java:729) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:332) at hudson.slaves.NodeProvisioner.access$900(NodeProvisioner.java:63) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:823) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
      
      

            vlatombe Vincent Latombe
            jniebuhr Jochen Niebuhr
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: