Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49817

Accept kubeconfig file as credentials

    Details

    • Similar Issues:

      Description

      It's unclear to me why the Kubernetes plugin requires so much configuration. In general, Jenkins should be able to talk to Kubernetes with a KUBECONFIG and nothing else. Yet, when I give it a KUBECONFIG via a Jenkins secret, it still seems to connect to 'kubernetes.svc.default' which seems incorrect to me.

       

      (Also, I didn't want to open a new bug for this, but why does this setup require the Jenkins instance to have port 5000 open? That's an operational burden for me...)

        Attachments

          Activity

          Hide
          colemickens Cole Mickens added a comment -

          Further, even when I add the URL, it seems to having TLS problems now. Further indicating that it's probably not treating my "credentials" as a KUBECONFIG. I suspect I'm going to have problems even if I extract the CA cert (and insert it where it asks for the "key"........).

          What "credentials" are even expected to the plugin? There's no indication if it takes a token, a client cert, etc. Thanks.

          Show
          colemickens Cole Mickens added a comment - Further, even when I add the URL, it seems to having TLS problems now. Further indicating that it's probably not treating my "credentials" as a KUBECONFIG. I suspect I'm going to have problems even if I extract the CA cert (and insert it where it asks for the "key"........). What "credentials" are even expected to the plugin? There's no indication if it takes a token, a client cert, etc. Thanks.
          Hide
          colemickens Cole Mickens added a comment -

          Looking further, it looks like I may have been giving Jenkins an invalid KUBECONFIG file due to an azure-cli/aks issue (https://github.com/Azure/azure-cli/issues/5709).

           

          However, I've created a "Secret File" credential with a valid kubeconfig, and it still doesn't really seem to be using it for the server URL / CA cert.

          Show
          colemickens Cole Mickens added a comment - Looking further, it looks like I may have been giving Jenkins an invalid KUBECONFIG file due to an azure-cli/aks issue ( https://github.com/Azure/azure-cli/issues/5709).   However, I've created a "Secret File" credential with a valid kubeconfig, and it still doesn't really seem to be using it for the server URL / CA cert.
          Hide
          colemickens Cole Mickens added a comment -

          I can only get this working when I put the kubeconfig in $JENKINS_HOME. I don't necessarily have access to JENKINS_HOME and want to use a Jenkins secret instead.

          (If anyone can point me to the code, I might be able to help?)

          Show
          colemickens Cole Mickens added a comment - I can only get this working when I put the kubeconfig in $JENKINS_HOME. I don't necessarily have access to JENKINS_HOME and want to use a Jenkins secret instead. (If anyone can point me to the code, I might be able to help?)
          Hide
          csanchez Carlos Sanchez added a comment -

          Kubeconfig can't be used as credentials. Credentials can be certificates, user/passwd, or tokens. You need to extract those from kubeconfig

          Configure Jenkins, adding the Kubernetes cloud under configuration, setting Kubernetes URL to the container engine cluster endpoint or simply https://kubernetes.default.svc.cluster.local. Under credentials, click Add and select Kubernetes Service Account, or alternatively use the Kubernetes API username and password. Select 'Certificate' as credentials type if the kubernetes cluster is configured to use client certificates for authentication.

          Using Kubernetes Service Account will cause the plugin to use the default token mounted inside the Jenkins pod. See Configure Service Accounts for Pods for more information.

          credentials code is in https://github.com/jenkinsci/kubernetes-credentials-plugin/

          Show
          csanchez Carlos Sanchez added a comment - Kubeconfig can't be used as credentials. Credentials can be certificates, user/passwd, or tokens. You need to extract those from kubeconfig Configure Jenkins, adding the Kubernetes cloud under configuration, setting Kubernetes URL to the container engine cluster endpoint or simply https://kubernetes.default.svc.cluster.local . Under credentials, click Add and select Kubernetes Service Account, or alternatively use the Kubernetes API username and password. Select 'Certificate' as credentials type if the kubernetes cluster is configured to use client certificates for authentication. Using Kubernetes Service Account will cause the plugin to use the default token mounted inside the Jenkins pod. See Configure Service Accounts for Pods for more information. credentials code is in https://github.com/jenkinsci/kubernetes-credentials-plugin/
          Hide
          colemickens Cole Mickens added a comment -

          Are you opposed to allowing the user to specify kubeconfig creds? This is how credentials are distributed for every single cluster I've ever used. Most or all deployment tools output a kubeconfig, AKS/GKE both have `get-credentials` commands that output kubeconfig files, etc.

          It looks like the interface might have to be extended, or another added that the kubernetes-plugin could then probe/utilize...

          I think it would be a major UX win if the user could upload a kubeconfig file and be good to. Extracting the URL, CA cert, etc is just throwaway work. Maybe this is a result of how the kubernetes client works?

          Show
          colemickens Cole Mickens added a comment - Are you opposed to allowing the user to specify kubeconfig creds? This is how credentials are distributed for every single cluster I've ever used. Most or all deployment tools output a kubeconfig, AKS/GKE both have `get-credentials` commands that output kubeconfig files, etc. It looks like the interface might have to be extended, or another added that the kubernetes-plugin could then probe/utilize... I think it would be a major UX win if the user could upload a kubeconfig file and be good to. Extracting the URL, CA cert, etc is just throwaway work. Maybe this is a result of how the kubernetes client works?
          Hide
          csanchez Carlos Sanchez added a comment -

          What I've typically seen is the usage of serviceaccounts and tokens, for security reasons you wouldn't use the full admin account in the plugin
          That said I'm not opposed to add an option to use a kubeconfig, the code that checks credentials type is here
          https://github.com/jenkinsci/kubernetes-plugin/blob/master/src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesFactoryAdapter.java#L121

          The kubernetes client can read the kubeconfig from file, that's what the Config.autoConfigure() do, if kubeconfig is present in the filesystem will use it.
          See the code here https://github.com/fabric8io/kubernetes-client/blob/f9b8ea2f25e95a232678ebc0ab66d0ee52490e05/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/Config.java#L372

          Show
          csanchez Carlos Sanchez added a comment - What I've typically seen is the usage of serviceaccounts and tokens, for security reasons you wouldn't use the full admin account in the plugin That said I'm not opposed to add an option to use a kubeconfig, the code that checks credentials type is here https://github.com/jenkinsci/kubernetes-plugin/blob/master/src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesFactoryAdapter.java#L121 The kubernetes client can read the kubeconfig from file, that's what the Config.autoConfigure() do, if kubeconfig is present in the filesystem will use it. See the code here https://github.com/fabric8io/kubernetes-client/blob/f9b8ea2f25e95a232678ebc0ab66d0ee52490e05/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/Config.java#L372
          Hide
          colemickens Cole Mickens added a comment -

          A kubeconfig file doesn't mean or imply a root account. As an example, the kubeconfig I'm pulling apart and handing the plugin currently is for a rather limited user. GKE, Tectonic both issue non-root user accounts this way as well. AKS will when it gets to RBAC.

          Anyway, thank you for the pointers! I'll take a look when I get a chance.

          Show
          colemickens Cole Mickens added a comment - A kubeconfig file doesn't mean or imply a root account. As an example, the kubeconfig I'm pulling apart and handing the plugin currently is for a rather limited user. GKE, Tectonic both issue non-root user accounts this way as well. AKS will when it gets to RBAC. Anyway, thank you for the pointers! I'll take a look when I get a chance.
          Hide
          csanchez Carlos Sanchez added a comment -

          It's not root, but it is not a limited service account specifically tailored for jenkins as defined in https://github.com/jenkinsci/kubernetes-plugin/blob/master/src/main/kubernetes/service-account.yml

          Show
          csanchez Carlos Sanchez added a comment - It's not root, but it is not a limited service account specifically tailored for jenkins as defined in https://github.com/jenkinsci/kubernetes-plugin/blob/master/src/main/kubernetes/service-account.yml
          Show
          csanchez Carlos Sanchez added a comment - PR at https://github.com/jenkinsci/kubernetes-plugin/pull/294

            People

            • Assignee:
              csanchez Carlos Sanchez
              Reporter:
              colemickens Cole Mickens
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: