Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42582

ssh-agent not applied in kubernetes container

    Details

    • Similar Issues:

      Description

      the ssh-agent configuration isn't applied when pipeline is run on a docker container using the kubernetes plugin

       

      as an example, this pipeline works fine:

      sshagent in basic pipeline
      node {
          stage('Pre-Build') {
              sshagent (credentials: ['jenkins-master-ssh']) {
                  sh 'ssh -vT -o "StrictHostKeyChecking=no" git@github.com'
              }
          }
      }
      

      the job will fail, but the console output will clearly show the error from github 

      You've successfully authenticated, but GitHub does not provide shell access.

       

      whereas

      sshagent not applied in container 
      podTemplate(label: 'jenkpod', containers: [containerTemplate(name: 'golang', image: 'golang:1.8', ttyEnabled: true, command: 'cat')]) {
          node ('jenkpod') { container('golang') {
      
              stage('Pre-Build') {
                  sshagent (credentials: ['jenkins-master-ssh']) {
                      sh 'ssh -vT -o "StrictHostKeyChecking=no" git@github.com'
                  }
              }
      
          } }
      }
      

      fails with public key error:

      debug1: Authentications that can continue: publickey
      debug1: Next authentication method: publickey
      debug1: Trying private key: /root/.ssh/id_rsa
      debug1: Trying private key: /root/.ssh/id_dsa
      debug1: Trying private key: /root/.ssh/id_ecdsa
      debug1: Trying private key: /root/.ssh/id_ed25519
      debug1: No more authentication methods to try.
      Permission denied (publickey).

       

      this seems closely related to -JENKINS-32624 sshagent{} ignored when executed in docker.image().inside

       

        Attachments

          Issue Links

            Activity

            Hide
            fgsalomon Fran Garcia added a comment - - edited

            Using version 1.3.2 most of the times it seems to work but sometimes I get the error

            ERROR: Failed to run ssh-add

            (Almost) complete log:

            First time build. Skipping changelog.
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] container
            [Pipeline] {
            [Pipeline] withCredentials
            [Pipeline] {
            [Pipeline] withEnv
            [Pipeline] {
            [Pipeline] timeout
            Timeout set to expire in 1 hr 0 min
            [Pipeline] {
            [Pipeline] stage
            [Pipeline] { (Test)
            [Pipeline] container
            [Pipeline] {
            [Pipeline] sshagent
            [ssh-agent] Using credentials git-repos (SSH credential for Git repos)
            [ssh-agent] Looking for ssh-agent implementation...
            [ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
            Executing shell script inside container [ubuntu] of pod [bowiepod-47nkj-nld78]
            Executing command: "ssh-agent" 
            printf "EXITCODE %3d" $?; exit
            SSH_AUTH_SOCK=/tmp/ssh-oYnHo47QlMEc/agent.22; export SSH_AUTH_SOCK;
            SSH_AGENT_PID=23; export SSH_AGENT_PID;
            echo Agent pid 23;
            EXITCODE   0SSH_AUTH_SOCK=/tmp/ssh-oYnHo47QlMEc/agent.22
            SSH_AGENT_PID=23
            Executing shell script inside container [ubuntu] of pod [bowiepod-47nkj-nld78]
            Executing command: "ssh-add" "/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key" 
            printf "EXITCODE %3d" $?; exit
            EXITCODE   0EXITCODE   0Identity added: /home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key (/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key)
            Identity added: /home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key (/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key)
            [Pipeline] // sshagent
            [Pipeline] }
            [Pipeline] // container
            Post stage
            [Pipeline] junit
            Recording test results
            No test report files were found. Configuration error?
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] stage
            [Pipeline] { (Doc)
            Stage 'Doc' skipped due to earlier failure(s)
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] stage
            [Pipeline] { (Lint)
            Stage 'Lint' skipped due to earlier failure(s)
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] stage
            [Pipeline] { (SonarQube)
            Stage 'SonarQube' skipped due to earlier failure(s)
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] stage
            [Pipeline] { (Deploy)
            Stage 'Deploy' skipped due to earlier failure(s)
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] stage
            [Pipeline] { (Declarative: Post Actions)
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] }
            [Pipeline] // timeout
            [Pipeline] }
            [Pipeline] // withEnv
            [Pipeline] }
            [Pipeline] // withCredentials
            [Pipeline] }
            [Pipeline] // container
            [Pipeline] }
            [Pipeline] // node
            [Pipeline] }
            [Pipeline] // podTemplate
            [Pipeline] End of Pipeline
            ERROR: Failed to run ssh-add
            Finished: FAILURE

            Can it be a race condition?

            Show
            fgsalomon Fran Garcia added a comment - - edited Using version 1.3.2 most of the times it seems to work but sometimes I get the error ERROR: Failed to run ssh-add (Almost) complete log: First time build. Skipping changelog. [Pipeline] } [Pipeline] // stage [Pipeline] container [Pipeline] { [Pipeline] withCredentials [Pipeline] { [Pipeline] withEnv [Pipeline] { [Pipeline] timeout Timeout set to expire in 1 hr 0 min [Pipeline] { [Pipeline] stage [Pipeline] { (Test) [Pipeline] container [Pipeline] { [Pipeline] sshagent [ssh-agent] Using credentials git-repos (SSH credential for Git repos) [ssh-agent] Looking for ssh-agent implementation... [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine) Executing shell script inside container [ubuntu] of pod [bowiepod-47nkj-nld78] Executing command: "ssh-agent" printf "EXITCODE %3d" $?; exit SSH_AUTH_SOCK=/tmp/ssh-oYnHo47QlMEc/agent.22; export SSH_AUTH_SOCK; SSH_AGENT_PID=23; export SSH_AGENT_PID; echo Agent pid 23; EXITCODE 0SSH_AUTH_SOCK=/tmp/ssh-oYnHo47QlMEc/agent.22 SSH_AGENT_PID=23 Executing shell script inside container [ubuntu] of pod [bowiepod-47nkj-nld78] Executing command: "ssh-add" "/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key" printf "EXITCODE %3d" $?; exit EXITCODE 0EXITCODE 0Identity added: /home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key (/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key) Identity added: /home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key (/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key) [Pipeline] // sshagent [Pipeline] } [Pipeline] // container Post stage [Pipeline] junit Recording test results No test report files were found. Configuration error? [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Doc) Stage 'Doc' skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Lint) Stage 'Lint' skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (SonarQube) Stage 'SonarQube' skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Deploy) Stage 'Deploy' skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Declarative: Post Actions) [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // timeout [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline ERROR: Failed to run ssh-add Finished: FAILURE Can it be a race condition?
            Hide
            berni_ Bernhard Kaszt added a comment -

            We also sometimes get the exactly same error. It often works, but randomly throws this error.

            And it's not clear why it fails then. There is no error message except that ssh-add failed to run.

             

            Show
            berni_ Bernhard Kaszt added a comment - We also sometimes get the exactly same error. It often works, but randomly throws this error. And it's not clear why it fails then. There is no error message except that ssh-add failed to run.  
            Hide
            akmjenkins ASHOK MOHANTY added a comment -

            With Kub ver 1.12.4 and kub-Credential 0.3.1, I am seeing this ssh-agent issue. Can you please check.
            [ssh-agent] Looking for ssh-agent implementation...
            Could not find ssh-agent: IOException: container [dind] does not exist in pod [jenkins-slave-lcmzb-0g359]
            Check if ssh-agent is installed and in PATH
            [ssh-agent] Java/JNR ssh-agent

            And then finally, it shows this error -

            [Pipeline] End of Pipeline
            java.io.IOException: container [dind] does not exist in pod [jenkins-slave-lcmzb-0g359]
            at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:435)

            Show
            akmjenkins ASHOK MOHANTY added a comment - With Kub ver 1.12.4 and kub-Credential 0.3.1, I am seeing this ssh-agent issue. Can you please check. [ssh-agent] Looking for ssh-agent implementation... Could not find ssh-agent: IOException: container [dind] does not exist in pod [jenkins-slave-lcmzb-0g359] Check if ssh-agent is installed and in PATH [ssh-agent] Java/JNR ssh-agent And then finally, it shows this error - [Pipeline] End of Pipeline java.io.IOException: container [dind] does not exist in pod [jenkins-slave-lcmzb-0g359] at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:435)
            Hide
            jglick Jesse Glick added a comment -

            ASHOK MOHANTY whatever you are seeing, it sounds like an unrelated issue, and perhaps just a user error—incorrect pod definition.

            Show
            jglick Jesse Glick added a comment - ASHOK MOHANTY whatever you are seeing, it sounds like an unrelated issue, and perhaps just a user error—incorrect pod definition.
            Hide
            akmjenkins ASHOK MOHANTY added a comment -

            Thanks, able to catch the user  code issue.

            Show
            akmjenkins ASHOK MOHANTY added a comment - Thanks, able to catch the user  code issue.

              People

              • Assignee:
                seakip18 Matthew Ludlum
                Reporter:
                jknurek J Knurek
              • Votes:
                6 Vote for this issue
                Watchers:
                22 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: