Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32624

sshagent{} ignored when executed in docker.image().inside{...}

    Details

    • Similar Issues:

      Description

      sshagent('my-ssh-key'){...} is ignored in docker.image('cloudbees:java-build-tools').inside{...}. The same sshagent step works in 'standard' linux node.

      See attached logs

      sshagent(){...} ignored in docker.image(...).inside{...} 
      docker.image ('cloudbees/java-build-tools:0.0.7.1').inside {
          sh 'ls -al ~/.ssh/ || true'
          sshagent(['aws-cleclerc-ssh-key-ubuntu-cleclerc']) {
             sh 'echo SSH_AUTH_SOCK=$SSH_AUTH_SOCK'
             sh 'ls -al $SSH_AUTH_SOCK || true'
             sh "ssh -vvv -o StrictHostKeyChecking=no ubuntu@docker-agent.beesshop.org uname -a"
          }
      }
      
      sshagent(){...} WORKS in node{...} 
      node {
          sh 'ls -al ~/.ssh/ || true'
          sshagent(['aws-cleclerc-ssh-key-ubuntu-cleclerc']) {
             sh 'echo SSH_AUTH_SOCK=$SSH_AUTH_SOCK'
             sh 'ls -al $SSH_AUTH_SOCK || true'
             sh "ssh -vvv -o StrictHostKeyChecking=no ubuntu@docker-agent.beesshop.org uname -a"
          }
      }
      

        Attachments

          Issue Links

            Activity

            Hide
            cleclerc Cyrille Le Clerc added a comment -

            According to Jesse Glick, this issue depends on JENKINS-27152 to handle the tmp directory properly.

            Show
            cleclerc Cyrille Le Clerc added a comment - According to Jesse Glick , this issue depends on JENKINS-27152 to handle the tmp directory properly.
            Hide
            jglick Jesse Glick added a comment -

            A facet of JENKINS-27152.

            Show
            jglick Jesse Glick added a comment - A facet of JENKINS-27152 .
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/com/cloudbees/jenkins/plugins/sshagent/RemoteAgentFactory.java
            src/main/java/com/cloudbees/jenkins/plugins/sshagent/SSHAgentBuildWrapper.java
            src/main/java/com/cloudbees/jenkins/plugins/sshagent/SSHAgentStepExecution.java
            src/main/java/com/cloudbees/jenkins/plugins/sshagent/jna/AgentServer.java
            src/main/java/com/cloudbees/jenkins/plugins/sshagent/jna/JNRRemoteAgent.java
            src/main/java/com/cloudbees/jenkins/plugins/sshagent/jna/JNRRemoteAgentFactory.java
            src/main/java/com/cloudbees/jenkins/plugins/sshagent/jna/JNRRemoteAgentStarter.java
            src/main/java/com/cloudbees/jenkins/plugins/sshagent/mina/MinaRemoteAgentFactory.java
            http://jenkins-ci.org/commit/ssh-agent-plugin/d2e0f53b6eda56012c2bd5d9bc553b31667462a2
            Log:
            JENKINS-32624 JENKINS-27152 Use a standardized temporary directory when possible.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/com/cloudbees/jenkins/plugins/sshagent/RemoteAgentFactory.java src/main/java/com/cloudbees/jenkins/plugins/sshagent/SSHAgentBuildWrapper.java src/main/java/com/cloudbees/jenkins/plugins/sshagent/SSHAgentStepExecution.java src/main/java/com/cloudbees/jenkins/plugins/sshagent/jna/AgentServer.java src/main/java/com/cloudbees/jenkins/plugins/sshagent/jna/JNRRemoteAgent.java src/main/java/com/cloudbees/jenkins/plugins/sshagent/jna/JNRRemoteAgentFactory.java src/main/java/com/cloudbees/jenkins/plugins/sshagent/jna/JNRRemoteAgentStarter.java src/main/java/com/cloudbees/jenkins/plugins/sshagent/mina/MinaRemoteAgentFactory.java http://jenkins-ci.org/commit/ssh-agent-plugin/d2e0f53b6eda56012c2bd5d9bc553b31667462a2 Log: JENKINS-32624 JENKINS-27152 Use a standardized temporary directory when possible.
            Hide
            cleclerc Cyrille Le Clerc added a comment - - edited

            Successfully tested.

            Pipeline Script

            node {
                docker.image ('cloudbees/java-build-tools:0.0.7.1').inside {
                    sh 'whoami'
                    sh 'ls -al ~/.ssh/ || true'
                    sshagent(['aws-cleclerc-ssh-key-ubuntu-cleclerc']) {
                       sh 'echo SSH_AUTH_SOCK=$SSH_AUTH_SOCK'
                       sh 'ls -al $SSH_AUTH_SOCK || true'
                       sh "ssh -o StrictHostKeyChecking=no ubuntu@jenkins-agent.beesshop.org uname -a"
                    }
                }
            }
            

            build logs

            Started by user admin
            [Pipeline] Allocate node : Start
            Running on jenkins-agent.example.com in /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent
            [Pipeline] node {
            [Pipeline] sh
            [test-ssh-agent] Running shell script
            + docker inspect -f . cloudbees/java-build-tools:0.0.7.1
            .
            [Pipeline] Run build steps inside a Docker container : Start
            $ docker run -t -d -u 1000:1000 -w /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent -v /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent:/home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent:rw -v /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp:/home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** cloudbees/java-build-tools:0.0.7.1 cat
            [Pipeline] withDockerContainer {
            [Pipeline] sh
            [test-ssh-agent] Running shell script
            + whoami
            jenkins
            [Pipeline] sh
            [test-ssh-agent] Running shell script
            + ls -al /home/jenkins/.ssh/
            ls: cannot access /home/jenkins/.ssh/: No such file or directory
            + true
            [Pipeline] SSH Agent : Start
            [ssh-agent] Using credentials ubuntu (cleclerc-us-east.pem)
            [ssh-agent] Looking for ssh-agent implementation...
            [ssh-agent]   Java/JNR ssh-agent
            [ssh-agent] Started.
            [Pipeline] sshagent {
            [Pipeline] sh
            [test-ssh-agent] Running shell script
            + echo SSH_AUTH_SOCK=/home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp/ssh330183201329153851
            SSH_AUTH_SOCK=/home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp/ssh330183201329153851
            [Pipeline] sh
            [test-ssh-agent] Running shell script
            + ls -al /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp/ssh330183201329153851
            srw------- 1 jenkins jenkins 0 Mar  4 10:03 /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp/ssh330183201329153851
            [Pipeline] sh
            [test-ssh-agent] Running shell script
            + ssh -o StrictHostKeyChecking=no ubuntu@jenkins-agent.example.com uname -a
            Warning: Permanently added 'jenkins-agent.example.com,52.90.106.69' (ECDSA) to the list of known hosts.
            Linux ip-10-0-0-151 3.13.0-76-generic #120-Ubuntu SMP Mon Jan 18 15:59:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            [Pipeline] } //sshagent
            [Pipeline] SSH Agent : End
            [Pipeline] } //withDockerContainer
            $ docker stop e24c314f6d82ac7d15033a996a56bf33e25096b4b47a0138f1c205559c4adc32
            $ docker rm -f e24c314f6d82ac7d15033a996a56bf33e25096b4b47a0138f1c205559c4adc32
            [Pipeline] Run build steps inside a Docker container : End
            [Pipeline] } //node
            [Pipeline] Allocate node : End
            [Pipeline] End of Pipeline
            Finished: SUCCESS
            
            Show
            cleclerc Cyrille Le Clerc added a comment - - edited Successfully tested. Pipeline Script node { docker.image ( 'cloudbees/java-build-tools:0.0.7.1' ).inside { sh 'whoami' sh 'ls -al ~/.ssh/ || true ' sshagent([ 'aws-cleclerc-ssh-key-ubuntu-cleclerc' ]) { sh 'echo SSH_AUTH_SOCK=$SSH_AUTH_SOCK' sh 'ls -al $SSH_AUTH_SOCK || true ' sh "ssh -o StrictHostKeyChecking=no ubuntu@jenkins-agent.beesshop.org uname -a" } } } build logs Started by user admin [Pipeline] Allocate node : Start Running on jenkins-agent.example.com in /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent [Pipeline] node { [Pipeline] sh [test-ssh-agent] Running shell script + docker inspect -f . cloudbees/java-build-tools:0.0.7.1 . [Pipeline] Run build steps inside a Docker container : Start $ docker run -t -d -u 1000:1000 -w /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent -v /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent:/home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent:rw -v /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp:/home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** cloudbees/java-build-tools:0.0.7.1 cat [Pipeline] withDockerContainer { [Pipeline] sh [test-ssh-agent] Running shell script + whoami jenkins [Pipeline] sh [test-ssh-agent] Running shell script + ls -al /home/jenkins/.ssh/ ls: cannot access /home/jenkins/.ssh/: No such file or directory + true [Pipeline] SSH Agent : Start [ssh-agent] Using credentials ubuntu (cleclerc-us-east.pem) [ssh-agent] Looking for ssh-agent implementation... [ssh-agent] Java/JNR ssh-agent [ssh-agent] Started. [Pipeline] sshagent { [Pipeline] sh [test-ssh-agent] Running shell script + echo SSH_AUTH_SOCK=/home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp/ssh330183201329153851 SSH_AUTH_SOCK=/home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp/ssh330183201329153851 [Pipeline] sh [test-ssh-agent] Running shell script + ls -al /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp/ssh330183201329153851 srw------- 1 jenkins jenkins 0 Mar 4 10:03 /home/ubuntu/jenkins-aws-home/workspace/tests/test-ssh-agent@tmp/ssh330183201329153851 [Pipeline] sh [test-ssh-agent] Running shell script + ssh -o StrictHostKeyChecking=no ubuntu@jenkins-agent.example.com uname -a Warning: Permanently added 'jenkins-agent.example.com,52.90.106.69' (ECDSA) to the list of known hosts. Linux ip-10-0-0-151 3.13.0-76-generic #120-Ubuntu SMP Mon Jan 18 15:59:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux [Pipeline] } //sshagent [Pipeline] SSH Agent : End [Pipeline] } //withDockerContainer $ docker stop e24c314f6d82ac7d15033a996a56bf33e25096b4b47a0138f1c205559c4adc32 $ docker rm -f e24c314f6d82ac7d15033a996a56bf33e25096b4b47a0138f1c205559c4adc32 [Pipeline] Run build steps inside a Docker container : End [Pipeline] } //node [Pipeline] Allocate node : End [Pipeline] End of Pipeline Finished: SUCCESS
            Hide
            sdomula Stan Domula added a comment -

            Still not fully fixed.

            If the socket name gets longer than 108 chars it tries to create the socket in /tmp that doesn't work inside docker. So this is still an issue when using GitHub Organistaion Plugin. There it happens that paths are getting longer.

            Show
            sdomula Stan Domula added a comment - Still not fully fixed. If the socket name gets longer than 108 chars it tries to create the socket in /tmp that doesn't work inside docker. So this is still an issue when using GitHub Organistaion Plugin. There it happens that paths are getting longer.
            Hide
            jspiewak Joshua Spiewak added a comment -

            If one uses the standard location of `/var/lib/jenkins` as the remote root on the Docker daemon agent, for a multi-branch pipeline job the shortest possible path that minimize will generate is too long:

            echo "/var/lib/jenkins/workspace/a-R6R3BLSUWMBHF2ZTDGNA6MR4UMMWUNRCZRZ2356BQKJBSSLTLWYA@tmp/ssh9122165901071866621" | wc -c
            109
            
            Show
            jspiewak Joshua Spiewak added a comment - If one uses the standard location of `/var/lib/jenkins` as the remote root on the Docker daemon agent, for a multi-branch pipeline job the shortest possible path that minimize will generate is too long: echo "/ var /lib/jenkins/workspace/a-R6R3BLSUWMBHF2ZTDGNA6MR4UMMWUNRCZRZ2356BQKJBSSLTLWYA@tmp/ssh9122165901071866621" | wc -c 109
            Hide
            jglick Jesse Glick added a comment -

            There is no “standard location” for Docker-based agent; depends on the image you use. So you need to use something shorter. The path name length limitation for sshagent sockets is an unfortunate artifact of a weird default in typical Linux kernels. I do not know of a satisfactory fix.

            Show
            jglick Jesse Glick added a comment - There is no “standard location” for Docker-based agent; depends on the image you use. So you need to use something shorter. The path name length limitation for sshagent sockets is an unfortunate artifact of a weird default in typical Linux kernels. I do not know of a satisfactory fix.
            Hide
            jspiewak Joshua Spiewak added a comment -

            Sure, we don't have to call it a "standard location" for agents, but given that the Jenkins package on, say, Ubuntu, installs to /var/lib/jenkins, maintaining a similar location on the agents would seem to be a sensible way to keep systems consistent, and far more obvious than setting it to /j.

            Perhaps while this is being iterated upon there could be a feature toggle that could allow folks to fallback to the previous behavior if they need to?

            Show
            jspiewak Joshua Spiewak added a comment - Sure, we don't have to call it a "standard location" for agents, but given that the Jenkins package on, say, Ubuntu, installs to /var/lib/jenkins, maintaining a similar location on the agents would seem to be a sensible way to keep systems consistent, and far more obvious than setting it to /j. Perhaps while this is being iterated upon there could be a feature toggle that could allow folks to fallback to the previous behavior if they need to?
            Hide
            jglick Jesse Glick added a comment -

            Which previous behavior are you referring to?

            Show
            jglick Jesse Glick added a comment - Which previous behavior are you referring to?
            Hide
            erliz Stanislav Vetlovskiy added a comment - - edited

            I faced with the same issue, all was good on simple pipeline, but when i switched to multi-branch pipeline job, tasks in sshagent fall with error

            SSH_AUTH_SOCK=/tmp/ssh3569037635123687207
            + ls -al /tmp/ssh3569037635123687207
            ls: cannot access /tmp/ssh3569037635123687207: No such file or directory
            

            is there any advice to fix it or some workaround?

            Show
            erliz Stanislav Vetlovskiy added a comment - - edited I faced with the same issue, all was good on simple pipeline, but when i switched to multi-branch pipeline job, tasks in sshagent fall with error SSH_AUTH_SOCK=/tmp/ssh3569037635123687207 + ls -al /tmp/ssh3569037635123687207 ls: cannot access /tmp/ssh3569037635123687207: No such file or directory is there any advice to fix it or some workaround?
            Hide
            jspiewak Joshua Spiewak added a comment -

            I am referring to the previous behavior that stored the workspace in sub-folders without the base32 of the hash that was introduced for JENKINS-34564.

            Show
            jspiewak Joshua Spiewak added a comment - I am referring to the previous behavior that stored the workspace in sub-folders without the base32 of the hash that was introduced for JENKINS-34564 .
            Hide
            jglick Jesse Glick added a comment -

            The previous naming pattern for branch projects can be restored with a system property, see release notes.

            Show
            jglick Jesse Glick added a comment - The previous naming pattern for branch projects can be restored with a system property, see release notes.

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                cleclerc Cyrille Le Clerc
              • Votes:
                1 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: