Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20879

SSH Credentials (private key with passphrase) do not work

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: git-client-plugin
    • Labels:
    • Environment:
    • Similar Issues:

      Description

      Am I doing something wrong, or is this not something that is supported?

      When I try to add a Git SCM using SSH credentials, it fails with the following error (see SCMError.png):

      	
      Failed to connect to repository : Command "ls-remote -h git@github.com:AppDirect/StandingCloud.git HEAD" returned status code 128:
      stdout: 
      stderr: Permission denied (publickey). 
      fatal: The remote end hung up unexpectedly
      

      I've configured the private key properly (as far as I know anyway)... See PrivateKeyConfiguration.png

      I've also tried the same configuration under a credential domain in case git was hung with a message like this (See CredentialDomain.png):

      The authenticity of host 'github.com (192.30.252.128)' can't be established.
      RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
      Are you sure you want to continue connecting (yes/no)?
      

      I've tested and the private key that I am using does have access:

      [root@jenkins ~]# ssh -T -i /dev/shm/id_rsa git@github.com
      The authenticity of host 'github.com (192.30.252.128)' can't be established.
      RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
      Are you sure you want to continue connecting (yes/no)? yes
      Warning: Permanently added 'github.com,192.30.252.128' (RSA) to the list of known hosts.
      Enter passphrase for key '/dev/shm/id_rsa': 
      Hi nshenry03! You've successfully authenticated, but GitHub does not provide shell access.
      

      As a workaround I can add/create a key as the jenkins user; however, it would be great if I could use the SSH Credentials plugin so that the key is backed up and restored if I move to a new Jenkins server.

        Attachments

          Issue Links

            Activity

            nshenry03 Nick Henry created issue -
            Hide
            richvdh Richard van der Hoff added a comment -

            My impression is that the git plugin doesn't work with ssh keys which have a non-empty passphrase. Can you confirm if it works ok on a key with an empty passphrase?

            Show
            richvdh Richard van der Hoff added a comment - My impression is that the git plugin doesn't work with ssh keys which have a non-empty passphrase. Can you confirm if it works ok on a key with an empty passphrase?
            dnozay Damien Nozay made changes -
            Field Original Value New Value
            Summary SSH Credentials do not work to connect to private GitHub repository SSH Credentials (private key with passphrase) do not work
            Hide
            dnozay Damien Nozay added a comment -

            The issue is not limited to private GitHub repositories.
            I have confirmed on my system that a non-empty passphrase is the issue.

            Show
            dnozay Damien Nozay added a comment - The issue is not limited to private GitHub repositories. I have confirmed on my system that a non-empty passphrase is the issue.
            Hide
            dnozay Damien Nozay added a comment -

            /var/log/secure when attempting to use credentials that have a passphrase
            Feb 25 11:16:57 git sshd[32273]: Failed password for jenkins from X.X.X.X port 57524 ssh2
            Feb 25 11:16:57 git sshd[32273]: Failed password for jenkins from X.X.X.X port 57524 ssh2
            Feb 25 11:16:57 git sshd[32274]: Connection closed by X.X.X.X

            /var/log/secure when attempting to use credentials that do not have a passphrase
            Feb 25 11:17:00 git sshd[32276]: Accepted publickey for jenkins from X.X.X.X port 57526 ssh2
            Feb 25 11:17:00 git sshd[32276]: pam_unix(sshd:session): session opened for user jenkins by (uid=0)
            Feb 25 11:17:00 git sshd[32279]: Received disconnect from X.X.X.X: 11: disconnected by user
            Feb 25 11:17:00 git sshd[32276]: pam_unix(sshd:session): session closed for user jenkins

            Show
            dnozay Damien Nozay added a comment - /var/log/secure when attempting to use credentials that have a passphrase Feb 25 11:16:57 git sshd [32273] : Failed password for jenkins from X.X.X.X port 57524 ssh2 Feb 25 11:16:57 git sshd [32273] : Failed password for jenkins from X.X.X.X port 57524 ssh2 Feb 25 11:16:57 git sshd [32274] : Connection closed by X.X.X.X /var/log/secure when attempting to use credentials that do not have a passphrase Feb 25 11:17:00 git sshd [32276] : Accepted publickey for jenkins from X.X.X.X port 57526 ssh2 Feb 25 11:17:00 git sshd [32276] : pam_unix(sshd:session): session opened for user jenkins by (uid=0) Feb 25 11:17:00 git sshd [32279] : Received disconnect from X.X.X.X: 11: disconnected by user Feb 25 11:17:00 git sshd [32276] : pam_unix(sshd:session): session closed for user jenkins
            Hide
            markewaite Mark Waite added a comment -

            Does the credentials plugin support ssh credentials which use a passphrase? I don't recall ever seeing any any location to enter the passphrase, so I assumed there was no support in Jenkins overall for ssh credentials with passphrases.

            Show
            markewaite Mark Waite added a comment - Does the credentials plugin support ssh credentials which use a passphrase? I don't recall ever seeing any any location to enter the passphrase, so I assumed there was no support in Jenkins overall for ssh credentials with passphrases.
            Hide
            sbfawkes Simon Fawkes added a comment -

            Is anyone planning on fixing this issue soon

            Show
            sbfawkes Simon Fawkes added a comment - Is anyone planning on fixing this issue soon
            Hide
            markewaite Mark Waite added a comment -

            I don't think there is any plan to support ssh keys which require a passphrase. You'll need to use an ssh key which does not require a passphrase.

            I can confirm that ssh keys without a passphrase work very well both to GitHub and to other ssh repositories (like Debian Linux).

            Show
            markewaite Mark Waite added a comment - I don't think there is any plan to support ssh keys which require a passphrase. You'll need to use an ssh key which does not require a passphrase. I can confirm that ssh keys without a passphrase work very well both to GitHub and to other ssh repositories (like Debian Linux).
            Hide
            tfnico tfnico added a comment -

            Private key in the text field works fine for me, both with and without passphrase.

            Show
            tfnico tfnico added a comment - Private key in the text field works fine for me, both with and without passphrase.
            Hide
            markewaite Mark Waite added a comment - - edited

            tfnico Could you explain further how you configure Jenkins to allow a private key with a passphrase to work with the git plugin?

            You said that "Private key in the text field works fine for me, both with and without passphrase", yet when I've added a credential to Jenkins using a private key which requires a passphrase, even if I enter the passphrase, I'm not able to use that private key based credential from the git plugin to checkout a repository whose access is controlled by that private key.

            The steps I took in my failed attempt included:

            1. Create a new user named "private" on my Debian Linux 7.5 (Wheezy)
              $ sudo useradd private
            2. Use ssh-keygen to define a passphrase protected ssh key for that user
              $ sudo su - private
              $ ssh-keygen
            3. Configure git for the "private" user
              $ git config --global user.name "Private User"
              $ git config --global user.email mwaite@wheezy64b
            4. Create a git repository in the .ssh directory
              $ cd .ssh
              $ git init
              $ git add .
              $ git commit -m "First checkin"
            5. Create a bare git repository copy of that .ssh directory
              $ cd ~
              $ git clone --bare .ssh ssh.git
            6. Configure bare repository as "origin" of .ssh repo
              $ cd ~/.ssh
              $ git remote add origin ../ssh.git
              $ git branch --set-upstream master origin/
            7. Allow user "private" to login without password prompt if agent provides key
              $ cd ~/.ssh
              $ cp id_rsa.pub authorized_keys
            8. Confirm user "private" can login without password prompt
              $ eval $(ssh-agent)
              $ ssh-add
              $ ssh wheezy64b ls .ssh
            9. Confirm user "private" can clone ssh repo without password prompt
              $ git clone ssh://localhost/~private/ssh.git tmp
            10. Add private key from the user "private" to Jenkins as credential, including the passphrase in "Advanced"
            11. Define a job which uses that credential
            12. Confirm that "git ls-remote" fails to connect during job definition
            13. Confirm that the job fails with message that credentials were not correct
            Show
            markewaite Mark Waite added a comment - - edited tfnico Could you explain further how you configure Jenkins to allow a private key with a passphrase to work with the git plugin? You said that "Private key in the text field works fine for me, both with and without passphrase", yet when I've added a credential to Jenkins using a private key which requires a passphrase, even if I enter the passphrase, I'm not able to use that private key based credential from the git plugin to checkout a repository whose access is controlled by that private key. The steps I took in my failed attempt included: Create a new user named "private" on my Debian Linux 7.5 (Wheezy) $ sudo useradd private Use ssh-keygen to define a passphrase protected ssh key for that user $ sudo su - private $ ssh-keygen Configure git for the "private" user $ git config --global user.name "Private User" $ git config --global user.email mwaite@wheezy64b Create a git repository in the .ssh directory $ cd .ssh $ git init $ git add . $ git commit -m "First checkin" Create a bare git repository copy of that .ssh directory $ cd ~ $ git clone --bare .ssh ssh.git Configure bare repository as "origin" of .ssh repo $ cd ~/.ssh $ git remote add origin ../ssh.git $ git branch --set-upstream master origin/ Allow user "private" to login without password prompt if agent provides key $ cd ~/.ssh $ cp id_rsa.pub authorized_keys Confirm user "private" can login without password prompt $ eval $(ssh-agent) $ ssh-add $ ssh wheezy64b ls .ssh Confirm user "private" can clone ssh repo without password prompt $ git clone ssh://localhost/~private/ssh.git tmp Add private key from the user "private" to Jenkins as credential, including the passphrase in "Advanced" Define a job which uses that credential Confirm that "git ls-remote" fails to connect during job definition Confirm that the job fails with message that credentials were not correct
            Hide
            unguiculus unguiculus added a comment -

            I'm having the same issue for keys with a passphrase. The Credentials Plugin does support SSH credentials with a passphrase because the SSH Agent Plugin uses them correctly. So, as a workaround, you can just leave the Git credentials empty and use the SSH Agent plugin.

            Show
            unguiculus unguiculus added a comment - I'm having the same issue for keys with a passphrase. The Credentials Plugin does support SSH credentials with a passphrase because the SSH Agent Plugin uses them correctly. So, as a workaround, you can just leave the Git credentials empty and use the SSH Agent plugin.
            Hide
            fnawroth_bp Felix Nawroth added a comment -

            Can you estimate when this will be fixed? To use the SSH Agent plugin as workaround would cost us a lot of configuration effort, since we're building on Windows, too.

            I can confirm that the Credentials plugin with password-protected SSH keys works well in general, both with key on disc/password in Jenkins and key in Jenkins/password in Jenkins. For example we connect our SSH slaves this way. Only Git fails to use the password-protected keys.

            Show
            fnawroth_bp Felix Nawroth added a comment - Can you estimate when this will be fixed? To use the SSH Agent plugin as workaround would cost us a lot of configuration effort, since we're building on Windows, too. I can confirm that the Credentials plugin with password-protected SSH keys works well in general, both with key on disc/password in Jenkins and key in Jenkins/password in Jenkins. For example we connect our SSH slaves this way. Only Git fails to use the password-protected keys.
            Hide
            markewaite Mark Waite added a comment -

            Felix Nawroth I found that I can use a passphrase protected ssh key with the latest git plugin (2.2.5) and the latest git client plugin (1.10.1) if I use a git URL of the form ssh://username@hostname/repository. That form works as expected on both Windows and Linux, with the ssh passphrase entered into the "Advanced" section of the credentials panel.

            I don't have any estimate for when this will be fixed. I don't plan to work on fixing it, especially since I've found at least one URL form which works as expected with a passphrase protected ssh key.

            I have recently been able to add an authentication test case to the git client plugin automated tests, so there is progress towards allowing further testing of these types of cases, but I don't intend to work on this specific case, since I believe it is relatively uncommon, and has a work around.

            Show
            markewaite Mark Waite added a comment - Felix Nawroth I found that I can use a passphrase protected ssh key with the latest git plugin (2.2.5) and the latest git client plugin (1.10.1) if I use a git URL of the form ssh://username@hostname/repository. That form works as expected on both Windows and Linux, with the ssh passphrase entered into the "Advanced" section of the credentials panel. I don't have any estimate for when this will be fixed. I don't plan to work on fixing it, especially since I've found at least one URL form which works as expected with a passphrase protected ssh key. I have recently been able to add an authentication test case to the git client plugin automated tests, so there is progress towards allowing further testing of these types of cases, but I don't intend to work on this specific case, since I believe it is relatively uncommon, and has a work around.
            Hide
            tduong Townsend Duong added a comment -

            I am getting this error with both forms of URL ssh://hostname:port/repository and ssh://username@hostname:port/repository with an SSH key. If I change to a stored credential that uses an SSH key with no passphrase, then the ssh://username@hostname:port/repository URL form works.

            The form without the username should work when the stored username in the credential gets used. It might also work now if the user running jenkins has a ~/.ssh/config file specifying the SSH username to be used with this git server, as I do this in one of my jenkins installations where all the SSH keys are managed in the OS filesystem level instead of the stored credentials. But in my new test system, only the ssh://username@hostname:port/repository form is working.

            This test system is running with git plugin 2.2.6 and git client plugin 1.10.2 on Linux.

            Show
            tduong Townsend Duong added a comment - I am getting this error with both forms of URL ssh://hostname:port/repository and ssh://username@hostname:port/repository with an SSH key. If I change to a stored credential that uses an SSH key with no passphrase, then the ssh://username@hostname:port/repository URL form works. The form without the username should work when the stored username in the credential gets used. It might also work now if the user running jenkins has a ~/.ssh/config file specifying the SSH username to be used with this git server, as I do this in one of my jenkins installations where all the SSH keys are managed in the OS filesystem level instead of the stored credentials. But in my new test system, only the ssh://username@hostname:port/repository form is working. This test system is running with git plugin 2.2.6 and git client plugin 1.10.2 on Linux.
            danielbeck Daniel Beck made changes -
            Link This issue is duplicated by JENKINS-25194 [ JENKINS-25194 ]
            Hide
            jglick Jesse Glick added a comment -

            JENKINS-25194 has some more details and a suggestion to not rely on SSH_ASKPASS. The simplest workaround is to switch the job to use JGit, for which a passphrase works fine (for me).

            Show
            jglick Jesse Glick added a comment - JENKINS-25194 has some more details and a suggestion to not rely on SSH_ASKPASS . The simplest workaround is to switch the job to use JGit, for which a passphrase works fine (for me).
            jglick Jesse Glick made changes -
            Component/s git [ 15543 ]
            jglick Jesse Glick made changes -
            Labels ssh
            markewaite Mark Waite made changes -
            Assignee Nicolas De Loof [ ndeloof ]
            markewaite Mark Waite made changes -
            Link This issue is duplicated by JENKINS-27998 [ JENKINS-27998 ]
            Hide
            tom_gl Thomas de Grenier de Latour added a comment - - edited

            Here is an attempt at fixing polling with GitCli and a passphrase-protected SSH private key:
            https://github.com/jenkinsci/git-client-plugin/pull/168
            Works for me on Ubuntu 12.04, with this patch on top of the git-client-plugin 1.17.1 and git-plugin 2.3.5 (and git 2.4.0 from the git-core PPA).

            Show
            tom_gl Thomas de Grenier de Latour added a comment - - edited Here is an attempt at fixing polling with GitCli and a passphrase-protected SSH private key: https://github.com/jenkinsci/git-client-plugin/pull/168 Works for me on Ubuntu 12.04, with this patch on top of the git-client-plugin 1.17.1 and git-plugin 2.3.5 (and git 2.4.0 from the git-core PPA).
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Thomas de Grenier de Latour
            Path:
            src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
            http://jenkins-ci.org/commit/git-client-plugin/21bc7357e20649bf3c96ed1cd12471ea330abe5d
            Log:
            JENKINS-20879 - make sure $SSH_ASKPASS is not ignored

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Thomas de Grenier de Latour Path: src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java http://jenkins-ci.org/commit/git-client-plugin/21bc7357e20649bf3c96ed1cd12471ea330abe5d Log: JENKINS-20879 - make sure $SSH_ASKPASS is not ignored
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Mark Waite
            Path:
            src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
            http://jenkins-ci.org/commit/git-client-plugin/0e67e3d4003124d98db540b86f27d0c4d1347493
            Log:
            Merge pull request #168 from thomasgl-orange/JENKINS-20879

            JENKINS-20879 - make sure $SSH_ASKPASS is not ignored on Unix.

            Does not fix the passphrase protected private keys on Windows,
            just on Unix machines.

            Compare: https://github.com/jenkinsci/git-client-plugin/compare/2f0cd5d3ce52...0e67e3d40031

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Mark Waite Path: src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java http://jenkins-ci.org/commit/git-client-plugin/0e67e3d4003124d98db540b86f27d0c4d1347493 Log: Merge pull request #168 from thomasgl-orange/ JENKINS-20879 JENKINS-20879 - make sure $SSH_ASKPASS is not ignored on Unix. Does not fix the passphrase protected private keys on Windows, just on Unix machines. Compare: https://github.com/jenkinsci/git-client-plugin/compare/2f0cd5d3ce52...0e67e3d40031
            Hide
            markewaite Mark Waite added a comment -

            Thanks Thomas de Grenier de Latour for the pull request. It should be included in the next release of the git client plugin (likely either 1.17.2 or 1.18.0).

            Please note that the pull request only helped in my testing on Linux machines, not on Windows. The Windows machines continue to behave incorrectly with command line git when using a passphrase protected private key. Refer to the comments in the pull request for more details on the cases tested and the test results.

            Show
            markewaite Mark Waite added a comment - Thanks Thomas de Grenier de Latour for the pull request. It should be included in the next release of the git client plugin (likely either 1.17.2 or 1.18.0). Please note that the pull request only helped in my testing on Linux machines, not on Windows. The Windows machines continue to behave incorrectly with command line git when using a passphrase protected private key. Refer to the comments in the pull request for more details on the cases tested and the test results.
            markewaite Mark Waite made changes -
            Link This issue is duplicated by JENKINS-20638 [ JENKINS-20638 ]
            Hide
            markewaite Mark Waite added a comment -

            The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin. Problems detected in the pre-release should be e-mailed to Mark Waite and Nicolas De Loof.

            I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases.

            Show
            markewaite Mark Waite added a comment - The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin . Problems detected in the pre-release should be e-mailed to Mark Waite and Nicolas De Loof . I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases.
            Hide
            markewaite Mark Waite added a comment -

            Included in git client plugin 1.18.0 released 18 July 2015 as fix for Unix based clients. Not clear that a fix is possible for Windows based clients.

            Show
            markewaite Mark Waite added a comment - Included in git client plugin 1.18.0 released 18 July 2015 as fix for Unix based clients. Not clear that a fix is possible for Windows based clients.
            markewaite Mark Waite made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            markewaite Mark Waite made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            Hide
            dejayc Dejay Clayton added a comment - - edited

            I'm experiencing this issue for the first time, after my coworker updated all Jenkins plugins. git-client was upgraded to v1.18.0. I'm not sure what version was previously installed.

            Keys without passwords work fine; keys with passwords no longer work.

            If I log into the shell account for the Jenkins tomcat user, I see the following message echoed to the screen whenever Jenkins attempts to verify the Git credentials:

            "Enter passphrase for key '/home/tomcat/tomcat/temp/ssh8175827570904316216key':

            This problem persists in git-client versions 1.18 and 1.19. I'm not sure what version was installed when passwords seemed to work properly.

            I'm using Jenkins 1.622 on Ubuntu 14.04.3 LTS, and Jenkins 1.617 on Yosemite. My repositories are hosted on GitHub.

            Show
            dejayc Dejay Clayton added a comment - - edited I'm experiencing this issue for the first time, after my coworker updated all Jenkins plugins. git-client was upgraded to v1.18.0. I'm not sure what version was previously installed. Keys without passwords work fine; keys with passwords no longer work. If I log into the shell account for the Jenkins tomcat user, I see the following message echoed to the screen whenever Jenkins attempts to verify the Git credentials: "Enter passphrase for key '/home/tomcat/tomcat/temp/ssh8175827570904316216key': This problem persists in git-client versions 1.18 and 1.19. I'm not sure what version was installed when passwords seemed to work properly. I'm using Jenkins 1.622 on Ubuntu 14.04.3 LTS, and Jenkins 1.617 on Yosemite. My repositories are hosted on GitHub.
            dejayc Dejay Clayton made changes -
            Resolution Fixed [ 1 ]
            Status Closed [ 6 ] Reopened [ 4 ]
            Hide
            markewaite Mark Waite added a comment - - edited

            Dejay Clayton sorry to hear that it is not working for you. It will need more investigation to understand if the problem can be duplicated.

            In the future, please don't quadruple my work by reopening the bug and the 3 duplicates with the same comment in each of the reopened bug reports. There are relatively few working on the plugin, and making those few people work on a bug and 3 duplicates seems wasteful.

            When I performed the testing, I used git client plugin 1.18.0 and git plugin 2.4.0. I do not have access to a MacOS system, so my testing was performed on CentOS 6, CentOS 7, Debian 6, Debian 7, Debian 8, and Ubuntu 14. The testing passed on those platforms. The testing failed on all the tested Windows platforms (Windows 7, Windows 8, Windows Server 2011).

            Show
            markewaite Mark Waite added a comment - - edited Dejay Clayton sorry to hear that it is not working for you. It will need more investigation to understand if the problem can be duplicated. In the future, please don't quadruple my work by reopening the bug and the 3 duplicates with the same comment in each of the reopened bug reports. There are relatively few working on the plugin, and making those few people work on a bug and 3 duplicates seems wasteful. When I performed the testing, I used git client plugin 1.18.0 and git plugin 2.4.0. I do not have access to a MacOS system, so my testing was performed on CentOS 6, CentOS 7, Debian 6, Debian 7, Debian 8, and Ubuntu 14. The testing passed on those platforms. The testing failed on all the tested Windows platforms (Windows 7, Windows 8, Windows Server 2011).
            Hide
            markewaite Mark Waite added a comment - - edited

            Steps I took to confirm it was working on my existing Ubuntu 14.04 x64 master (previously using git client plugin 1.18.0, then git client plugin 1.19.0, and now using an unreleased version of the git client plugin that is adding submodule authentication):

            1. Create a new user "hasphrase"
            2. Login as user "hasphrase"
            3. Generate passphrase protected private key
            4. Create a private git repo for that user at /var/lib/git/hasphrase/bin.git
            5. Confirm other system users cannot clone that repo
            6. Create multi-configuration Jenkins job attempting to use that repo, use Elastic Axis plugin and Platform Labeler plugin to run job on "windows || linux || freebsd", restrict polling to only run from linux machines
            7. Confirm Jenkins job cannot read the repo (exception polling, exception building)
            8. Define Jenkins credential with private key and passphrase of user "hasphrase"
            9. Modify Jenkins job to use that newly defined credential
            10. Confirm job can read the repo (no exception polling, linux and freebsd slaves succeed, windows slaves fail)

            Steps I took to confirm it failed on a freshly constructed Docker instance (using git client plugin 1.19.0 and git plugin 2.4.0):

            1. Run my [master-with-plugins Docker instance]()
            2. Define job attempting to use that repo
            3. Confirm job cannot read the repo (no credential defined)
            4. Define new credential with the passphrase protected private key

            Exception reported while defining the credential in the Docker instance:

                Caused by: java.lang.NullPointerException
                    at com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey.getPrivateKeys(BasicSSHUserPrivateKey.java:126)
                    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.createSshKeyFile(CliGitAPIImpl.java:1432)
                    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1300)
                    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1282)
                    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1273)
                    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:2404)
                    at hudson.plugins.git.UserRemoteConfig$DescriptorImpl.doCheckUrl(UserRemoteConfig.java:156)
                    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                    at java.lang.reflect.Method.invoke(Method.java:497)
                    at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298)
                    at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161)
                    at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96)
                    at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:121)
                    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
                    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
                    ... 56 more
            

            Additional attempts included checking that the Docker instance could use a private key without passphrase (which it could), and that the prompt for the host key fingerprint was not the root reason why the job failed (it did not seem to be the root reason, since even after the private key without passphrase worked, the private key with passphrase was still prompting for the password).

            When I copied the job which uses the passphrase protected key, the polling log reported there was no existing build so it scheduled a build without polling. When that job ran, it hung with a prompt in the Docker window requesting a passphrase for a temporary key file.

                Enter passphrase for key '/tmp/ssh5560754303468136093key':
            

            Future experiments might include:

            1. Define the DISPLAY environment variable for Jenkins system-wide
            2. Try unreleased git client plugin on Docker instance
            3. Review createSshKeyFile null pointer exception
            4. Implement passphrase protected private key test in CredentialsTest
            Show
            markewaite Mark Waite added a comment - - edited Steps I took to confirm it was working on my existing Ubuntu 14.04 x64 master (previously using git client plugin 1.18.0, then git client plugin 1.19.0, and now using an unreleased version of the git client plugin that is adding submodule authentication): Create a new user "hasphrase" Login as user "hasphrase" Generate passphrase protected private key Create a private git repo for that user at /var/lib/git/hasphrase/bin.git Confirm other system users cannot clone that repo Create multi-configuration Jenkins job attempting to use that repo, use Elastic Axis plugin and Platform Labeler plugin to run job on "windows || linux || freebsd", restrict polling to only run from linux machines Confirm Jenkins job cannot read the repo (exception polling, exception building) Define Jenkins credential with private key and passphrase of user "hasphrase" Modify Jenkins job to use that newly defined credential Confirm job can read the repo (no exception polling, linux and freebsd slaves succeed, windows slaves fail) Steps I took to confirm it failed on a freshly constructed Docker instance (using git client plugin 1.19.0 and git plugin 2.4.0): Run my [master-with-plugins Docker instance] () Define job attempting to use that repo Confirm job cannot read the repo (no credential defined) Define new credential with the passphrase protected private key Exception reported while defining the credential in the Docker instance: Caused by: java.lang.NullPointerException at com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey.getPrivateKeys(BasicSSHUserPrivateKey.java:126) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.createSshKeyFile(CliGitAPIImpl.java:1432) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1300) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1282) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1273) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:2404) at hudson.plugins.git.UserRemoteConfig$DescriptorImpl.doCheckUrl(UserRemoteConfig.java:156) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:121) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) ... 56 more Additional attempts included checking that the Docker instance could use a private key without passphrase (which it could), and that the prompt for the host key fingerprint was not the root reason why the job failed (it did not seem to be the root reason, since even after the private key without passphrase worked, the private key with passphrase was still prompting for the password). When I copied the job which uses the passphrase protected key, the polling log reported there was no existing build so it scheduled a build without polling. When that job ran, it hung with a prompt in the Docker window requesting a passphrase for a temporary key file. Enter passphrase for key '/tmp/ssh5560754303468136093key' : Future experiments might include: Define the DISPLAY environment variable for Jenkins system-wide Try unreleased git client plugin on Docker instance Review createSshKeyFile null pointer exception Implement passphrase protected private key test in CredentialsTest
            Hide
            dejayc Dejay Clayton added a comment -

            Mark,

            Sorry for the extra work. I was following your lead, since you've duplicated your previous comment across all three open tickets. Might it make sense to close two of those tickets as duplicates?

            Now I'm very confused. I've done extensive testing on one of my Jenkins instances, and the other instance I've left completely alone. Both Jenkins instances now work when connecting to GitHub with password-protected private keys, whereas yesterday, neither did. One instance is running git-client 1.17, and the other is running 1.19.

            GitHub support staff proclaims that they've made no changes on their end; thus, I have no explanation for the behavior I've seen yesterday. However, now that I've followed your troubleshooting steps to verify that password-protected keys work fine with both local server paths as well as SSH paths to remote hosts, I'll definitely run those steps again if I ever see connectivity problems regarding GitHub specifically.

            Show
            dejayc Dejay Clayton added a comment - Mark, Sorry for the extra work. I was following your lead, since you've duplicated your previous comment across all three open tickets. Might it make sense to close two of those tickets as duplicates? Now I'm very confused. I've done extensive testing on one of my Jenkins instances, and the other instance I've left completely alone. Both Jenkins instances now work when connecting to GitHub with password-protected private keys, whereas yesterday, neither did. One instance is running git-client 1.17, and the other is running 1.19. GitHub support staff proclaims that they've made no changes on their end; thus, I have no explanation for the behavior I've seen yesterday. However, now that I've followed your troubleshooting steps to verify that password-protected keys work fine with both local server paths as well as SSH paths to remote hosts, I'll definitely run those steps again if I ever see connectivity problems regarding GitHub specifically.
            Hide
            markewaite Mark Waite added a comment -

            Dejay Clayton Thanks for reporting your further results. I'm glad it started working for you. I suspect there may be some ordering dependencies which are made visible on the first connection between a slave and the ssh process of the server repository. At least that's why I think my Docker instance fails and my main Ubuntu machine succeeds. My main Ubuntu machine has already communicated with the ssh daemon on each of my internal slaves that I use for tests, while the Docker instance is seeing the ssh connection for the very first time.

            I won't be able to investigate further this weekend, but hope to continue studying this further in the future.

            Show
            markewaite Mark Waite added a comment - Dejay Clayton Thanks for reporting your further results. I'm glad it started working for you. I suspect there may be some ordering dependencies which are made visible on the first connection between a slave and the ssh process of the server repository. At least that's why I think my Docker instance fails and my main Ubuntu machine succeeds. My main Ubuntu machine has already communicated with the ssh daemon on each of my internal slaves that I use for tests, while the Docker instance is seeing the ssh connection for the very first time. I won't be able to investigate further this weekend, but hope to continue studying this further in the future.
            Hide
            timesking ZeHong Yin added a comment -

            I have the same issue only when job run on slave, the Swarm plugin will show "Enter passphrase for key '/tmp/ssh6736030097350101131key'".

            Jenkins : 1.6.32, git-client-plugin 1.19

            Show
            timesking ZeHong Yin added a comment - I have the same issue only when job run on slave, the Swarm plugin will show "Enter passphrase for key '/tmp/ssh6736030097350101131key'". Jenkins : 1.6.32, git-client-plugin 1.19
            Hide
            mbidewel Mark Bidewell added a comment -

            If I have an SSH key with a passphrase, I get a request for the passphrase using the Git or SSH-Agent plugin on a slave. JGit works fine and the key works fine on the master.

            Show
            mbidewel Mark Bidewell added a comment - If I have an SSH key with a passphrase, I get a request for the passphrase using the Git or SSH-Agent plugin on a slave. JGit works fine and the key works fine on the master.
            Hide
            sherpapsy John Grasett added a comment -

            I was having a heck of a time using a passphrase protected key with git in Jenkins. I had an exact mirror of our production server on a VM and it worked fine, on the production box I could ssh and use git from the command line with the key and passphrase, but in Jenkins job config, I was getting output that was the same as if I had entered incorrect passphrase on the command line.

            The only difference between two systems was that the passphrase on the production system had a $ in it. No other symbols, just characters and numbers and a $.

            I created a new key with no symbols in the passphrase, and it works fine.

            There appears to be a bug (or limitation) in Jenkins or the SSH/Git plugins where a passphrase containing a $ does not work.

            I'm wondering if I should open this as a bug, not sure its affecting the person having problems here?

            Cheers!

            Show
            sherpapsy John Grasett added a comment - I was having a heck of a time using a passphrase protected key with git in Jenkins. I had an exact mirror of our production server on a VM and it worked fine, on the production box I could ssh and use git from the command line with the key and passphrase, but in Jenkins job config, I was getting output that was the same as if I had entered incorrect passphrase on the command line. The only difference between two systems was that the passphrase on the production system had a $ in it. No other symbols, just characters and numbers and a $. I created a new key with no symbols in the passphrase, and it works fine. There appears to be a bug (or limitation) in Jenkins or the SSH/Git plugins where a passphrase containing a $ does not work. I'm wondering if I should open this as a bug, not sure its affecting the person having problems here? Cheers!
            Hide
            nfalco79 Nikolas Falco added a comment -

            Same problem here.
            Jenkins 1.625.3
            SSH Agent Plugin 1.9
            SSH Credentials Plugin 1.11

            I configure a GIT as SCM to our internal git server (based on gitolite). The configured credential use ssh key.
            If the jenkins credential key does have a passphrase authentication work and I could download repo. If I past a ssh key passphrase and configure it then I got an error:

            ERROR: Error fetching remote repo 'origin'
            hudson.plugins.git.GitException: Failed to fetch from ssh://rad@git.example.com/myrepo.git
            	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:763)
            	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1012)
            	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1043)
            	at hudson.scm.SCM.checkout(SCM.java:485)
            	at hudson.model.AbstractProject.checkout(AbstractProject.java:1275)
            	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610)
            	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
            	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532)
            	at hudson.model.Run.execute(Run.java:1741)
            	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
            	at hudson.model.ResourceController.execute(ResourceController.java:98)
            	at hudson.model.Executor.run(Executor.java:408)
            Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress ssh://rad@git.example.com/myrepo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
            stdout: 
            stderr: Permission denied (publickey).
            fatal: The remote end hung up unexpectedly
            	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1710)
            	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1454)
            	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:63)
            	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:314)
            	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:761)
            
            Show
            nfalco79 Nikolas Falco added a comment - Same problem here. Jenkins 1.625.3 SSH Agent Plugin 1.9 SSH Credentials Plugin 1.11 I configure a GIT as SCM to our internal git server (based on gitolite). The configured credential use ssh key. If the jenkins credential key does have a passphrase authentication work and I could download repo. If I past a ssh key passphrase and configure it then I got an error: ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from ssh: //rad@git.example.com/myrepo.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:763) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1012) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1043) at hudson.scm.SCM.checkout(SCM.java:485) at hudson.model.AbstractProject.checkout(AbstractProject.java:1275) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532) at hudson.model.Run.execute(Run.java:1741) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:408) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress ssh: //rad@git.example.com/myrepo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: Permission denied (publickey). fatal: The remote end hung up unexpectedly at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1710) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1454) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:63) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:314) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:761)
            Hide
            markewaite Mark Waite added a comment - - edited

            My attempts to create a case which will use a passphrase protected credential have consistently failed with command line git. I still don't know how others are able to use passphrase protected private keys with command line git, but I am not. I add the private key as a credential, I enter the passphrase into the credential, and it does not authenticate. I even attempted installing the ssh-agent plugin and then tried enabling it for one job. Still no luck.

            JENKINS-32834 includes a report from one user that a change made in git client plugin 1.19.3 broke passphrase handling for him (BatchMode=yes was added to the ssh arguments). I reverted the problem change and released it as git client plugin 1.19.4. There definitely are users who are able to use passphrase protected rsa private keys. I'm not one of them.

            I was able to create a job which uses a password protected private key so long as I use JGit as the git implementation. The same job definition fails when I use command line git.

            Show
            markewaite Mark Waite added a comment - - edited My attempts to create a case which will use a passphrase protected credential have consistently failed with command line git. I still don't know how others are able to use passphrase protected private keys with command line git, but I am not. I add the private key as a credential, I enter the passphrase into the credential, and it does not authenticate. I even attempted installing the ssh-agent plugin and then tried enabling it for one job. Still no luck. JENKINS-32834 includes a report from one user that a change made in git client plugin 1.19.3 broke passphrase handling for him (BatchMode=yes was added to the ssh arguments). I reverted the problem change and released it as git client plugin 1.19.4. There definitely are users who are able to use passphrase protected rsa private keys. I'm not one of them. I was able to create a job which uses a password protected private key so long as I use JGit as the git implementation. The same job definition fails when I use command line git.
            agallardo Alberto Gallardo made changes -
            Hide
            agallardo Alberto Gallardo added a comment - - edited

            I'm the reporter of JENKINS-32834. I'm using a file on Jenkins as private key:

            Note: Credentials work independently of configured username.

            Environment:
            Jenkins ver. 1647
            on jetty-winstone-2.9
            Java oracle jdk1.7.0_79
            Git: 2.4.2
            Git Client: 1.19.4
            ssh-credentials: 1.11

            Show
            agallardo Alberto Gallardo added a comment - - edited I'm the reporter of JENKINS-32834 . I'm using a file on Jenkins as private key: Note: Credentials work independently of configured username. Environment: Jenkins ver. 1647 on jetty-winstone-2.9 Java oracle jdk1.7.0_79 Git: 2.4.2 Git Client: 1.19.4 ssh-credentials: 1.11
            Hide
            agallardo Alberto Gallardo added a comment -

            I still don't know how others are able to use passphrase protected private keys with command line git, but I am not. I add the private key as a credential, I enter the passphrase into the credential, and it does not authenticate.

            Mark Waite

            What do you mean with "command line git"? For each of the GitHub repositories I have configured, I need a different ssh key. I have followed the GitHub instructions to generate the keys, with a subtle difference: as I need different keys, I create different files:

            # (In jenkins)
            $sudo su -s /bin/bash - jenkins
            $ssh-keygen -t rsa -b 4096 -C "tests.jenkins@ci" -f ~/.ssh/jenkins.tests.id_rsa
            # Should ask for a password
            

            I test the connection:

            $ssh-agent bash -c 'ssh-add ~/.ssh/jenkins.tests.id_rsa; git -c core.askpass=true ls-remote ssh://git@github..../.../tests.git'
            # Should ask for the password
            

            This should list the remotes:

            09631f7054adb54221cb3a5736c8605a36c0ad79	HEAD
            174f469223c8787fcb161f34c5219d0f3df945fc	refs/heads/patch-1
            09631f7054adb54221cb3a5736c8605a36c0ad79	refs/heads/test1
            174f469223c8787fcb161f34c5219d0f3df945fc	refs/pull/2/head
            57d3cf7fc2a2908fe1907372921a71110fb0de38	refs/pull/2/merge
            

            Afterwards, it's only adding new credentials as shown in previous comment.

            Show
            agallardo Alberto Gallardo added a comment - I still don't know how others are able to use passphrase protected private keys with command line git, but I am not. I add the private key as a credential, I enter the passphrase into the credential, and it does not authenticate. Mark Waite What do you mean with " command line git "? For each of the GitHub repositories I have configured, I need a different ssh key. I have followed the GitHub instructions to generate the keys, with a subtle difference: as I need different keys, I create different files: # (In jenkins) $sudo su -s /bin/bash - jenkins $ssh-keygen -t rsa -b 4096 -C "tests.jenkins@ci" -f ~/.ssh/jenkins.tests.id_rsa # Should ask for a password I test the connection: $ssh-agent bash -c 'ssh-add ~/.ssh/jenkins.tests.id_rsa; git -c core.askpass=true ls-remote ssh://git@github..../.../tests.git' # Should ask for the password This should list the remotes: 09631f7054adb54221cb3a5736c8605a36c0ad79 HEAD 174f469223c8787fcb161f34c5219d0f3df945fc refs/heads/patch-1 09631f7054adb54221cb3a5736c8605a36c0ad79 refs/heads/test1 174f469223c8787fcb161f34c5219d0f3df945fc refs/pull/2/head 57d3cf7fc2a2908fe1907372921a71110fb0de38 refs/pull/2/merge Afterwards, it's only adding new credentials as shown in previous comment.
            Hide
            markewaite Mark Waite added a comment -

            What do you mean with "command line git"?

            I'm able to configure and successfully run a project which uses an RSA private key which requires a passphrase, if I use the JGIt implementation. I have not been able to successfully run a project using an RSA private key which requires a passphrase if I use the command line git (default) implementation.

            Show
            markewaite Mark Waite added a comment - What do you mean with "command line git"? I'm able to configure and successfully run a project which uses an RSA private key which requires a passphrase, if I use the JGIt implementation. I have not been able to successfully run a project using an RSA private key which requires a passphrase if I use the command line git (default) implementation.
            Hide
            agallardo Alberto Gallardo added a comment - - edited

            I have to admit that I wasn't familiarized with all these plugins until yesterday. After taking a look at the doc for the Git Plugin I understand your point.

            First of all, my jenkins runs a git 1.7.12.4. Now, I have added some logs and dug a bit more into the git-client guts, and am even more clueless than before: it looks as if jenkins

            1. set GIT_SSH=/tmp/ssh12345678901234567890.sh and SSH_ASKPASS=/tmp/pass12345678901234567890.sh to respective tmp files:

            /tmp/ssh12345678901234567890.sh
            #!/bin/sh
            if [ -z "${DISPLAY}" ]; then
              DISPLAY=:123.456
              export DISPLAY
            fi
            ssh -i "/tmp/ssh12345678901234567890key" -l "jenkins" -o StrictHostKeyChecking=no "$@"
            
            /tmp/pass12345678901234567890
            #!/bin/sh
            /bin/echo "mySecretPassword"
            

            2. and then executed

            git -c core.askpass=true ls-remote -h git@github.mygithubserver.org:USER/tests.git HEAD # timeout=10
            

            3. what results in git invoking

            ssh -i /var/lib/jenkins/.ssh/tests.id_rsa -o StrictHostKeyChecking=no git@github.mygithubserver.org 'git-upload-pack '\''/USER/tests.git'\'''
            

            And this, when invoked from the command line, prompts for the test.id_rsa password, because ssh (and not git) wants to use the key. So the question is: how could this even work if the SSH_ASKPASS is never used?

            I don't know why, but my assumption about the password promt is wrong: jenkins manages somehow to invoke git -c core.askpass=..., and this in turn successfully invokes both the ssh and the pass scripts. But I cannot get this working in a terminal .

            Show
            agallardo Alberto Gallardo added a comment - - edited I have to admit that I wasn't familiarized with all these plugins until yesterday. After taking a look at the doc for the Git Plugin I understand your point. First of all, my jenkins runs a git 1.7.12.4. Now, I have added some logs and dug a bit more into the git-client guts, and am even more clueless than before: it looks as if jenkins 1. set GIT_SSH=/tmp/ssh12345678901234567890.sh and SSH_ASKPASS=/tmp/pass12345678901234567890.sh to respective tmp files: /tmp/ssh12345678901234567890.sh #!/bin/sh if [ -z "${DISPLAY}" ]; then DISPLAY=:123.456 export DISPLAY fi ssh -i "/tmp/ssh12345678901234567890key" -l "jenkins" -o StrictHostKeyChecking=no "$@" /tmp/pass12345678901234567890 #!/bin/sh /bin/echo "mySecretPassword" 2. and then executed git -c core.askpass=true ls-remote -h git@github.mygithubserver.org:USER/tests.git HEAD # timeout=10 3. what results in git invoking ssh -i /var/lib/jenkins/.ssh/tests.id_rsa -o StrictHostKeyChecking=no git@github.mygithubserver.org 'git-upload-pack '\''/USER/tests.git'\''' And this, when invoked from the command line, prompts for the test.id_rsa password, because ssh (and not git ) wants to use the key. So the question is: how could this even work if the SSH_ASKPASS is never used? I don't know why, but my assumption about the password promt is wrong: jenkins manages somehow to invoke git -c core.askpass=... , and this in turn successfully invokes both the ssh and the pass scripts. But I cannot get this working in a terminal .
            Hide
            agallardo Alberto Gallardo added a comment -

            By the way, according to the git-scm book, SSH_ASKPASS is overriden by core.askpass=true.

            Show
            agallardo Alberto Gallardo added a comment - By the way, according to the git-scm book , SSH_ASKPASS is overriden by core.askpass=true .
            Hide
            agallardo Alberto Gallardo added a comment -

            After a few hours I could make it work from the console. A post in stackexchange pointed me in the right direction: my fault was to not realize that jenkins executes the commands without attached terminal.

            If ssh needs a passphrase, it will read the passphrase from the current terminal if it was run from a terminal. If ssh does not have a terminal associated with it but DISPLAY and SSH_ASKPASS are set, it will execute the program specified by SSH_ASKPASS

            @Mark Waite I could reproduce jenkins behaviour from the linux console adding setsid to the ssh invokation in GIT_SSH:

            /tmp/ssh12345678901234567890.sh
            #!/bin/sh
            if [ -z "${DISPLAY}" ]; then
              DISPLAY=:123.456
              export DISPLAY
            fi
            # setsid: create a new session dettached from the console
            setsid ssh -i "/tmp/ssh12345678901234567890key" -l "jenkins" -o StrictHostKeyChecking=no "$@"
            

            I can also confirm that the -c core.askpass=true git config is unnecessary.

            Show
            agallardo Alberto Gallardo added a comment - After a few hours I could make it work from the console. A post in stackexchange pointed me in the right direction: my fault was to not realize that jenkins executes the commands without attached terminal. If ssh needs a passphrase, it will read the passphrase from the current terminal if it was run from a terminal. If ssh does not have a terminal associated with it but DISPLAY and SSH_ASKPASS are set, it will execute the program specified by SSH_ASKPASS @ Mark Waite I could reproduce jenkins behaviour from the linux console adding setsid to the ssh invokation in GIT_SSH : /tmp/ssh12345678901234567890.sh #!/bin/sh if [ -z "${DISPLAY}" ]; then DISPLAY=:123.456 export DISPLAY fi # setsid: create a new session dettached from the console setsid ssh -i "/tmp/ssh12345678901234567890key" -l "jenkins" -o StrictHostKeyChecking=no "$@" I can also confirm that the -c core.askpass=true git config is unnecessary.
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 152690 ] JNJira + In-Review [ 186170 ]
            Hide
            harel_e Harel E. added a comment -

            I had similar issue and ssh credentials. After several passphrase tests it boiled down to the $ sign.
            passphrase that contained the $ sign didn't work.

            Hope that helps.

            Harel

            Show
            harel_e Harel E. added a comment - I had similar issue and ssh credentials. After several passphrase tests it boiled down to the $ sign. passphrase that contained the $ sign didn't work. Hope that helps. Harel
            Hide
            markewaite Mark Waite added a comment -

            Harel E. thanks for the report. It is quite valuable to know which passphrase characters have shown problems for users. I've included a passphrase with a $ sign in my test data creation script. I'll include that in the verification of any changes to the passphrase support in the git plugin and the git client plugin.

            Show
            markewaite Mark Waite added a comment - Harel E. thanks for the report. It is quite valuable to know which passphrase characters have shown problems for users. I've included a passphrase with a $ sign in my test data creation script. I'll include that in the verification of any changes to the passphrase support in the git plugin and the git client plugin.
            Hide
            mkonecny Michael Konečný added a comment -

            For me, the passphrase only contained lowercase alphabetical characters, but still didn't work.

            Show
            mkonecny Michael Konečný added a comment - For me, the passphrase only contained lowercase alphabetical characters, but still didn't work.
            Hide
            stephenconnolly Stephen Connolly added a comment -

            Mark Waite

            Have you made any progress on not sending Credentials instances over to remote agents

            https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/GitSCM.java#L736

            Show
            stephenconnolly Stephen Connolly added a comment - Mark Waite Have you made any progress on not sending Credentials instances over to remote agents https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/GitSCM.java#L736
            Hide
            markewaite Mark Waite added a comment -

            Stephen Connolly I have not made any progress on JENKINS-37899 or not sending credentials instances to remote agents. I'm unlikely to make any short term progress on it because I want to resolve the authentication failures in git plugin 3.0 (JENKINS-38138, JENKINS-38179, JENKINS-38194) and the submodule failures in git plugin 3.0 (JENKINS-37495).

            Show
            markewaite Mark Waite added a comment - Stephen Connolly I have not made any progress on JENKINS-37899 or not sending credentials instances to remote agents. I'm unlikely to make any short term progress on it because I want to resolve the authentication failures in git plugin 3.0 ( JENKINS-38138 , JENKINS-38179 , JENKINS-38194 ) and the submodule failures in git plugin 3.0 ( JENKINS-37495 ).
            Hide
            megaserg Sergey Serebryakov added a comment -

            Tip: if running Jenkins inside of Docker container, don't provide -it flags, otherwise Jenkins will think it's run from a terminal and start asking for passphrase instead of using the one provided in Credentials settings.

            Show
            megaserg Sergey Serebryakov added a comment - Tip: if running Jenkins inside of Docker container, don't provide -it  flags, otherwise Jenkins will think it's run from a terminal and start asking for passphrase instead of using the one provided in Credentials settings.
            Hide
            markewaite Mark Waite added a comment -

            That's a good suggestion, though I really intend that the plugin will never prompt for a passphrase, where run with a controlling terminal or not.

            Show
            markewaite Mark Waite added a comment - That's a good suggestion, though I really intend that the plugin will never prompt for a passphrase, where run with a controlling terminal or not.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Mark Waite
            Path:
            pom.xml
            src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
            http://jenkins-ci.org/commit/git-client-plugin/882ecdfb53d627eeeea130720685681ce2553193
            Log:
            Optionally detach ssh authenticated git calls from terminal

            When I run CredentialsTest from my terminal window on Ubuntu 16.04, the
            test fails for ssh keys which need a passphrase. If I run the tests
            from my IDE, or if I prepend "setsid" to the maven command that runs
            the tests, the tests pass.

            The ssh command called by git seems to require the DISPLAY variable, and
            the GIT_SSH variable, and must be detached from the controlling terminal.
            If any one of those is missing (at least on Ubuntu 16), the passphrase
            prompt will not be answered.

            Command line maven builds now include the property:

            org.jenkinsci.plugins.gitclient.CliGitAPIImpl.useSETSID=true

            Default runtime value of the property is false. Users will not run
            with this change unless they specifically set that property to true.
            Most users don't run Jenkins with a controlling terminal attached,
            so they don't need the change.

            Sets useSETSID=true in surefire target so that command line invocations
            of the tests will prefix the "git" command with setsid when it is used
            in an ssh private key context.

            Allows command line run of CredentialsTest with passphrase protected
            private keys.

            Setting BatchMode=yes in the ssh command does not have the same result.
            The setsid call was the only technique I found that reliably allowed
            the ssh call performed by command line git to consistently process the
            script defined in the SSH_ASKPASS variable.

            See JENKINS-20879 and JENKINS-25194 for more details.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Mark Waite Path: pom.xml src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java http://jenkins-ci.org/commit/git-client-plugin/882ecdfb53d627eeeea130720685681ce2553193 Log: Optionally detach ssh authenticated git calls from terminal When I run CredentialsTest from my terminal window on Ubuntu 16.04, the test fails for ssh keys which need a passphrase. If I run the tests from my IDE, or if I prepend "setsid" to the maven command that runs the tests, the tests pass. The ssh command called by git seems to require the DISPLAY variable, and the GIT_SSH variable, and must be detached from the controlling terminal. If any one of those is missing (at least on Ubuntu 16), the passphrase prompt will not be answered. Command line maven builds now include the property: org.jenkinsci.plugins.gitclient.CliGitAPIImpl.useSETSID=true Default runtime value of the property is false. Users will not run with this change unless they specifically set that property to true. Most users don't run Jenkins with a controlling terminal attached, so they don't need the change. Sets useSETSID=true in surefire target so that command line invocations of the tests will prefix the "git" command with setsid when it is used in an ssh private key context. Allows command line run of CredentialsTest with passphrase protected private keys. Setting BatchMode=yes in the ssh command does not have the same result. The setsid call was the only technique I found that reliably allowed the ssh call performed by command line git to consistently process the script defined in the SSH_ASKPASS variable. See JENKINS-20879 and JENKINS-25194 for more details.
            markewaite Mark Waite made changes -
            Status Reopened [ 4 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            jglick Jesse Glick made changes -
            Link This issue relates to SECURITY-445 [ SECURITY-445 ]
            Hide
            markewaite Mark Waite added a comment - - edited

            Git client plugin 2.5.0 (released 27 Jul 2017) added support for setsid in the /usr/local directory tree, since that seems to be where it is stored on MacOS.

            Show
            markewaite Mark Waite added a comment - - edited Git client plugin 2.5.0 (released 27 Jul 2017) added support for setsid in the /usr/local directory tree, since that seems to be where it is stored on MacOS.
            Hide
            allan_burdajewicz Allan BURDAJEWICZ added a comment -

            Mark Waite I think you meant git-client 2.5.0 ?

            Show
            allan_burdajewicz Allan BURDAJEWICZ added a comment - Mark Waite I think you meant git-client 2.5.0 ?
            Hide
            markewaite Mark Waite added a comment -

            You're correct Allan BURDAJEWICZ, I should have said "git client plugin". The git plugin release 2.5.0 was already a year old by the time git client plugin 2.5.0 was released.

            Show
            markewaite Mark Waite added a comment - You're correct Allan BURDAJEWICZ , I should have said "git client plugin". The git plugin release 2.5.0 was already a year old by the time git client plugin 2.5.0 was released.
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal CD-150 (Web Link)" [ 19130 ]
            markewaite Mark Waite made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            markewaite Mark Waite made changes -
            Link This issue is related to JENKINS-53134 [ JENKINS-53134 ]
            markewaite Mark Waite made changes -
            Link This issue is duplicated by JENKINS-53134 [ JENKINS-53134 ]
            markewaite Mark Waite made changes -
            Link This issue is related to JENKINS-53134 [ JENKINS-53134 ]

              People

              • Assignee:
                Unassigned
                Reporter:
                nshenry03 Nick Henry
              • Votes:
                20 Vote for this issue
                Watchers:
                35 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: