Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52354

Git plugin unable to use SSH credentials with passphrase

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Not A Defect
    • Component/s: git-client-plugin
    • Labels:
      None
    • Environment:
      SSH Agent Plugin 1.15
      SSH Credentials Plugin 1.14
      Git plugin 3.9.1
      Jenkins ver. 2.121.1
      ssh-keygen from openssh 7.7p1-1
    • Similar Issues:

      Description

      Hello.

      If ssh credentials used with a passphrase, then Jenkins unable to fetch anything from the remote repo.

      Step to reproduce:

      1. Create a key with passphrase `ssh-keygen -b 2048 -t rsa -f test_rsa`
      2. Add credentials 'SSH Username with private key' from the previous step
      3. Add repo on git with access by the key
      4. Create freestyle project with ssh git source and using credentials
        Result:
      5. Remove passphrase from the key with `ssh-keygen -pf test_rsa`
      6. Edit credentials regarding
      7. Try to fetch the source code again
        Result:
         

      I would like to give any necessary additional information.

        Attachments

          Activity

          Hide
          markewaite Mark Waite added a comment -

          There is a known problem with special character handling in passphrases due to the way the git plugin provides the passphrase to command line git. Does the chosen passphrase contain any characters which are interpreted as 'special' by the shell? If you remove the special characters from the passphrase, does it then work?

          You referenced ssh-keygen from OpenSSH 7.7p1.1. That is not an OpenSSH version string I recognize from the places where I know that OpenSSH 7.7 is deployed (Windows with command line git 2.17.0 and later, Debian Testing distribution "Buster", and OpenBSD 6.3 and later). What operating system are you running?

          What version of command line git are you running?

          Can you explain in more detail what you mean in step 6 when you say "Edit credentials regarding"? Does that mean you push the updated public key to the gitlab server, or is there something else involved in that step?

          What operating system and version is hosting the gitlab server?

          What gitlab version is running on your gitlab server?

          What version of the Jenkins git client plugin are you running? I assume git client plugin 2.7.2 (the most recent release)

          Show
          markewaite Mark Waite added a comment - There is a known problem with special character handling in passphrases due to the way the git plugin provides the passphrase to command line git. Does the chosen passphrase contain any characters which are interpreted as 'special' by the shell? If you remove the special characters from the passphrase, does it then work? You referenced ssh-keygen from OpenSSH 7.7p1.1. That is not an OpenSSH version string I recognize from the places where I know that OpenSSH 7.7 is deployed (Windows with command line git 2.17.0 and later, Debian Testing distribution "Buster", and OpenBSD 6.3 and later). What operating system are you running? What version of command line git are you running? Can you explain in more detail what you mean in step 6 when you say "Edit credentials regarding"? Does that mean you push the updated public key to the gitlab server, or is there something else involved in that step? What operating system and version is hosting the gitlab server? What gitlab version is running on your gitlab server? What version of the Jenkins git client plugin are you running? I assume git client plugin 2.7.2 (the most recent release)
          Hide
          felixoid Mikhail Shiryaev added a comment -
          • Any passphrase causes it. I used '123456'
          • https://www.archlinux.org/packages/core/x86_64/openssh/ is my working station. Jenkins is installed on Debian 8.11. And it is the same result with keys created with openssh-client 1:6.7p1-5+deb8u4
          • git version 2.1.4
          • I removed the passphrase from private key without a changing of the public part. Then I updated the private part in Jenkins Credentials
          • Debian 9.4
          • Gitlab 10.6.5, revision dfc9d0d632
          • That's correct, git plugin is 2.7.2
          Show
          felixoid Mikhail Shiryaev added a comment - Any passphrase causes it. I used '123456' https://www.archlinux.org/packages/core/x86_64/openssh/ is my working station. Jenkins is installed on Debian 8.11. And it is the same result with keys created with openssh-client 1:6.7p1-5+deb8u4 git version 2.1.4 I removed the passphrase from private key without a changing of the public part. Then I updated the private part in Jenkins Credentials Debian 9.4 Gitlab 10.6.5, revision dfc9d0d632 That's correct, git plugin is 2.7.2
          Hide
          markewaite Mark Waite added a comment - - edited

          Thanks for the additional details. I can't duplicate the problem you're seeing. Here are the steps that I tried in attempting to duplicate the problem:

          1. Run a docker image of Arch linux latest to generate a passphrase protected private key (passphrase 123456)
                mark-pc2 $ docker run --rm -i -t docker run --rm -t -i pritunl/archlinux:latest /bin/bash
                02a19dfb0dd1 # pacman -Syu openssh git
                02a19dfb0dd1 # useradd --create-home mwaite
                02a19dfb0dd1 # su - mwaite
                mwaite@02a19dfb0dd1 $ ssh-keygen -b 2048 -t rsa -f test_rsa
                mwaite@02a19dfb0dd1 $ cat test_rsa.pub
                mwaite@02a19dfb0dd1 $ cat test_rsa
            
          2. Create a new ssh key on my gitlab account using test_rsa.pub as the public key
          3. Clone a private gitlab repository to that Arch linux Docker container
                mwaite@02a19dfb0dd1 $ export GIT_SSH_COMMAND='ssh -i test_rsa'
                mwaite@02a19dfb0dd1 $ git clone git@gitlab.com:MarkEWaite/tasks.git
                Enter passphrase for key 'test_rsa': 123456
                mwaite@02a19dfb0dd1 $ ls -altrd tasks
                drwxr-xr-x 3 mwaite mwaite 4096 Jul  4 11:56 tasks
            
          4. Create a Jenkins credential with that private key and passphrase
          5. Create a Freestyle job using that private key to clone that same gitlab repository
          6. Run that Freestyle job on agents running CentOS 6 (git 2.17.0.1181.g093e983b0), CentOS 7 (git 1.8.3.1), Debian 8 (git 2.1.4), Debian 9 (git 2.11.0), Ubuntu 14 (git 1.9.1), Ubuntu 16 (git 2.7.4), and Windows 10 (git 2.16.0)

          Those steps all succeeded.

          The public portions of the Jenkins master I run are described in a Docker image on my lts-with-plugins branch. That includes all the plugins I'm using and the versions of those plugins.

          At the time I ran my tests, gitlab.com was running "GitLab Enterprise Edition 11.1.0-rc2-ee".

          One of the comments in the openssl issue 47 hints that very old versions of ssh may be unable to read the newest versions of the configuration file. It says:

          However some old ssh clients indeed assume the legacy pkcs1 format for id_rsa keys

          You might check the ssh version on the agent that is performing the clone.

          Show
          markewaite Mark Waite added a comment - - edited Thanks for the additional details. I can't duplicate the problem you're seeing. Here are the steps that I tried in attempting to duplicate the problem: Run a docker image of Arch linux latest to generate a passphrase protected private key (passphrase 123456) mark-pc2 $ docker run --rm -i -t docker run --rm -t -i pritunl/archlinux:latest /bin/bash 02a19dfb0dd1 # pacman -Syu openssh git 02a19dfb0dd1 # useradd --create-home mwaite 02a19dfb0dd1 # su - mwaite mwaite@02a19dfb0dd1 $ ssh-keygen -b 2048 -t rsa -f test_rsa mwaite@02a19dfb0dd1 $ cat test_rsa.pub mwaite@02a19dfb0dd1 $ cat test_rsa Create a new ssh key on my gitlab account using test_rsa.pub as the public key Clone a private gitlab repository to that Arch linux Docker container mwaite@02a19dfb0dd1 $ export GIT_SSH_COMMAND='ssh -i test_rsa' mwaite@02a19dfb0dd1 $ git clone git@gitlab.com:MarkEWaite/tasks.git Enter passphrase for key 'test_rsa': 123456 mwaite@02a19dfb0dd1 $ ls -altrd tasks drwxr-xr-x 3 mwaite mwaite 4096 Jul 4 11:56 tasks Create a Jenkins credential with that private key and passphrase Create a Freestyle job using that private key to clone that same gitlab repository Run that Freestyle job on agents running CentOS 6 (git 2.17.0.1181.g093e983b0), CentOS 7 (git 1.8.3.1), Debian 8 (git 2.1.4), Debian 9 (git 2.11.0), Ubuntu 14 (git 1.9.1), Ubuntu 16 (git 2.7.4), and Windows 10 (git 2.16.0) Those steps all succeeded. The public portions of the Jenkins master I run are described in a Docker image on my lts-with-plugins branch . That includes all the plugins I'm using and the versions of those plugins. At the time I ran my tests, gitlab.com was running "GitLab Enterprise Edition 11.1.0-rc2-ee". One of the comments in the openssl issue 47 hints that very old versions of ssh may be unable to read the newest versions of the configuration file. It says: However some old ssh clients indeed assume the legacy pkcs1 format for id_rsa keys You might check the ssh version on the agent that is performing the clone.
          Hide
          felixoid Mikhail Shiryaev added a comment - - edited

          Hi. Thank you for the answer.

           

          I created backup of jenkins_home and restored it on fresh docker jenkins/jenkins:2.121.1 and it works there. I guess, it's indeed some ssl problem. It doesn't work, actually, also with gitlab.com from our production jenkins.

          Show
          felixoid Mikhail Shiryaev added a comment - - edited Hi. Thank you for the answer.   I created backup of jenkins_home and restored it on fresh docker jenkins/jenkins:2.121.1 and it works there. I guess, it's indeed some ssl problem. It doesn't work, actually, also with gitlab.com from our production jenkins.

            People

            • Assignee:
              Unassigned
              Reporter:
              felixoid Mikhail Shiryaev
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: