Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35567

multibranch project and ssh+git broken

    Details

    • Similar Issues:

      Description

      Hi,

      I try to create a pipeline project, but get always these errors:

      1st try: using "real" hostname with

      same result as JENKINS-35565 (connection refused)

      2nd try: using localhost/127.0.0.1 (I tried both, same result)

      first with "Credentials" set to "empty"

      {{Started
      Creating git repository in /var/lib/jenkins/caches/git-b8e0a72444a3fca4da8b9cbf35c0bac2
      Setting origin to ssh://git@localhost:2222/repo.git
      Fetching origin...
      FATAL: Failed to recompute children of MyTestProject
      java.lang.IllegalStateException: Cannot open session, connection is not authenticated.
      at com.trilead.ssh2.Connection.openSession(Connection.java:1127)
      at org.jenkinsci.plugins.gitclient.trilead.TrileadSession.exec(TrileadSession.java:32)
      at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:262)
      at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:161)
      at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:136)
      at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122)
      at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1138)
      at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:130)
      at org.jenkinsci.plugins.gitclient.JGitAPIImpl.fetch(JGitAPIImpl.java:678)
      at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:174)
      at jenkins.scm.api.SCMSource.fetch(SCMSource.java:146)
      at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:294)
      at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:157)
      at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:122)
      at hudson.model.ResourceController.execute(ResourceController.java:98)
      at hudson.model.Executor.run(Executor.java:410)
      Finished: FAILURE

      theres simply no connection attempt inside /var/log/auth.log

      It seems that it fully ignores ~/.ssh/config (int the correct home of the jenkins user), since it does not use the private key specified there.

      If I specify a private key inside the "credentials" (i.e use private key from jenkins-master home), same error occurs.

      it seems the branch indexing simply ignores every credential provider or standard ssh/git behaviour.

        Attachments

          Issue Links

            Activity

            flow86 Florian Doersch created issue -
            markewaite Mark Waite made changes -
            Field Original Value New Value
            Attachment repo-def.png [ 32977 ]
            Hide
            markewaite Mark Waite added a comment - - edited

            I can't duplicate your problem. Can you provide more details of the specific steps you took when you saw the problem?

            Steps I took which confirmed that I can create a multi-branch pipeline project which uses an RSA private key to access a private remote repository:

            1. Create a new "Multibranch Pipeline" job named "JENKINS-35567-multi-branch-pipeline-with-rsa-private-key"
            2. Select "Git" as the source control for that job
            3. Assign "git@github.com:MarkEWaite/docker-private.git" as the repo URL
            4. Assign a valid RSA private key to that repo URL from the drop-down list
            5. Leave the default Build Configuration Mode - "By Jenkinsfile"

            When I did that, the docker-private repo was indexed and the branches on that repo with a Jenkinsfile were all created. I successfully ran that test from my Ubuntu 14.04 computer and from a Docker instance.

            Refer to

            Show
            markewaite Mark Waite added a comment - - edited I can't duplicate your problem. Can you provide more details of the specific steps you took when you saw the problem? Steps I took which confirmed that I can create a multi-branch pipeline project which uses an RSA private key to access a private remote repository: Create a new "Multibranch Pipeline" job named " JENKINS-35567 -multi-branch-pipeline-with-rsa-private-key" Select "Git" as the source control for that job Assign "git@github.com:MarkEWaite/docker-private.git" as the repo URL Assign a valid RSA private key to that repo URL from the drop-down list Leave the default Build Configuration Mode - "By Jenkinsfile" When I did that, the docker-private repo was indexed and the branches on that repo with a Jenkinsfile were all created. I successfully ran that test from my Ubuntu 14.04 computer and from a Docker instance. Refer to
            stephenconnolly Stephen Connolly made changes -
            Component/s ssh-credentials-plugin [ 17424 ]
            Hide
            stephenconnolly Stephen Connolly added a comment -

            Does not look to be an issue with the ssh-credentials plugin

            Show
            stephenconnolly Stephen Connolly added a comment - Does not look to be an issue with the ssh-credentials plugin
            stephenconnolly Stephen Connolly made changes -
            Component/s branch-api-plugin [ 18621 ]
            stephenconnolly Stephen Connolly made changes -
            Assignee stephenconnolly [ stephenconnolly ] Mark Waite [ markewaite ]
            markewaite Mark Waite made changes -
            Assignee Mark Waite [ markewaite ]
            Hide
            markewaite Mark Waite added a comment -

            Florian Doersch you might check that you have JGit enabled as a valid git implementation in your Jenkins system configuration. The stack trace seems to hint that the code is trying to use JGit, and if it is not enabled, that might cause some surprises.

            Show
            markewaite Mark Waite added a comment - Florian Doersch you might check that you have JGit enabled as a valid git implementation in your Jenkins system configuration. The stack trace seems to hint that the code is trying to use JGit, and if it is not enabled, that might cause some surprises.
            flow86 Florian Doersch made changes -
            Attachment jenkins-jgit.PNG [ 33049 ]
            Hide
            flow86 Florian Doersch added a comment -

            Hi,

            JGit is enabled in the tools:

            Okay I tried again:

            1. Create a new "Multibranch Pipeline" job "myjob"
            2. added source "Git" with url git@gitlab.mydomain.de:2222/repo.git
            3. added credentials "ssh username with key", user "git", private key "From the Jenkins master ~/.ssh"
            4. selected these credentials
            5. pressed save

            now getting:

            Caused by: java.io.IOException: There was a problem while connecting to gitlab.mydomain.de:22
            at com.trilead.ssh2.Connection.connect(Connection.java:818)
            at com.trilead.ssh2.Connection.connect(Connection.java:687)
            at com.trilead.ssh2.Connection.connect(Connection.java:587)
            at org.jenkinsci.plugins.gitclient.trilead.TrileadSessionFactory.getSession(TrileadSessionFactory.java:29)
            ... 15 more

            why does it ignore the port in the url and also the setting in ~/.ssh/config ???

            7. changed location to "ssh://git@gitlab.mydomain.de:2222/repo.git" ( added ssh:// )

            now getting

            java.lang.IllegalStateException: Cannot open session, connection is not authenticated.
            at com.trilead.ssh2.Connection.openSession(Connection.java:1127)
            at org.jenkinsci.plugins.gitclient.trilead.TrileadSession.exec(TrileadSession.java:32)
            at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:262)
            at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:161)
            at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:136)
            at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122)
            at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1138)
            at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:130)
            at org.jenkinsci.plugins.gitclient.JGitAPIImpl.fetch(JGitAPIImpl.java:678)
            at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:174)
            at jenkins.scm.api.SCMSource.fetch(SCMSource.java:146)
            at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:294)
            at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:157)
            at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:122)
            at hudson.model.ResourceController.execute(ResourceController.java:98)
            at hudson.model.Executor.run(Executor.java:410)
            Finished: FAILURE

            Show
            flow86 Florian Doersch added a comment - Hi, JGit is enabled in the tools: Okay I tried again: 1. Create a new "Multibranch Pipeline" job "myjob" 2. added source "Git" with url git@gitlab.mydomain.de:2222/repo.git 3. added credentials "ssh username with key", user "git", private key "From the Jenkins master ~/.ssh" 4. selected these credentials 5. pressed save now getting: Caused by: java.io.IOException: There was a problem while connecting to gitlab.mydomain.de:22 at com.trilead.ssh2.Connection.connect(Connection.java:818) at com.trilead.ssh2.Connection.connect(Connection.java:687) at com.trilead.ssh2.Connection.connect(Connection.java:587) at org.jenkinsci.plugins.gitclient.trilead.TrileadSessionFactory.getSession(TrileadSessionFactory.java:29) ... 15 more why does it ignore the port in the url and also the setting in ~/.ssh/config ??? 7. changed location to "ssh://git@gitlab.mydomain.de:2222/repo.git" ( added ssh:// ) now getting java.lang.IllegalStateException: Cannot open session, connection is not authenticated. at com.trilead.ssh2.Connection.openSession(Connection.java:1127) at org.jenkinsci.plugins.gitclient.trilead.TrileadSession.exec(TrileadSession.java:32) at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:262) at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:161) at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:136) at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122) at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1138) at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:130) at org.jenkinsci.plugins.gitclient.JGitAPIImpl.fetch(JGitAPIImpl.java:678) at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:174) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:146) at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:294) at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:157) at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:122) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:410) Finished: FAILURE
            Hide
            flow86 Florian Doersch added a comment -

            another info to point "7."

            "Branch Indexing (Jun 18, 2016 7:55:47 AM) "

            /var/log/auth.log:

            Jun 18 07:55:47 gitlab sshd[10127]: Received disconnect from x.x.x.x: 11: Closed due to user request. [preauth]

            Show
            flow86 Florian Doersch added a comment - another info to point "7." "Branch Indexing (Jun 18, 2016 7:55:47 AM) " /var/log/auth.log: Jun 18 07:55:47 gitlab sshd [10127] : Received disconnect from x.x.x.x: 11: Closed due to user request. [preauth]
            Hide
            markewaite Mark Waite added a comment -

            I think JENKINS-26327 may have some useful information.

            Show
            markewaite Mark Waite added a comment - I think JENKINS-26327 may have some useful information.
            Hide
            flow86 Florian Doersch added a comment -

            sorry, but I dont think thats the point here, we dont get far enough to care about the repository-part of the url currently

            1. If I do not specify "ssh://" on the url the port on the url is ignored at all => BUG
            2. If I do specify "ssh://" on the url, no authentication is done ("Closed due to user request. [preauth]"=) => BUG
            3. Master's ~/.ssh/config or even Slave's ~/.ssh/config gets completly ignored, but is not on "Standard Builds" => BUG

            Show
            flow86 Florian Doersch added a comment - sorry, but I dont think thats the point here, we dont get far enough to care about the repository-part of the url currently 1. If I do not specify "ssh://" on the url the port on the url is ignored at all => BUG 2. If I do specify "ssh://" on the url, no authentication is done ("Closed due to user request. [preauth] "=) => BUG 3. Master's ~/.ssh/config or even Slave's ~/.ssh/config gets completly ignored, but is not on "Standard Builds" => BUG
            Hide
            markewaite Mark Waite added a comment - - edited

            I'm not sure I'm understanding your point 1, or at least I see something different from command line git behavior.

            You said

            1. If I do not specify "ssh://" on the url the port on the url is ignored at all => BUG

            If I attempt to use command line git to clone a repository using an ssh url without the ssh://, I cannot include a port number in the URL (or at least, I haven't found a way to include a port number in the URL). I've tried with git 1.7.1 on CentOS 6.7 and with git 2.9.0 on Ubuntu 14.04.

            Embedding port number in git URL like git@hostname:port/directory

            The commands I tried (and results) were:

            Command line git 2.9.0, Ubuntu 14.04

            $ git clone mwaite@wheezy64b.markwaite.net:45022/var/lib/git/mwaite/jenkins/git-client-plugin.git
            Cloning into 'git-client-plugin'...
            fatal: '45022/var/lib/git/mwaite/jenkins/git-client-plugin.git' does not appear to be a git repository
            fatal: Could not read from remote repository.
            
            Please make sure you have the correct access rights
            and the repository exists.
            

            Command line git 1.7.1, CentOS 6.7

            $ git clone mwaite@wheezy64b.markwaite.net:45022/var/lib/git/mwaite/jenkins/git-client-plugin.git
            Initialized empty Git repository in /home/mwaite/tmp/git-client-plugin/.git/
            fatal: '45022/var/lib/git/mwaite/jenkins/git-client-plugin.git' does not appear to be a git repository
            fatal: The remote end hung up unexpectedly
            

            The git clone documentation calls that "scp-line syntax" and seems to provide no way to include a port number in the syntax (other than through an alias in the ~/.ssh/config file).

            Embedding port number in git URL like ssh://git@hostname:port/directory

            If I perform the same clone commands on the same machines, with an ssh:// protocol URL, it works as expected.

            Command line git 2.9.0, Ubuntu 14.04

            $ git clone ssh://mwaite@wheezy64b.markwaite.net:45022/var/lib/git/mwaite/jenkins/git-client-plugin.git
            Cloning into 'git-client-plugin'...
            remote: Counting objects: 11504, done.
            remote: Compressing objects: 100% (4289/4289), done.
            remote: Total 11504 (delta 4180), reused 11499 (delta 4176)
            Receiving objects: 100% (11504/11504), 1.88 MiB | 0 bytes/s, done.
            Resolving deltas: 100% (4180/4180), done.
            Checking connectivity... done.
            

            Command line git 1.7.1, CentOS 6.7

            $ git clone ssh://mwaite@wheezy64b.markwaite.net:45022/var/lib/git/mwaite/jenkins/git-client-plugin.git
            Initialized empty Git repository in /home/mwaite/tmp/git-client-plugin/.git/
            remote: Counting objects: 11504, done.
            remote: Compressing objects: 100% (4289/4289), done.
            remote: Total 11504 (delta 4180), reused 11499 (delta 4176)
            Receiving objects: 100% (11504/11504), 1.88 MiB, done.
            Resolving deltas: 100% (4180/4180), done.
            

            I don't think item 1 in your list is a bug, or if it is a bug, it is a bug in the command line git implementations I checked, not a bug in the git plugin, the git client plugin, or the multibranch plugin.

            Can you help me understand what I've missed on your first point?

            Show
            markewaite Mark Waite added a comment - - edited I'm not sure I'm understanding your point 1, or at least I see something different from command line git behavior. You said 1. If I do not specify "ssh://" on the url the port on the url is ignored at all => BUG If I attempt to use command line git to clone a repository using an ssh url without the ssh://, I cannot include a port number in the URL (or at least, I haven't found a way to include a port number in the URL). I've tried with git 1.7.1 on CentOS 6.7 and with git 2.9.0 on Ubuntu 14.04. Embedding port number in git URL like git@hostname:port/directory The commands I tried (and results) were: Command line git 2.9.0, Ubuntu 14.04 $ git clone mwaite@wheezy64b.markwaite.net:45022/var/lib/git/mwaite/jenkins/git-client-plugin.git Cloning into 'git-client-plugin'... fatal: '45022/var/lib/git/mwaite/jenkins/git-client-plugin.git' does not appear to be a git repository fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. Command line git 1.7.1, CentOS 6.7 $ git clone mwaite@wheezy64b.markwaite.net:45022/var/lib/git/mwaite/jenkins/git-client-plugin.git Initialized empty Git repository in /home/mwaite/tmp/git-client-plugin/.git/ fatal: '45022/var/lib/git/mwaite/jenkins/git-client-plugin.git' does not appear to be a git repository fatal: The remote end hung up unexpectedly The git clone documentation calls that "scp-line syntax" and seems to provide no way to include a port number in the syntax (other than through an alias in the ~/.ssh/config file). Embedding port number in git URL like ssh://git@hostname:port/directory If I perform the same clone commands on the same machines, with an ssh:// protocol URL, it works as expected. Command line git 2.9.0, Ubuntu 14.04 $ git clone ssh://mwaite@wheezy64b.markwaite.net:45022/var/lib/git/mwaite/jenkins/git-client-plugin.git Cloning into 'git-client-plugin'... remote: Counting objects: 11504, done. remote: Compressing objects: 100% (4289/4289), done. remote: Total 11504 (delta 4180), reused 11499 (delta 4176) Receiving objects: 100% (11504/11504), 1.88 MiB | 0 bytes/s, done. Resolving deltas: 100% (4180/4180), done. Checking connectivity... done. Command line git 1.7.1, CentOS 6.7 $ git clone ssh://mwaite@wheezy64b.markwaite.net:45022/var/lib/git/mwaite/jenkins/git-client-plugin.git Initialized empty Git repository in /home/mwaite/tmp/git-client-plugin/.git/ remote: Counting objects: 11504, done. remote: Compressing objects: 100% (4289/4289), done. remote: Total 11504 (delta 4180), reused 11499 (delta 4176) Receiving objects: 100% (11504/11504), 1.88 MiB, done. Resolving deltas: 100% (4180/4180), done. I don't think item 1 in your list is a bug, or if it is a bug, it is a bug in the command line git implementations I checked, not a bug in the git plugin, the git client plugin, or the multibranch plugin. Can you help me understand what I've missed on your first point?
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 171877 ] JNJira + In-Review [ 184529 ]
            Hide
            flow86 Florian Doersch added a comment -

            yes you're right, the first one should not be considered as bug, if you want to specify a custom port, one has to use "ssh://" syntax.

            But: at least the second one is definitly a bug - and a blocker for me since branch indexing fails with "Cannot open session, connection is not authenticated."

            Show
            flow86 Florian Doersch added a comment - yes you're right, the first one should not be considered as bug, if you want to specify a custom port, one has to use "ssh://" syntax. But: at least the second one is definitly a bug - and a blocker for me since branch indexing fails with "Cannot open session, connection is not authenticated."
            Hide
            flow86 Florian Doersch added a comment -

            another interesting fact:

            If you use the same clone url to build a freestyle project or a pipeline project (not a multibranch one!) checkout works.

            It seems the bug resides inside the multibranch pipeline stuff.

            Show
            flow86 Florian Doersch added a comment - another interesting fact: If you use the same clone url to build a freestyle project or a pipeline project (not a multibranch one!) checkout works. It seems the bug resides inside the multibranch pipeline stuff.
            flow86 Florian Doersch made changes -
            Component/s multi-branch-project-plugin [ 21127 ]
            Hide
            ptho ptho ho added a comment - - edited

            i have the same issue. Pipeline project works but multibranch pipeline project doesn't.

            Setting origin to git@somecompany.com:RnD/test.git
            Fetching origin...
            FATAL: Failed to recompute children of test_multibranch_pipeline
            java.lang.IllegalStateException: Cannot open session, connection is not authenticated.

            Add jgit too to GIT under Global Tool Configuration, still the same issue.

            A bug somewhere in multibranch pipeline. tried different things, still don't get GIT to work for multibranch pipeline.

            Show
            ptho ptho ho added a comment - - edited i have the same issue. Pipeline project works but multibranch pipeline project doesn't. Setting origin to git@somecompany.com:RnD/test.git Fetching origin... FATAL: Failed to recompute children of test_multibranch_pipeline java.lang.IllegalStateException: Cannot open session, connection is not authenticated. Add jgit too to GIT under Global Tool Configuration, still the same issue. A bug somewhere in multibranch pipeline. tried different things, still don't get GIT to work for multibranch pipeline.
            Hide
            sriniu190 Srinivasan Krishnan added a comment -

            I'm facing the same issue.
            and jgit workaround didn't help.

            Show
            sriniu190 Srinivasan Krishnan added a comment - I'm facing the same issue. and jgit workaround didn't help.
            Hide
            markewaite Mark Waite added a comment -

            If you're interested in trying the latest change to the git plugin, it may include a fix for some of the cases reported here.

            For example, if you rely on the processing of ~/.ssh/config to connect to your git repository, then the most recent git plugin change will allow you to use command line git to scan within the multi-branch plugin. Refer to the prototype build if you're willing to help test it.

            Show
            markewaite Mark Waite added a comment - If you're interested in trying the latest change to the git plugin, it may include a fix for some of the cases reported here. For example, if you rely on the processing of ~/.ssh/config to connect to your git repository, then the most recent git plugin change will allow you to use command line git to scan within the multi-branch plugin. Refer to the prototype build if you're willing to help test it.
            markewaite Mark Waite made changes -
            Link This issue duplicates JENKINS-37297 [ JENKINS-37297 ]
            markewaite Mark Waite made changes -
            Link This issue duplicates JENKINS-33983 [ JENKINS-33983 ]
            markewaite Mark Waite made changes -
            Link This issue duplicates JENKINS-36958 [ JENKINS-36958 ]
            Hide
            jglick Jesse Glick added a comment -

            Wrong component.

            Show
            jglick Jesse Glick added a comment - Wrong component.
            jglick Jesse Glick made changes -
            Component/s multi-branch-project-plugin [ 21127 ]
            Labels multibranch
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-35565 [ JENKINS-35565 ]
            Hide
            markewaite Mark Waite added a comment -

            I believe this is covered by git client plugin PR424

            Show
            markewaite Mark Waite added a comment - I believe this is covered by git client plugin PR424
            Hide
            markewaite Mark Waite added a comment -

            Included in git plugin 2.6.0, released 2 Sep 2016.

            Show
            markewaite Mark Waite added a comment - Included in git plugin 2.6.0 , released 2 Sep 2016.
            markewaite Mark Waite made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]

              People

              • Assignee:
                Unassigned
                Reporter:
                flow86 Florian Doersch
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: