Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-25212

http clone of git usercontent folder fails with some git versions, even though ssh clone works

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: git-usercontent-plugin
    • Labels:
      None
    • Environment:
      git-usercontent-plugin 1.4
      git-plugin 2.2.7
      git-client-plugin 1.11.1
      Jenkins 1.580.1
      Debian Jessie (git 1.9.4.0)
      Windows 7 (msysgit 1.9.4)
      CentOS 7 (git 1.8.3.0)
    • Similar Issues:

      Description

      The git-usercontent plugin reports an error when I try to clone the repository outside Jenkins using git command line to clone through the http URL (for example http://mark-pc1:45089/.

      The following ssh based clone works in all my tests:

      git clone ssh://mark-pc1:49438/userContent.git
      

      The following http based clone fails (on the specified git versions):

      git clone http://mark-pc1:8080/userContent.git
      

      The http based clone succeeds on Debian Wheezy (git 1.7.10.4).

      The http based clone fails on Debian Jessie (git 1.9.4.0), Ubuntu 14.04 (git 1.9.1), and CentOS 7 (git 1.8.3.0).

      The command line git error message in the failure is:

      mwaite@mark-pc1:~$ git clone http://mark-pc1:8080/userContent.git
      Cloning into 'userContent'...
      fatal: http://mark-pc1:8080/userContent.git/info/refs not valid: is this a git repository?
      

      The work around is to use ssh to clone the repository instead of http.

        Attachments

          Activity

          Hide
          markewaite Mark Waite added a comment -

          I confirmed this is still a bug with git-usercontent-plugin 1.4, though the behavior has changed. Previously, when cloning through http to a newer version of git, it reported info/refs not valid. Now it reports that it is cloning an empty repository, and creates the repository with no commits, even though the repository clones with all its commits through ssh.

          Show
          markewaite Mark Waite added a comment - I confirmed this is still a bug with git-usercontent-plugin 1.4, though the behavior has changed. Previously, when cloning through http to a newer version of git, it reported info/refs not valid. Now it reports that it is cloning an empty repository, and creates the repository with no commits, even though the repository clones with all its commits through ssh.
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          Confirmed "info/refs not valid" error. I need to understand what is it that the git-client doesn't like. curl reports that the URL in question reports a seemingly valid content back.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - Confirmed "info/refs not valid" error. I need to understand what is it that the git-client doesn't like. curl reports that the URL in question reports a seemingly valid content back.
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          After a painful debugging, I found out that jgit recognizes "Accept-Encoding" header and performs gzip compression on its own in SmartOutputStream. It also uses the ServletResponse.setContentLength().

          But at least on mvn hpi:run, this results in double content encoding via gzip, because either the servlet engine or some intermediate filter is also responding to the "Accept-Encoding" header and inserting GZIPOutputStream. Yet the Content-Length header value remains intact, and therefore curl that invokes the command gets the truncated output.

          There are two issues here. Whoever inserting GZIPOutputStream needs to mangle the Content-Length}} header (or at least voids that), and second, I need to think about ways to avoid double gzip encoding.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - After a painful debugging, I found out that jgit recognizes "Accept-Encoding" header and performs gzip compression on its own in SmartOutputStream . It also uses the ServletResponse.setContentLength() . But at least on mvn hpi:run , this results in double content encoding via gzip, because either the servlet engine or some intermediate filter is also responding to the "Accept-Encoding" header and inserting GZIPOutputStream . Yet the Content-Length header value remains intact, and therefore curl that invokes the command gets the truncated output. There are two issues here. Whoever inserting GZIPOutputStream needs to mangle the Content-Length}} header (or at least voids that), and second, I need to think about ways to avoid double gzip encoding.
          Hide
          kohsuke Kohsuke Kawaguchi added a comment - - edited

          After futher thinking and digging, I think the root cause is that CompressionFilter was using setHeader("Content-Encoding","gzip") as a trigger to automatically insert transparent gzip compression.

          This just doesn't work with servlets that handle gzip content encoding on its own, such as jgit.

          In https://github.com/stapler/stapler/compare/08d582d...5b6496c9 I made a fix to change the way the transparent content compression kicks in. This will be in Stapler 1.234.

          Because this is a core change, git-server plugin should work around this in the mean time by hiding "Accept-Encoding" from the request.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - - edited After futher thinking and digging, I think the root cause is that CompressionFilter was using setHeader("Content-Encoding","gzip") as a trigger to automatically insert transparent gzip compression. This just doesn't work with servlets that handle gzip content encoding on its own, such as jgit. In https://github.com/stapler/stapler/compare/08d582d...5b6496c9 I made a fix to change the way the transparent content compression kicks in. This will be in Stapler 1.234. Because this is a core change, git-server plugin should work around this in the mean time by hiding "Accept-Encoding" from the request.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          src/main/java/org/jenkinsci/plugins/gitserver/HttpGitRepository.java
          src/main/java/org/jenkinsci/plugins/gitserver/Jenkins2521Workaround.java
          http://jenkins-ci.org/commit/git-server-plugin/12842a74e8199742530ae16a70994ee66d730bf6
          Log:
          [FIXED JENKINS-25212]

          Fixing this problem in this plugin by not allowing jgit to compress the response. This can be removed once we can depend on the core that has Stapler 1.234.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/org/jenkinsci/plugins/gitserver/HttpGitRepository.java src/main/java/org/jenkinsci/plugins/gitserver/Jenkins2521Workaround.java http://jenkins-ci.org/commit/git-server-plugin/12842a74e8199742530ae16a70994ee66d730bf6 Log: [FIXED JENKINS-25212] Fixing this problem in this plugin by not allowing jgit to compress the response. This can be removed once we can depend on the core that has Stapler 1.234.
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          git-server-plugin 1.6 is released to address this problem.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - git-server-plugin 1.6 is released to address this problem.

            People

            • Assignee:
              Unassigned
              Reporter:
              markewaite Mark Waite
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: