Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49757

Git plugin calls fetch twice per checkout

    Details

    • Type: Bug
    • Status: Fixed but Unreleased (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: git-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      When performing a git clone operation,

      git-client and git-plugin both triggers a git fetch.

       

      Below are the two lines of code running in the same git clone command both doing fetches.

       

      The second link is part of the GITSCM step, it triggers the git-client-plugin's CloneCommand (first link) to perform the clone operation. And afterwards, the GITSCM step performs a fetch once the CloneCommand Completes.

      However, within the CloneCommand (first link) itself, a fetch command was already done.

       

      https://github.com/jenkinsci/git-client-plugin/blob/4241e818dbe27fb78093094f29ebcd2203166610/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L609

       

      https://github.com/jenkinsci/git-plugin/blob/cb58c74efafafd08bc03bba2787997c6a0272df0/src/main/java/hudson/plugins/git/GitSCM.java#L1153

       

      I am aware there are needs to support for git clone to fetch all refs first -JENKINS-31393-. The suggestion provided in that ticket was to use "honor git refspec", so shouldn't "honor git refspec" skip the first fetch here?

       

      Thanks,

      Sam

        Attachments

          Issue Links

            Activity

            Hide
            dogin2006 Kaiyuan Zhao added a comment -

            Mark Waite

            Using "${GERRIT_REFSPEC}" fix the first problem.

             

            10:38:45 Cloning the remote Git repository
            10:38:45 Using shallow clone
            10:38:45 shallow clone depth 2
            10:38:45 Avoid fetching tags
            10:38:45 Honoring refspec on initial clone
            10:38:45 Cloning repository hide
            10:38:45 > git init hide # timeout=10
            10:38:45 Fetching upstream changes from hide
            10:38:45 > git --version # timeout=10
            10:38:45 using GIT_SSH to set credentials 
            10:38:45 > git fetch --no-tags --progress hide refs/changes/50/4750/1 --depth=2 # timeout=10
            10:39:34 > git config remote.origin.url hide # timeout=10
            10:39:34 > git config --add remote.origin.fetch refs/changes/50/4750/1 # timeout=10
            10:39:34 > git config remote.origin.url hide # timeout=10
            10:39:34 Fetching upstream changes from hide
            10:39:34 using GIT_SSH to set credentials 
            10:39:34 > git fetch --no-tags --progress hide refs/changes/50/4750/1 --depth=2 # timeout=10
            10:40:34 > git rev-parse 44fd5ca794ef1d4ccbfb437d94ef804626dea726^{commit} # timeout=10
            10:40:34 Checking out Revision 44fd5ca794ef1d4ccbfb437d94ef804626dea726 (refs/changes/50/4750/1)
            10:40:34 > git config core.sparsecheckout # timeout=10
            10:40:34 > git checkout -f 44fd5ca794ef1d4ccbfb437d94ef804626dea726 # timeout=10
            10:40:38 Commit message: "hide"
            10:40:38 > git rev-parse FETCH_HEAD^{commit} # timeout=10
            10:40:38 > git rev-list --no-walk eb696053ba8f4c94c388db0334b886879e6aaa42 # timeout=10
            

            But it's also triggering fetch twice and the second fetch is not cost free.

            In our case, both fetch cost about one minute.

            Show
            dogin2006 Kaiyuan Zhao added a comment - Mark Waite Using "${GERRIT_REFSPEC}" fix the first problem.   10:38:45 Cloning the remote Git repository 10:38:45 Using shallow clone 10:38:45 shallow clone depth 2 10:38:45 Avoid fetching tags 10:38:45 Honoring refspec on initial clone 10:38:45 Cloning repository hide 10:38:45 > git init hide # timeout=10 10:38:45 Fetching upstream changes from hide 10:38:45 > git --version # timeout=10 10:38:45 using GIT_SSH to set credentials 10:38:45 > git fetch --no-tags --progress hide refs/changes/50/4750/1 --depth=2 # timeout=10 10:39:34 > git config remote.origin.url hide # timeout=10 10:39:34 > git config --add remote.origin.fetch refs/changes/50/4750/1 # timeout=10 10:39:34 > git config remote.origin.url hide # timeout=10 10:39:34 Fetching upstream changes from hide 10:39:34 using GIT_SSH to set credentials 10:39:34 > git fetch --no-tags --progress hide refs/changes/50/4750/1 --depth=2 # timeout=10 10:40:34 > git rev-parse 44fd5ca794ef1d4ccbfb437d94ef804626dea726^{commit} # timeout=10 10:40:34 Checking out Revision 44fd5ca794ef1d4ccbfb437d94ef804626dea726 (refs/changes/50/4750/1) 10:40:34 > git config core.sparsecheckout # timeout=10 10:40:34 > git checkout -f 44fd5ca794ef1d4ccbfb437d94ef804626dea726 # timeout=10 10:40:38 Commit message: "hide" 10:40:38 > git rev-parse FETCH_HEAD^{commit} # timeout=10 10:40:38 > git rev-list --no-walk eb696053ba8f4c94c388db0334b886879e6aaa42 # timeout=10 But it's also triggering fetch twice and the second fetch is not cost free. In our case, both fetch cost about one minute.
            Hide
            markewaite Mark Waite added a comment -

            Kaiyuan Zhao you're welcome to propose a pull request to the plugin to improve its behavior. Alternately, you could try replacing the "depth=2" with a reference repository to see if that would reduce the fetch time.

            Show
            markewaite Mark Waite added a comment - Kaiyuan Zhao you're welcome to propose a pull request to the plugin to improve its behavior. Alternately, you could try replacing the "depth=2" with a reference repository to see if that would reduce the fetch time.
            Hide
            oliverp Oliver Pereira added a comment -

            Using a reference repository doesn't fix the issue.

            Using the command line git clone including reference repository, it takes approx 11 seconds and the same configuration via the checkout scm option takes 1 minute 3x seconds.

             

            Show
            oliverp Oliver Pereira added a comment - Using a reference repository doesn't fix the issue. Using the command line git clone including reference repository, it takes approx 11 seconds and the same configuration via the checkout scm option takes 1 minute 3x seconds.  
            Hide
            rishabhbudhouliya Rishabh Budhouliya added a comment -

            I have attempted to fix this issue: 
            https://github.com/jenkinsci/git-plugin/pull/845

            Show
            rishabhbudhouliya Rishabh Budhouliya added a comment - I have attempted to fix this issue:  https://github.com/jenkinsci/git-plugin/pull/845
            Hide
            markewaite Mark Waite added a comment -

            The fix from Rishabh Budhouliya has been merged to the master branch and will be included in the next release of the git plugin

            Show
            markewaite Mark Waite added a comment - The fix from Rishabh Budhouliya has been merged to the master branch and will be included in the next release of the git plugin

              People

              • Assignee:
                Unassigned
                Reporter:
                slcy sam leung
              • Votes:
                2 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: