Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56404

Impossible exclude redundant/double fetches

    Details

    • Similar Issues:

      Description

      When hasGitRepo is false, retrieveChanges() tries firstly to Clone repository and then fetch according to available list of repositories.
      https://github.com/jenkinsci/git-plugin/blob/e9d15cd3a390f1d1849b9d6cbee707bc85213ab0/src/main/java/hudson/plugins/git/GitSCM.java#L1104-L1110
      But when it clones it picks first repository name and injects ALL branches fetch https://github.com/jenkinsci/git-client-plugin/blob/2b217bd90a93462112176febbb5f30e05141c674/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L704-L712
      So, when refspecs is empty - it fetch all branches (git default), when it null it fetches all branches.

      The workaround is to set some refspec from branch i.e. https://github.com/jenkinsci/git-plugin/blob/a7ad6eb38a00397c4a656b53d99bfbad8ea3b2cc/src/main/java/jenkins/plugins/git/GitSCMSourceDefaults.java#L108-L115
      But, later it will be fetched anyway.

      Solution:
      Don't fetch anything during clone operation (useful part is clean, fetch etc). OR pass repositories to CloneComand and remove separate fetch iteration .

        Attachments

          Issue Links

            Activity

            Hide
            markewaite Mark Waite added a comment -

            As far as I can tell from testing, when a refspec is defined and the clone option "honor refspec on inital clone" is enabled, the refspec is honored in both fetches that occur. Do you have a case where the clone option "honor refspec on initial clone" is not having the desired result?

            It is an intentional choice to retain compatibility with the existing (flawed) behavior of using a default refspec which fetches all branches. Too many users depend on fetching all branches without realizing they have that dependency. If the default behavior were changed, they would need to adjust the definitions of those jobs.

            Multibranch pipeline intentionally changed its behavior relatively early in development. Originally it used the default refspec for each job defined by the multibranch pipeline. A breaking change was accepted that uses a narrow refspec for exactly the branch dedicated to the specific job. Even that early stage breaking change caused some challenges for user, but the challenge was considered worth it due to the savings provided to multibranch pipeline jobs by the narrow refspec.

            Show
            markewaite Mark Waite added a comment - As far as I can tell from testing, when a refspec is defined and the clone option " honor refspec on inital clone " is enabled, the refspec is honored in both fetches that occur. Do you have a case where the clone option " honor refspec on initial clone " is not having the desired result? It is an intentional choice to retain compatibility with the existing (flawed) behavior of using a default refspec which fetches all branches. Too many users depend on fetching all branches without realizing they have that dependency. If the default behavior were changed, they would need to adjust the definitions of those jobs. Multibranch pipeline intentionally changed its behavior relatively early in development. Originally it used the default refspec for each job defined by the multibranch pipeline. A breaking change was accepted that uses a narrow refspec for exactly the branch dedicated to the specific job. Even that early stage breaking change caused some challenges for user, but the challenge was considered worth it due to the savings provided to multibranch pipeline jobs by the narrow refspec.
            Hide
            integer Kanstantsin Shautsou added a comment -

            "It is an intentional choice to retain compatibility with the existing (flawed) behavior of using a default refspec which fetches all branches. Too many users depend on fetching all branches without realizing they have that dependency. If the default behavior were changed, they would need to adjust the definitions of those jobs."
            That absolutely unrelated, refspec must tbe added only when repositories are not defined, but they are DEFINED and ignored, code is taking ONLY origin name from the first repository (refspec).

            ", the refspec is honored in both fetches that occur. "
            It shouldn't do two fetches, on huge instances it's only adding unneeded load on git server - this is a BUG.
            When refspecs (repositories) are clearly specified it's fetching ANYWAY all branches, because it takes from the list of repositories.

            In current situatiton it's IMPOSSIBLE to avoid double fetches: impossible to inherite and reimplement retrieve method, impossible to exclude fetch during clone or exclude all fetches that happen later. This is critical for performance and huge instances.

            Show
            integer Kanstantsin Shautsou added a comment - "It is an intentional choice to retain compatibility with the existing (flawed) behavior of using a default refspec which fetches all branches. Too many users depend on fetching all branches without realizing they have that dependency. If the default behavior were changed, they would need to adjust the definitions of those jobs." That absolutely unrelated, refspec must tbe added only when repositories are not defined, but they are DEFINED and ignored, code is taking ONLY origin name from the first repository (refspec). ", the refspec is honored in both fetches that occur. " It shouldn't do two fetches, on huge instances it's only adding unneeded load on git server - this is a BUG. When refspecs (repositories) are clearly specified it's fetching ANYWAY all branches, because it takes from the list of repositories. In current situatiton it's IMPOSSIBLE to avoid double fetches: impossible to inherite and reimplement retrieve method, impossible to exclude fetch during clone or exclude all fetches that happen later. This is critical for performance and huge instances.

              People

              • Assignee:
                Unassigned
                Reporter:
                integer Kanstantsin Shautsou
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: