Details

    • Type: New Feature
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: git-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      For a project with many submodules, updating one submodule at a time can be quite slow.

      Instead, allow the option to update <n> submodules in parallel by using the jobs flag.

      Excerpt from Git SCM docs.

      -j <n>
      --jobs <n>

      "This option is only valid for the update command. Clone new submodules in parallel with as many jobs. Defaults to the submodule.fetchJobs option."

        Attachments

          Activity

          Hide
          danjford Daniel Ford added a comment -

          Mark Waite, I noticed that this was automatically assigned to you. Can you give me any news regarding this?

          Show
          danjford Daniel Ford added a comment - Mark Waite , I noticed that this was automatically assigned to you. Can you give me any news regarding this?
          Hide
          markewaite Mark Waite added a comment - - edited

          I am unlikely to investigate this within the next 18 months or more.

          If submodule update is too slow, that may mean that you should use a reference repository which contains bare copies of all your submodules. Then git creates pointers to the reference copy instead of copying it from the remote repository.

          If it is crucial to you, you're encouraged to submit a pull request with the implementation, and with tests to confirm your implementation is well behaved.

          Show
          markewaite Mark Waite added a comment - - edited I am unlikely to investigate this within the next 18 months or more. If submodule update is too slow, that may mean that you should use a reference repository which contains bare copies of all your submodules. Then git creates pointers to the reference copy instead of copying it from the remote repository. If it is crucial to you, you're encouraged to submit a pull request with the implementation, and with tests to confirm your implementation is well behaved.
          Hide
          nre_ableton Nik Reiman added a comment - - edited

          This issue is also important to me, as at my company we have a very large git repo with over 30 submodules. Cloning speeds, even with reference repositories, are quite slow (especially on Windows). We're talking ~10 minutes just for the checkout step, but I have done some basic tests and we estimate we can get this down to just a minute or two with a combination of reference repositories and updating submodules in parallel.

          Mark Waite I am sympathetic towards your workload and understand that you have other priorities. I've done some Java development in the past, and I poked around the git-plugin and git-client-plugin source code. I think I've come up with a solution, but I'll probably need some help getting it merged since it's my first contribution to any Jenkins plugin. Your help and patience would be appreciated!

          Let's continue this discussion at GitHub, where I will submit a pull request soon.

          Show
          nre_ableton Nik Reiman added a comment - - edited This issue is also important to me, as at my company we have a very large git repo with over 30 submodules. Cloning speeds, even with reference repositories, are quite slow (especially on Windows). We're talking ~10 minutes just for the checkout step, but I have done some basic tests and we estimate we can get this down to just a minute or two with a combination of reference repositories and updating submodules in parallel. Mark Waite I am sympathetic towards your workload and understand that you have other priorities. I've done some Java development in the past, and I poked around the git-plugin and git-client-plugin source code. I think I've come up with a solution, but I'll probably need some help getting it merged since it's my first contribution to any Jenkins plugin. Your help and patience would be appreciated! Let's continue this discussion at GitHub, where I will submit a pull request soon.
          Show
          nre_ableton Nik Reiman added a comment - PR is here: https://github.com/jenkinsci/git-client-plugin/pull/348
          Hide
          nre_ableton Nik Reiman added a comment -

          PR to git-client-plugin has been merged, corresponding PR to git-plugin is here: https://github.com/jenkinsci/git-plugin/pull/612

          Show
          nre_ableton Nik Reiman added a comment - PR to git-client-plugin has been merged, corresponding PR to git-plugin is here:  https://github.com/jenkinsci/git-plugin/pull/612
          Hide
          nre_ableton Nik Reiman added a comment -

          Fixed and will be released with git plugin version 4.0.0 / git-client plugin version 3.0.0

          Show
          nre_ableton Nik Reiman added a comment - Fixed and will be released with git plugin version 4.0.0 / git-client plugin version 3.0.0

            People

            • Assignee:
              nre_ableton Nik Reiman
              Reporter:
              danjford Daniel Ford
            • Votes:
              3 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: