Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-5724

Choosing what to build is slow when using a lot of branches.

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: git-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      I have something like over 100 branches in git, and selecting which one to build is extremely slow (takes about 20 minutes). I think that I've tracked problem to filterTipBranches in GitUtil.java where the plugin loops through all branches.

        Attachments

          Activity

          Hide
          jyrkiput jyrkiput added a comment -

          Use case for this issue was resolved by implementing Gerrit Build chooser, so there's no need for this at the moment. Problem still exists, though.

          Show
          jyrkiput jyrkiput added a comment - Use case for this issue was resolved by implementing Gerrit Build chooser, so there's no need for this at the moment. Problem still exists, though.
          Hide
          jrogers Jonathan Rogers added a comment -

          This remains a big problem for anyone with a large number of branches. I eventually abandoned the git plugin entirely in favor of custom scripts to handle checkouts.

          Show
          jrogers Jonathan Rogers added a comment - This remains a big problem for anyone with a large number of branches. I eventually abandoned the git plugin entirely in favor of custom scripts to handle checkouts.
          Hide
          nh2 Niklas Hambuechen added a comment -

          I fixed that in this pull request:

          https://github.com/jenkinsci/git-plugin/pull/156

          It is much faster now, please try it and tell me if it works.

          Show
          nh2 Niklas Hambuechen added a comment - I fixed that in this pull request: https://github.com/jenkinsci/git-plugin/pull/156 It is much faster now, please try it and tell me if it works.
          Hide
          markewaite Mark Waite added a comment -

          After no feedback for 9 months, I'm closing this as resolved.

          Show
          markewaite Mark Waite added a comment - After no feedback for 9 months, I'm closing this as resolved.
          Hide
          nh2 Niklas Hambuechen added a comment -

          I don't think this is resolved; https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/util/GitUtils.java#L141 still looks like the original algorithm.

          Unfortunately I don't have time to perform a rebase on top of a new major release (as asked for in https://github.com/jenkinsci/git-plugin/pull/156#issuecomment-26217539) until earliest in 2 weeks.

          Show
          nh2 Niklas Hambuechen added a comment - I don't think this is resolved; https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/util/GitUtils.java#L141 still looks like the original algorithm. Unfortunately I don't have time to perform a rebase on top of a new major release (as asked for in https://github.com/jenkinsci/git-plugin/pull/156#issuecomment-26217539 ) until earliest in 2 weeks.
          Hide
          nh2 Niklas Hambuechen added a comment -

          Ok, I rebased it (see the Github pull-request).

          I also made a script to create a small test repo for reproducing this; find it at https://gist.github.com/nh2/9902001.

          Show
          nh2 Niklas Hambuechen added a comment - Ok, I rebased it (see the Github pull-request). I also made a script to create a small test repo for reproducing this; find it at https://gist.github.com/nh2/9902001 .
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Niklas Hamb├╝chen
          Path:
          src/main/java/hudson/plugins/git/util/GitUtils.java
          http://jenkins-ci.org/commit/git-plugin/ba3271bb40d3933aa5e6dd117581937b03ab1030
          Log:
          filterTipBranches: Use O instead of O(n^2) algorithm.

          Fixes JENKINS-5724.

          Before, this made slaves run on 100% CPU for days when idle.

          This makes it a few thousand times faster for a repo with 500 branches.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Niklas Hamb├╝chen Path: src/main/java/hudson/plugins/git/util/GitUtils.java http://jenkins-ci.org/commit/git-plugin/ba3271bb40d3933aa5e6dd117581937b03ab1030 Log: filterTipBranches: Use O instead of O(n^2) algorithm. Fixes JENKINS-5724 . Before, this made slaves run on 100% CPU for days when idle. This makes it a few thousand times faster for a repo with 500 branches.
          Hide
          markewaite Mark Waite added a comment -

          Confirmed resolved in git-plugin 2.1.0. Sample job using https://github.com/MarkEWaite/JENKINS-5724-many-branches went from 3 minutes for an empty build to 1.5 seconds for the same build with git-plugin 2.1.0.

          Show
          markewaite Mark Waite added a comment - Confirmed resolved in git-plugin 2.1.0. Sample job using https://github.com/MarkEWaite/JENKINS-5724-many-branches went from 3 minutes for an empty build to 1.5 seconds for the same build with git-plugin 2.1.0.

            People

            • Assignee:
              nh2 Niklas Hambuechen
              Reporter:
              jyrkiput jyrkiput
            • Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: