Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47169

SCM polling runs rev-parse for every tag

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Duplicate
    • Component/s: git-plugin
    • Environment:
      Windows 2012 R2
      Jenkins 2.7.4
      Git Client Plugin 2.5.0
      Git Plugin 3.5.1
    • Similar Issues:

      Description

      Whenever the git plugin checks for changes, it runs git rev-parse for every tag in a repository. We have a hundred jobs or so, all of which have many tags. This causes the check for new changes to take longer than a minute. Our builds are configured to check for changes every minute. The net result is that Jenkins is spending all its time/CPU checking for changes. 

      We upgraded to the latest version of each plugin yesterday. I don't remember what version we were at.

      Here is our Git polling log output:

      Started on Sep 27, 2017 2:05:03 PM
      Polling SCM changes on master
      Using strategy: Default
      [poll] Last Built Revision: Revision 6d88077c1ed8fea0a847c1ce81dc4a7c693b6d3c (origin/master)
       > git.exe rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repositories
       > git.exe config remote.origin.url ssh://git@********.git # timeout=10
      Cleaning workspace
       > git.exe rev-parse --verify HEAD # timeout=10
      Resetting working tree
       > git.exe reset --hard # timeout=10
       > git.exe clean -fdx # timeout=10
      Pruning obsolete local branches
      Fetching upstream changes from ssh://git@********.git
       > git.exe --version # timeout=10
       > git.exe fetch --tags --progress ssh://git@********.git +refs/heads/*:refs/remotes/origin/* --prune
      Polling for changes in
      Seen branch in repository origin/develop
      Seen branch in repository origin/master
      Seen 2 remote branches
       > git.exe tag -l # timeout=10
       > git.exe rev-parse "refs/tags/0.1.1^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/0.1.2^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/0.1.0^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/0.1.10^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/0.1.9^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/Dev^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/0.1.5^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/0.1.3^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/0.1.4^{commit}" # timeout=10
       > git.exe rev-parse "refs/tags/0.2.4^{commit}" # timeout=10

      I've attached a screenshot of the job's Git settings.

      I've tried leaving the "branch" field blank and using refs/heads/**. Same result.

        Attachments

          Issue Links

            Activity

            splatteredbits Aaron Jensen created issue -
            splatteredbits Aaron Jensen made changes -
            Field Original Value New Value
            Description Whenever the git plugin checks for changes, it runs git rev-parse for every tag in a repository. We have a hundred jobs or so, all of which have many tags. This causes the check for new changes to take longer than a minute. Our builds are configured to check for changes every minute. The net result is that Jenkins is spending all its time/CPU checking for changes. 

            We upgraded to the latest version of each plugin yesterday. I don't remember what version we were at.
            Whenever the git plugin checks for changes, it runs git rev-parse for every tag in a repository. We have a hundred jobs or so, all of which have many tags. This causes the check for new changes to take longer than a minute. Our builds are configured to check for changes every minute. The net result is that Jenkins is spending all its time/CPU checking for changes. 

            We upgraded to the latest version of each plugin yesterday. I don't remember what version we were at.

            Here is our Git polling log output:
            {code:java}
            Started on Sep 27, 2017 2:05:03 PM
            Polling SCM changes on master
            Using strategy: Default
            [poll] Last Built Revision: Revision 6d88077c1ed8fea0a847c1ce81dc4a7c693b6d3c (origin/master)
             > git.exe rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git.exe config remote.origin.url ssh://git@whsbitbucket.webmd.net:7999/automation/depot.git # timeout=10
            Cleaning workspace
             > git.exe rev-parse --verify HEAD # timeout=10
            Resetting working tree
             > git.exe reset --hard # timeout=10
             > git.exe clean -fdx # timeout=10
            Pruning obsolete local branches
            Fetching upstream changes from ssh://git@whsbitbucket.webmd.net:7999/automation/depot.git
             > git.exe --version # timeout=10
             > git.exe fetch --tags --progress ssh://git@whsbitbucket.webmd.net:7999/automation/depot.git +refs/heads/*:refs/remotes/origin/* --prune
            Polling for changes in
            Seen branch in repository origin/develop
            Seen branch in repository origin/master
            Seen 2 remote branches
             > git.exe tag -l # timeout=10
             > git.exe rev-parse "refs/tags/0.1.1^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.2^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.0^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.10^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.9^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/Dev^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.5^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.3^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.4^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.2.4^{commit}" # timeout=10{code}
            markewaite Mark Waite made changes -
            Assignee Mark Waite [ markewaite ]
            splatteredbits Aaron Jensen made changes -
            Description Whenever the git plugin checks for changes, it runs git rev-parse for every tag in a repository. We have a hundred jobs or so, all of which have many tags. This causes the check for new changes to take longer than a minute. Our builds are configured to check for changes every minute. The net result is that Jenkins is spending all its time/CPU checking for changes. 

            We upgraded to the latest version of each plugin yesterday. I don't remember what version we were at.

            Here is our Git polling log output:
            {code:java}
            Started on Sep 27, 2017 2:05:03 PM
            Polling SCM changes on master
            Using strategy: Default
            [poll] Last Built Revision: Revision 6d88077c1ed8fea0a847c1ce81dc4a7c693b6d3c (origin/master)
             > git.exe rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git.exe config remote.origin.url ssh://git@whsbitbucket.webmd.net:7999/automation/depot.git # timeout=10
            Cleaning workspace
             > git.exe rev-parse --verify HEAD # timeout=10
            Resetting working tree
             > git.exe reset --hard # timeout=10
             > git.exe clean -fdx # timeout=10
            Pruning obsolete local branches
            Fetching upstream changes from ssh://git@whsbitbucket.webmd.net:7999/automation/depot.git
             > git.exe --version # timeout=10
             > git.exe fetch --tags --progress ssh://git@whsbitbucket.webmd.net:7999/automation/depot.git +refs/heads/*:refs/remotes/origin/* --prune
            Polling for changes in
            Seen branch in repository origin/develop
            Seen branch in repository origin/master
            Seen 2 remote branches
             > git.exe tag -l # timeout=10
             > git.exe rev-parse "refs/tags/0.1.1^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.2^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.0^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.10^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.9^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/Dev^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.5^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.3^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.4^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.2.4^{commit}" # timeout=10{code}
            Whenever the git plugin checks for changes, it runs git rev-parse for every tag in a repository. We have a hundred jobs or so, all of which have many tags. This causes the check for new changes to take longer than a minute. Our builds are configured to check for changes every minute. The net result is that Jenkins is spending all its time/CPU checking for changes. 

            We upgraded to the latest version of each plugin yesterday. I don't remember what version we were at.

            Here is our Git polling log output:
            {code:java}
            Started on Sep 27, 2017 2:05:03 PM
            Polling SCM changes on master
            Using strategy: Default
            [poll] Last Built Revision: Revision 6d88077c1ed8fea0a847c1ce81dc4a7c693b6d3c (origin/master)
             > git.exe rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git.exe config remote.origin.url ssh://git@********.git # timeout=10
            Cleaning workspace
             > git.exe rev-parse --verify HEAD # timeout=10
            Resetting working tree
             > git.exe reset --hard # timeout=10
             > git.exe clean -fdx # timeout=10
            Pruning obsolete local branches
            Fetching upstream changes from ssh://git@********.git
             > git.exe --version # timeout=10
             > git.exe fetch --tags --progress ssh://git@********.git +refs/heads/*:refs/remotes/origin/* --prune
            Polling for changes in
            Seen branch in repository origin/develop
            Seen branch in repository origin/master
            Seen 2 remote branches
             > git.exe tag -l # timeout=10
             > git.exe rev-parse "refs/tags/0.1.1^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.2^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.0^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.10^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.9^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/Dev^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.5^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.3^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.4^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.2.4^{commit}" # timeout=10{code}
            I've attached a screenshot of the job's Git settings.
            Hide
            markewaite Mark Waite added a comment - - edited

            Your best short term solution is to downgrade the git plugin. One of the unintended side effects of a change in git plugin 3.3.0 (I believe) was to dramatically increase the number of calls to git rev-parse. Refer to JENKINS-45447 for more details.

            Another alternative may be to add the advanced clone option to your jobs to not fetch tags.

            Show
            markewaite Mark Waite added a comment - - edited Your best short term solution is to downgrade the git plugin. One of the unintended side effects of a change in git plugin 3.3.0 (I believe) was to dramatically increase the number of calls to git rev-parse. Refer to JENKINS-45447 for more details. Another alternative may be to add the advanced clone option to your jobs to not fetch tags.
            markewaite Mark Waite made changes -
            Link This issue duplicates JENKINS-45447 [ JENKINS-45447 ]
            splatteredbits Aaron Jensen made changes -
            Attachment JenkinsDepotGitSettings.png [ 39877 ]
            splatteredbits Aaron Jensen made changes -
            Description Whenever the git plugin checks for changes, it runs git rev-parse for every tag in a repository. We have a hundred jobs or so, all of which have many tags. This causes the check for new changes to take longer than a minute. Our builds are configured to check for changes every minute. The net result is that Jenkins is spending all its time/CPU checking for changes. 

            We upgraded to the latest version of each plugin yesterday. I don't remember what version we were at.

            Here is our Git polling log output:
            {code:java}
            Started on Sep 27, 2017 2:05:03 PM
            Polling SCM changes on master
            Using strategy: Default
            [poll] Last Built Revision: Revision 6d88077c1ed8fea0a847c1ce81dc4a7c693b6d3c (origin/master)
             > git.exe rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git.exe config remote.origin.url ssh://git@********.git # timeout=10
            Cleaning workspace
             > git.exe rev-parse --verify HEAD # timeout=10
            Resetting working tree
             > git.exe reset --hard # timeout=10
             > git.exe clean -fdx # timeout=10
            Pruning obsolete local branches
            Fetching upstream changes from ssh://git@********.git
             > git.exe --version # timeout=10
             > git.exe fetch --tags --progress ssh://git@********.git +refs/heads/*:refs/remotes/origin/* --prune
            Polling for changes in
            Seen branch in repository origin/develop
            Seen branch in repository origin/master
            Seen 2 remote branches
             > git.exe tag -l # timeout=10
             > git.exe rev-parse "refs/tags/0.1.1^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.2^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.0^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.10^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.9^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/Dev^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.5^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.3^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.4^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.2.4^{commit}" # timeout=10{code}
            I've attached a screenshot of the job's Git settings.
            Whenever the git plugin checks for changes, it runs git rev-parse for every tag in a repository. We have a hundred jobs or so, all of which have many tags. This causes the check for new changes to take longer than a minute. Our builds are configured to check for changes every minute. The net result is that Jenkins is spending all its time/CPU checking for changes. 

            We upgraded to the latest version of each plugin yesterday. I don't remember what version we were at.

            Here is our Git polling log output:
            {code:java}
            Started on Sep 27, 2017 2:05:03 PM
            Polling SCM changes on master
            Using strategy: Default
            [poll] Last Built Revision: Revision 6d88077c1ed8fea0a847c1ce81dc4a7c693b6d3c (origin/master)
             > git.exe rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git.exe config remote.origin.url ssh://git@********.git # timeout=10
            Cleaning workspace
             > git.exe rev-parse --verify HEAD # timeout=10
            Resetting working tree
             > git.exe reset --hard # timeout=10
             > git.exe clean -fdx # timeout=10
            Pruning obsolete local branches
            Fetching upstream changes from ssh://git@********.git
             > git.exe --version # timeout=10
             > git.exe fetch --tags --progress ssh://git@********.git +refs/heads/*:refs/remotes/origin/* --prune
            Polling for changes in
            Seen branch in repository origin/develop
            Seen branch in repository origin/master
            Seen 2 remote branches
             > git.exe tag -l # timeout=10
             > git.exe rev-parse "refs/tags/0.1.1^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.2^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.0^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.10^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.9^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/Dev^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.5^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.3^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.1.4^{commit}" # timeout=10
             > git.exe rev-parse "refs/tags/0.2.4^{commit}" # timeout=10{code}
            I've attached a screenshot of the job's Git settings.

            I've tried leaving the "branch" field blank and using {{refs/heads/**}}. Same result.
            Hide
            splatteredbits Aaron Jensen added a comment -

            Downgrading to 3.0.0 of the Git plugin fixed this problem. 

            Show
            splatteredbits Aaron Jensen added a comment - Downgrading to 3.0.0 of the Git plugin fixed this problem. 
            markewaite Mark Waite made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Duplicate [ 3 ]
            Hide
            markewaite Mark Waite added a comment -

            Pull requests have been submitted for the git client plugin and the git plugin to resolve this bug without removing the functionality added in 31fedc.

            Please help with the evaluation of those pull requests by:

            1. Install pre-release git-client-plugin on your Jenkins server
            2. Install pre-release git plugin on your Jenkins server
            3. Restart the Jenkins server to complete the plugin installation
            4. Poll with a freestyle job. which use "advanced branch specifications" (for example, a branch name with a wildcard character)
            5. Review the polling log of freestyle jobs which use "advanced branch specifications", checking for calls to git rev-parse.  There should be few or none.  Polling should be significantly faster than before this change
            6. Build a Review the build log of freestyle jobs which use "advanced branch specifications", checking for calls to git rev-parse.  There should be few or none.  Checkout should be significantly faster than before this change

            If you're comfortable with Java, it would also help to review the pull requests (git client plugin and git plugin).

             

            Show
            markewaite Mark Waite added a comment - Pull requests have been submitted for the  git client plugin  and the  git plugin  to resolve this bug without removing the functionality added in  31fedc . Please help with the evaluation of those pull requests by: Install  pre-release git-client-plugin  on your Jenkins server Install  pre-release git plugin  on your Jenkins server Restart the Jenkins server to complete the plugin installation Poll with a freestyle job. which use "advanced branch specifications" (for example, a branch name with a wildcard character) Review the polling log of freestyle jobs which use "advanced branch specifications", checking for calls to  git rev-parse .  There should be few or none.  Polling should be significantly faster than before this change Build a Review the build log of freestyle jobs which use "advanced branch specifications", checking for calls to  git rev-parse .  There should be few or none.  Checkout should be significantly faster than before this change If you're comfortable with Java, it would also help to review the pull requests ( git client plugin  and  git plugin ).  
            Hide
            markewaite Mark Waite added a comment -

            Will be fixed in git client plugin 2.7.0 and git plugin 3.7.0

            Show
            markewaite Mark Waite added a comment - Will be fixed in git client plugin 2.7.0 and git plugin 3.7.0
            markewaite Mark Waite made changes -
            Status Resolved [ 5 ] Closed [ 6 ]

              People

              • Assignee:
                Unassigned
                Reporter:
                splatteredbits Aaron Jensen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: