Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50556

git polling succeeds even when origin/master has nothing new

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: In Progress (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: git-plugin
    • Labels:
      None
    • Environment:
      Git Plugin 3.3.2
      Jenkins 2.60.1
    • Similar Issues:

      Description

      I have a repository which is setup to perform git polling, and it always triggers, suggesting there are changes, even though there are not.

      In this case, the job has multiple checkouts using a pipeline, but I think it would only happen in the case of a single checkout as well.

      Specifically, the branch name is master, and it checks first all the "remote/branch" combinations, (in this case, "origin/master") and then fails to find anything new (as origin/master was already built). Then, it checks the branch name on its own, thinking that maybe it's a tag or some other ref.

      This results in reporting a separate commit (likely whatever master happens to be at when the polling was created), and thus reports a new commit to build. This causes the build to trigger every polling timeout, even though there aren't any real new changes.

      Here's a subset of the actual output with repository names and paths redacted. I have the VERBOSE logging enabled for more information.

      Started on Apr 3, 2018 4:01:00 PM
      Using strategy: Default
      [poll] Last Built Revision: Revision 72c4cea985bb5ff14813d27dac94f34887cbd841 (refs/remotes/origin/master)
      using GIT_SSH to set credentials 
       > git ls-remote -h <groovyLibURL> # timeout=10
      Found 1 remote heads on <groovyLibURL>
      [poll] Latest remote head revision on refs/heads/master is: 72c4cea985bb5ff14813d27dac94f34887cbd841 - already built by 1738
      Using strategy: Default
      [poll] Last Built Revision: Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repositories
       > git config remote.origin.url <mainRepositoryURL> # timeout=10
      Fetching upstream changes from <mainRepositoryURL>
       > git --version # timeout=10
      using GIT_SSH to set credentials 
       > git fetch --no-tags --progress <mainRepositoryURL> +refs/heads/*:refs/remotes/origin/*
      Polling for changes in
      getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@3fcd3164[scmName=<mainRepositoryName>,remoteUrls=[<mainRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)},lastBuild=Build #1738 of Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)]) considering branches to build
      Qualifying master as a branch in repository origin -> origin/master
       > git rev-parse origin/master^{commit} # timeout=10
      rev-parse origin/master -> AnyObjectId[cc24044526e9d460c74a59fcdeb12e43a446cb7d]
      AnyObjectId[cc24044526e9d460c74a59fcdeb12e43a446cb7d] has already been built
       > git rev-parse master^{commit} # timeout=10
      Failed to rev-parse: master
      Using strategy: Default
      [poll] Last Built Revision: Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repositories
       > git config remote.origin.url <firstRepositoryURL> # timeout=10
      Fetching upstream changes from <firstRepositoryURL>
       > git --version # timeout=10
      using GIT_SSH to set credentials 
       > git fetch --no-tags --progress <firstRepositoryURL> +refs/heads/*:refs/remotes/origin/*
      Polling for changes in
      getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@857d3e89[scmName=<firstRepositoryName>,remoteUrls=[<firstRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)},lastBuild=Build #1738 of Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)]) considering branches to build
      Qualifying master as a branch in repository origin -> origin/master
       > git rev-parse origin/master^{commit} # timeout=10
      rev-parse origin/master -> AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc]
      AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc] has already been built
       > git rev-parse master^{commit} # timeout=10
      rev-parse master -> AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc]
      AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc] has already been built
      Using strategy: Default
      [poll] Last Built Revision: Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repositories
       > git config remote.origin.url <secondRepositoryURL> # timeout=10
      Fetching upstream changes from <secondRepositoryURL>
       > git --version # timeout=10
      using GIT_SSH to set credentials 
       > git fetch --no-tags --progress <secondRepositoryURL> +refs/heads/*:refs/remotes/origin/*
      Polling for changes in
      getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@2e191547[scmName=<secondRepositoryName>,remoteUrls=[<secondRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)},lastBuild=Build #1738 of Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)]) considering branches to build
      Qualifying master as a branch in repository origin -> origin/master
       > git rev-parse origin/master^{commit} # timeout=10
      rev-parse origin/master -> AnyObjectId[93a6d3510777096a6fbd7f16ac22c28ef4bd872e]
      AnyObjectId[93a6d3510777096a6fbd7f16ac22c28ef4bd872e] has already been built
       > git rev-parse master^{commit} # timeout=10
      rev-parse master -> AnyObjectId[4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8]
      Found a new commit AnyObjectId[4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8] to be built on master
      {0} seems to be a non-branch reference (tag?)
       > git log --full-history --no-abbrev --format=raw -M -m 93a6d3510777096a6fbd7f16ac22c28ef4bd872e..4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8 # timeout=10
      Using strategy: Default
      [poll] Last Built Revision: Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repositories
       > git config remote.origin.url <thirdRepositoryURL> # timeout=10
      Fetching upstream changes from <thirdRepositoryURL>
       > git --version # timeout=10
      using GIT_SSH to set credentials 
       > git fetch --no-tags --progress <thirdRepositoryURL> +refs/heads/*:refs/remotes/origin/*
      Polling for changes in
      getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@49f3b5e7[scmName=<thirdRepositoryName>,remoteUrls=[<thirdRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)},lastBuild=Build #1738 of Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)]) considering branches to build
      Qualifying master as a branch in repository origin -> origin/master
       > git rev-parse origin/master^{commit} # timeout=10
      rev-parse origin/master -> AnyObjectId[a4db49e3433ef3304c7956e81369c2ed5fa69b5d]
      AnyObjectId[a4db49e3433ef3304c7956e81369c2ed5fa69b5d] has already been built
       > git rev-parse master^{commit} # timeout=10
      rev-parse master -> AnyObjectId[00f059f63e272c44fb3f754be4e2f4a1b474c416]
      Found a new commit AnyObjectId[00f059f63e272c44fb3f754be4e2f4a1b474c416] to be built on master
      {0} seems to be a non-branch reference (tag?)
       > git log --full-history --no-abbrev --format=raw -M -m a4db49e3433ef3304c7956e81369c2ed5fa69b5d..00f059f63e272c44fb3f754be4e2f4a1b474c416 # timeout=10
      Using strategy: Default
      [poll] Last Built Revision: Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repositories
       > git config remote.origin.url <fourthRepositoryURL> # timeout=10
      Fetching upstream changes from <fourthRepositoryURL>
       > git --version # timeout=10
      using GIT_SSH to set credentials 
       > git fetch --no-tags --progress <fourthRepositoryURL> +refs/heads/*:refs/remotes/origin/*
      Polling for changes in
      getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@dff532d1[scmName=<fourthRepositoryName>,remoteUrls=[<fourthRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)},lastBuild=Build #1738 of Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)]) considering branches to build
      Qualifying master as a branch in repository origin -> origin/master
       > git rev-parse origin/master^{commit} # timeout=10
      rev-parse origin/master -> AnyObjectId[06188df26697ef38293a8533ce601934762c7d6c]
      AnyObjectId[06188df26697ef38293a8533ce601934762c7d6c] has already been built
       > git rev-parse master^{commit} # timeout=10
      Failed to rev-parse: master
      Done. Took 6.5 sec
      Changes found
      

      I am highly suspect that the relevant commit related to why this breaks is:

      c7dae80b85c1 ("JENKINS-21952 Resolve tags with slashes", 2014-02-27)

      I believe what is happening is that the "remote/branch" output is failing to find any new changes, so we fall back to attempting to resolve the name like a tag, but it so happens that the polling setup actually does end up with the branch checked out, and it results in us finding the local master name and reporting it.

      I don't really know how to resolve this yet.

        Attachments

          Issue Links

            Activity

            jekeller Jacob Keller created issue -
            Hide
            jekeller Jacob Keller added a comment - - edited

            It looks like this is performing polling in the workspace, so it happens that the workspace has master checked out at the older commit. I'm still unsure exactly why that happens, but some of the repositories appear to never actually check the branch out, so the revParse fails, while others do have it checked out.

            Show
            jekeller Jacob Keller added a comment - - edited It looks like this is performing polling in the workspace, so it happens that the workspace has master checked out at the older commit. I'm still unsure exactly why that happens, but some of the repositories appear to never actually check the branch out, so the revParse fails, while others do have it checked out.
            jekeller Jacob Keller made changes -
            Field Original Value New Value
            Environment Git Plugin 3.3.2
            Jenkins 2.60.1
            Hide
            jekeller Jacob Keller added a comment -

            This only happens if the job is setup to perform polling in the workspace. If it doesn't use the workspace polling, then it does not have this issue.

            Show
            jekeller Jacob Keller added a comment - This only happens if the job is setup to perform polling in the workspace. If it doesn't use the workspace polling, then it does not have this issue.
            Hide
            jekeller Jacob Keller added a comment -

            I made a separate jenkins instance to test this, and it appears by default, the git plugin does not actually checkout the commit and setup the local "master" reference. However, if anything happens to the repository in the workspace to actually get a local ref named master, we'll end up in a loop where polling always returns true.

            Show
            jekeller Jacob Keller added a comment - I made a separate jenkins instance to test this, and it appears by default, the git plugin does not actually checkout the commit and setup the local "master" reference. However, if anything happens to the repository in the workspace to actually get a local ref named master, we'll end up in a loop where polling always returns true.
            Hide
            jekeller Jacob Keller added a comment -

            I'm able to reproduce the issue by simply having the pipeline checkout master at some commit which was not "built", which causes the polling logic to treat it as an available tag. However, when we actually run the build, we'll never checkout that version again, so polling will always return true.

            Show
            jekeller Jacob Keller added a comment - I'm able to reproduce the issue by simply having the pipeline checkout master at some commit which was not "built", which causes the polling logic to treat it as an available tag. However, when we actually run the build, we'll never checkout that version again, so polling will always return true.
            Hide
            padamstx Phil Adams added a comment - - edited

            I'm having the same issue.  My github enterprise repo has two branches - "master" and "0.0.1" - and I have two jenkins jobs defined (one for each branch... i.e. the branch specifier in each job definition is set to "*/<branchname>").  In my github repository, I've configured the webhook url (https://<jenkins-host>/git/notifyCommit?url=<github-repo-url>&branches=master,0.0.1).   When I hit the webhook URL (testing from github), jenkins receives the POST and pokes the correct jobs, but it incorrectly things there are changes on each of the branches, so it triggers a build for each one.

            While trying to get the git push triggering to work with the GitHub plugin, I see the same exact behavior.   A push to one branch triggers both builds to start, even though the push was to only one of the branches.   So it seems like the basic issue here is when the git plugin looks to see if there are changes on a branch, it always thinks there are changes even when no commits have been pushed since the last build.

            Edit: In case it matters, I'm using Jenkins v2.89.2 with the Git Plugin v3.8.0

            Edit: Just wanted clarify a couple things...  I think my scenario is simpler than the one described by the issue originator.   I temporarily changed one of my jobs (for the 0.0.1 branch) to use polling (every 2 minutes for testing purposes) rather than the webhook.   Without any changes being pushed to my 0.0.1 branch, the first time the polling ran it thought there were changes present so it triggered a build.  And subsequent to that, each time the polling is initiated (every 2 minutes), it thinks there is a change and triggers a build.   Is there anything I can do via configuration to fix this?   Also, if there is some additional server logging that I can enable to try to determine why this is broken, I'm happy to do that as well.

            Thanks!

            Show
            padamstx Phil Adams added a comment - - edited I'm having the same issue.  My github enterprise repo has two branches - "master" and "0.0.1" - and I have two jenkins jobs defined (one for each branch... i.e. the branch specifier in each job definition is set to "*/<branchname>").  In my github repository, I've configured the webhook url (https://<jenkins-host>/git/notifyCommit?url=<github-repo-url>&branches=master,0.0.1).   When I hit the webhook URL (testing from github), jenkins receives the POST and pokes the correct jobs, but it incorrectly things there are changes on each of the branches, so it triggers a build for each one. While trying to get the git push triggering to work with the GitHub plugin, I see the same exact behavior.   A push to one branch triggers both builds to start, even though the push was to only one of the branches.   So it seems like the basic issue here is when the git plugin looks to see if there are changes on a branch, it always thinks there are changes even when no commits have been pushed since the last build. Edit: In case it matters, I'm using Jenkins v2.89.2 with the Git Plugin v3.8.0 Edit: Just wanted clarify a couple things...  I think my scenario is simpler than the one described by the issue originator.   I temporarily changed one of my jobs (for the 0.0.1 branch) to use polling (every 2 minutes for testing purposes) rather than the webhook.   Without any changes being pushed to my 0.0.1 branch, the first time the polling ran it thought there were changes present so it triggered a build.  And subsequent to that, each time the polling is initiated (every 2 minutes), it thinks there is a change and triggers a build.   Is there anything I can do via configuration to fix this?   Also, if there is some additional server logging that I can enable to try to determine why this is broken, I'm happy to do that as well. Thanks!
            Hide
            padamstx Phil Adams added a comment - - edited

            I just thought to check the "Polling log" section of my build results and for the offending job (which builds my 0.0.1 branch), I see this:

            ----------------------
            Started on Apr 6, 2018 5:02:00 PM Workspace is offline.
            Scheduling a new build to get a workspace. (nonexisting_workspace)
            Done. Took 0 ms
            Changes found

            ----------------------

            So... it looks like the job is triggered each time because the jenkins server can't find an existing workspace where a previous build has been performed.  Am I understanding that correctly?

            Is there no way to detect changes on a branch without having an existing workspace with the branch checked out?   In my case, our jenkins server assigns build jobs to agents running in a docker swarm, where the agent is effectively wiped clean after every build, hence a subsequent build will be done on a completely clean randomly-assigned agent.

            Show
            padamstx Phil Adams added a comment - - edited I just thought to check the "Polling log" section of my build results and for the offending job (which builds my 0.0.1 branch), I see this: ---------------------- Started on Apr 6, 2018 5:02:00 PM Workspace is offline. Scheduling a new build to get a workspace. (nonexisting_workspace) Done. Took 0 ms Changes found ---------------------- So... it looks like the job is triggered each time because the jenkins server can't find an existing workspace where a previous build has been performed.  Am I understanding that correctly? Is there no way to detect changes on a branch without having an existing workspace with the branch checked out?   In my case, our jenkins server assigns build jobs to agents running in a docker swarm, where the agent is effectively wiped clean after every build, hence a subsequent build will be done on a completely clean randomly-assigned agent.
            Hide
            jekeller Jacob Keller added a comment -

            Phil Adams, it could maybe use the remote polling method, which doesn't require the workspace, but that gets turned off for a variety of reasons.

             

            It sounds like a different issue than the one I am describing here. I do have a potential fix posted at https://github.com/jenkinsci/git-plugin/pull/579 and you'd be welcome to try it out and see if it fixes your problem, (though I doubt it would...)

            Show
            jekeller Jacob Keller added a comment - Phil Adams , it could maybe use the remote polling method, which doesn't require the workspace, but that gets turned off for a variety of reasons.   It sounds like a different issue than the one I am describing here. I do have a potential fix posted at https://github.com/jenkinsci/git-plugin/pull/579  and you'd be welcome to try it out and see if it fixes your problem, (though I doubt it would...)
            Hide
            padamstx Phil Adams added a comment -

            Thanks Jacob, do you know how I could tell the git plugin to use the remote polling method instead of requiring a workspace?

            Show
            padamstx Phil Adams added a comment - Thanks Jacob, do you know how I could tell the git plugin to use the remote polling method instead of requiring a workspace?
            Hide
            jekeller Jacob Keller added a comment -

            > Thanks Jacob, do you know how I could tell the git plugin to use the remote polling method instead of requiring a workspace?

            Make sure that you don't use any extensions which enable worktree polling. These include "author in change log", "change log to branch", "require workspace when polling", "Message exclusion", "User exclusion", or "Path restriction"

            I do not understand offhand why "author in change log" and "change log to branch" require this at the moment, but message exclusion, user exclusion, and path restriction require it because these features aren't implemented within the remote polling code. (I suspect it's possible to do so, but no one has done it).

            Additionally, remote polling is not setup to handle more than a single branch, so you need to make sure you only use one branch.

            It's in theory possible to fix these limitations, but I do not offhand know the work involved to do so.

            Show
            jekeller Jacob Keller added a comment - > Thanks Jacob, do you know how I could tell the git plugin to use the remote polling method instead of requiring a workspace? Make sure that you don't use any extensions which enable worktree polling. These include "author in change log", "change log to branch", "require workspace when polling", "Message exclusion", "User exclusion", or "Path restriction" I do not understand offhand why "author in change log" and "change log to branch" require this at the moment, but message exclusion, user exclusion, and path restriction require it because these features aren't implemented within the remote polling code. (I suspect it's possible to do so, but no one has done it). Additionally, remote polling is not setup to handle more than a single branch, so you need to make sure you only use one branch. It's in theory possible to fix these limitations, but I do not offhand know the work involved to do so.
            Hide
            padamstx Phil Adams added a comment -

            Jacob, thanks for the info... it turns out that I did in fact have the "author in change log" option selected, and that was why workspace polling was being used.   By disabling that option, the plugin started doing remote polling.

            Show
            padamstx Phil Adams added a comment - Jacob, thanks for the info... it turns out that I did in fact have the "author in change log" option selected, and that was why workspace polling was being used.   By disabling that option, the plugin started doing remote polling.
            Hide
            jekeller Jacob Keller added a comment -

            I submitted a pull request https://github.com/jenkinsci/git-plugin/pull/583 which should remove that restriction (as far as I can tell, it was inadvertently added and there's no reason that the extension "author in change log" depends on the workspace polling.

            Show
            jekeller Jacob Keller added a comment - I submitted a pull request https://github.com/jenkinsci/git-plugin/pull/583 which should remove that restriction (as far as I can tell, it was inadvertently added and there's no reason that the extension "author in change log" depends on the workspace polling.
            markewaite Mark Waite made changes -
            Description I have a repository which is setup to perform git polling, and it always triggers, suggesting there are changes, even though there are not.

            In this case, the job has multiple checkouts using a pipeline, but I think it would only happen in the case of a single checkout as well.

            Specifically, the branch name is master, and it checks first all the "remote/branch" combinations, (in this case, "origin/master") and then fails to find anything new (as origin/master was already built). Then, it checks the branch name on its own, thinking that maybe it's a tag or some other ref.

            This results in reporting a separate commit (likely whatever master happens to be at when the polling was created), and thus reports a new commit to build. This causes the build to trigger every polling timeout, even though there aren't any real new changes.

            Here's a subset of the actual output with repository names and paths redacted. I have the VERBOSE logging enabled for more information.

            {quote}
            Started on Apr 3, 2018 4:01:00 PM
            Using strategy: Default
            [poll] Last Built Revision: Revision 72c4cea985bb5ff14813d27dac94f34887cbd841 (refs/remotes/origin/master)
            using GIT_SSH to set credentials
             > git ls-remote -h <groovyLibURL> # timeout=10
            Found 1 remote heads on <groovyLibURL>
            [poll] Latest remote head revision on refs/heads/master is: 72c4cea985bb5ff14813d27dac94f34887cbd841 - already built by 1738
            Using strategy: Default
            [poll] Last Built Revision: Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <mainRepositoryURL> # timeout=10
            Fetching upstream changes from <mainRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <mainRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@3fcd3164[scmName=<mainRepositoryName>,remoteUrls=[<mainRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)},lastBuild=Build #1738 of Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[cc24044526e9d460c74a59fcdeb12e43a446cb7d]
            AnyObjectId[cc24044526e9d460c74a59fcdeb12e43a446cb7d] has already been built
             > git rev-parse master^{commit} # timeout=10
            Failed to rev-parse: master
            Using strategy: Default
            [poll] Last Built Revision: Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <firstRepositoryURL> # timeout=10
            Fetching upstream changes from <firstRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <firstRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@857d3e89[scmName=<firstRepositoryName>,remoteUrls=[<firstRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)},lastBuild=Build #1738 of Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc]
            AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc] has already been built
             > git rev-parse master^{commit} # timeout=10
            rev-parse master -> AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc]
            AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc] has already been built
            Using strategy: Default
            [poll] Last Built Revision: Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <secondRepositoryURL> # timeout=10
            Fetching upstream changes from <secondRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <secondRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@2e191547[scmName=<secondRepositoryName>,remoteUrls=[<secondRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)},lastBuild=Build #1738 of Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[93a6d3510777096a6fbd7f16ac22c28ef4bd872e]
            AnyObjectId[93a6d3510777096a6fbd7f16ac22c28ef4bd872e] has already been built
             > git rev-parse master^{commit} # timeout=10
            rev-parse master -> AnyObjectId[4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8]
            Found a new commit AnyObjectId[4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8] to be built on master
            {0} seems to be a non-branch reference (tag?)
             > git log --full-history --no-abbrev --format=raw -M -m 93a6d3510777096a6fbd7f16ac22c28ef4bd872e..4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8 # timeout=10
            Using strategy: Default
            [poll] Last Built Revision: Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <thirdRepositoryURL> # timeout=10
            Fetching upstream changes from <thirdRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <thirdRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@49f3b5e7[scmName=<thirdRepositoryName>,remoteUrls=[<thirdRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)},lastBuild=Build #1738 of Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[a4db49e3433ef3304c7956e81369c2ed5fa69b5d]
            AnyObjectId[a4db49e3433ef3304c7956e81369c2ed5fa69b5d] has already been built
             > git rev-parse master^{commit} # timeout=10
            rev-parse master -> AnyObjectId[00f059f63e272c44fb3f754be4e2f4a1b474c416]
            Found a new commit AnyObjectId[00f059f63e272c44fb3f754be4e2f4a1b474c416] to be built on master
            {0} seems to be a non-branch reference (tag?)
             > git log --full-history --no-abbrev --format=raw -M -m a4db49e3433ef3304c7956e81369c2ed5fa69b5d..00f059f63e272c44fb3f754be4e2f4a1b474c416 # timeout=10
            Using strategy: Default
            [poll] Last Built Revision: Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <fourthRepositoryURL> # timeout=10
            Fetching upstream changes from <fourthRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <fourthRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@dff532d1[scmName=<fourthRepositoryName>,remoteUrls=[<fourthRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)},lastBuild=Build #1738 of Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[06188df26697ef38293a8533ce601934762c7d6c]
            AnyObjectId[06188df26697ef38293a8533ce601934762c7d6c] has already been built
             > git rev-parse master^{commit} # timeout=10
            Failed to rev-parse: master
            Done. Took 6.5 sec
            Changes found
            {quote}

            I am highly suspect that the relevant commit related to why this breaks is:

            c7dae80b85c1 ("[JENKINS-21952] Resolve tags with slashes", 2014-02-27)

            I believe what is happening is that the "remote/branch" output is failing to find any new changes, so we fall back to attempting to resolve the name like a tag, but it so *happens* that the polling setup actually does end up with the branch checked out, and it results in us finding the local master name and reporting it.

            I don't really know how to resolve this yet.
            I have a repository which is setup to perform git polling, and it always triggers, suggesting there are changes, even though there are not.

            In this case, the job has multiple checkouts using a pipeline, but I think it would only happen in the case of a single checkout as well.

            Specifically, the branch name is master, and it checks first all the "remote/branch" combinations, (in this case, "origin/master") and then fails to find anything new (as origin/master was already built). Then, it checks the branch name on its own, thinking that maybe it's a tag or some other ref.

            This results in reporting a separate commit (likely whatever master happens to be at when the polling was created), and thus reports a new commit to build. This causes the build to trigger every polling timeout, even though there aren't any real new changes.

            Here's a subset of the actual output with repository names and paths redacted. I have the VERBOSE logging enabled for more information.

            {noformat}
            Started on Apr 3, 2018 4:01:00 PM
            Using strategy: Default
            [poll] Last Built Revision: Revision 72c4cea985bb5ff14813d27dac94f34887cbd841 (refs/remotes/origin/master)
            using GIT_SSH to set credentials
             > git ls-remote -h <groovyLibURL> # timeout=10
            Found 1 remote heads on <groovyLibURL>
            [poll] Latest remote head revision on refs/heads/master is: 72c4cea985bb5ff14813d27dac94f34887cbd841 - already built by 1738
            Using strategy: Default
            [poll] Last Built Revision: Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <mainRepositoryURL> # timeout=10
            Fetching upstream changes from <mainRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <mainRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@3fcd3164[scmName=<mainRepositoryName>,remoteUrls=[<mainRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)},lastBuild=Build #1738 of Revision cc24044526e9d460c74a59fcdeb12e43a446cb7d (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[cc24044526e9d460c74a59fcdeb12e43a446cb7d]
            AnyObjectId[cc24044526e9d460c74a59fcdeb12e43a446cb7d] has already been built
             > git rev-parse master^{commit} # timeout=10
            Failed to rev-parse: master
            Using strategy: Default
            [poll] Last Built Revision: Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <firstRepositoryURL> # timeout=10
            Fetching upstream changes from <firstRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <firstRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@857d3e89[scmName=<firstRepositoryName>,remoteUrls=[<firstRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)},lastBuild=Build #1738 of Revision d24e49e865522042434be7b1a46f592aae2608dc (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc]
            AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc] has already been built
             > git rev-parse master^{commit} # timeout=10
            rev-parse master -> AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc]
            AnyObjectId[d24e49e865522042434be7b1a46f592aae2608dc] has already been built
            Using strategy: Default
            [poll] Last Built Revision: Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <secondRepositoryURL> # timeout=10
            Fetching upstream changes from <secondRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <secondRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@2e191547[scmName=<secondRepositoryName>,remoteUrls=[<secondRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)},lastBuild=Build #1738 of Revision 93a6d3510777096a6fbd7f16ac22c28ef4bd872e (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[93a6d3510777096a6fbd7f16ac22c28ef4bd872e]
            AnyObjectId[93a6d3510777096a6fbd7f16ac22c28ef4bd872e] has already been built
             > git rev-parse master^{commit} # timeout=10
            rev-parse master -> AnyObjectId[4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8]
            Found a new commit AnyObjectId[4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8] to be built on master
            {0} seems to be a non-branch reference (tag?)
             > git log --full-history --no-abbrev --format=raw -M -m 93a6d3510777096a6fbd7f16ac22c28ef4bd872e..4bf94e5146026ba4e3bebef3b2ebf8a795d0d0f8 # timeout=10
            Using strategy: Default
            [poll] Last Built Revision: Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <thirdRepositoryURL> # timeout=10
            Fetching upstream changes from <thirdRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <thirdRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@49f3b5e7[scmName=<thirdRepositoryName>,remoteUrls=[<thirdRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)},lastBuild=Build #1738 of Revision a4db49e3433ef3304c7956e81369c2ed5fa69b5d (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[a4db49e3433ef3304c7956e81369c2ed5fa69b5d]
            AnyObjectId[a4db49e3433ef3304c7956e81369c2ed5fa69b5d] has already been built
             > git rev-parse master^{commit} # timeout=10
            rev-parse master -> AnyObjectId[00f059f63e272c44fb3f754be4e2f4a1b474c416]
            Found a new commit AnyObjectId[00f059f63e272c44fb3f754be4e2f4a1b474c416] to be built on master
            {0} seems to be a non-branch reference (tag?)
             > git log --full-history --no-abbrev --format=raw -M -m a4db49e3433ef3304c7956e81369c2ed5fa69b5d..00f059f63e272c44fb3f754be4e2f4a1b474c416 # timeout=10
            Using strategy: Default
            [poll] Last Built Revision: Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)
             > git rev-parse --is-inside-work-tree # timeout=10
            Fetching changes from the remote Git repositories
             > git config remote.origin.url <fourthRepositoryURL> # timeout=10
            Fetching upstream changes from <fourthRepositoryURL>
             > git --version # timeout=10
            using GIT_SSH to set credentials
             > git fetch --no-tags --progress <fourthRepositoryURL> +refs/heads/*:refs/remotes/origin/*
            Polling for changes in
            getCandidateRevisions(true,master,,,hudson.plugins.git.util.BuildData@dff532d1[scmName=<fourthRepositoryName>,remoteUrls=[<fourthRepositoryURL>],buildsByBranchName={origin/master=Build #1738 of Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)},lastBuild=Build #1738 of Revision 06188df26697ef38293a8533ce601934762c7d6c (origin/master)]) considering branches to build
            Qualifying master as a branch in repository origin -> origin/master
             > git rev-parse origin/master^{commit} # timeout=10
            rev-parse origin/master -> AnyObjectId[06188df26697ef38293a8533ce601934762c7d6c]
            AnyObjectId[06188df26697ef38293a8533ce601934762c7d6c] has already been built
             > git rev-parse master^{commit} # timeout=10
            Failed to rev-parse: master
            Done. Took 6.5 sec
            Changes found
            {noformat}

            I am highly suspect that the relevant commit related to why this breaks is:

            c7dae80b85c1 ("[JENKINS-21952] Resolve tags with slashes", 2014-02-27)

            I believe what is happening is that the "remote/branch" output is failing to find any new changes, so we fall back to attempting to resolve the name like a tag, but it so *happens* that the polling setup actually does end up with the branch checked out, and it results in us finding the local master name and reporting it.

            I don't really know how to resolve this yet.
            jekeller Jacob Keller made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jekeller Jacob Keller made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            Hide
            jekeller Jacob Keller added a comment -

            I believe I fixed this with the following pull request: https://github.com/jenkinsci/git-plugin/pull/579

            Show
            jekeller Jacob Keller added a comment - I believe I fixed this with the following pull request: https://github.com/jenkinsci/git-plugin/pull/579
            jekeller Jacob Keller made changes -
            Status In Review [ 10005 ] In Progress [ 3 ]
            Hide
            markewaite Mark Waite added a comment -

            I've been evaluating this pull request today. Evaluation is not complete, but it is looking good. I'd like to include this in the 4.0.0-beta1 release. However, I first need to merge several other changes (the require-jdk-8 PR and a few others that I've already evaluated).

            I need some help Raul Arabaolaza to understand the test failures on the pull request. I'll connect with him early next week for further discussion.

            Show
            markewaite Mark Waite added a comment - I've been evaluating this pull request today. Evaluation is not complete, but it is looking good. I'd like to include this in the 4.0.0-beta1 release. However, I first need to merge several other changes (the require-jdk-8 PR and a few others that I've already evaluated). I need some help Raul Arabaolaza to understand the test failures on the pull request. I'll connect with him early next week for further discussion.
            Hide
            jekeller Jacob Keller added a comment -

            I've had this pull request running as well in my test instance that I setup to test the thousands of branches case with the build details pull request as well, so far I haven't found any issues either.

            I'm curious what the ATH thing is as well. Hopefully we can get to the bottom of that one.

            Show
            jekeller Jacob Keller added a comment - I've had this pull request running as well in my test instance that I setup to test the thousands of branches case with the build details pull request as well, so far I haven't found any issues either. I'm curious what the ATH thing is as well. Hopefully we can get to the bottom of that one.
            markewaite Mark Waite made changes -
            Link This issue relates to JENKINS-50401 [ JENKINS-50401 ]

              People

              • Assignee:
                markewaite Mark Waite
                Reporter:
                jekeller Jacob Keller
              • Votes:
                2 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: