Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10131

Git polling shouldn't need a workspace on a slave.

    Details

    • Type: Improvement
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: git-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      What happened:

      I have two slaves (1 & 2). I took slave 1 down for maintenance and a bunch of old, rarely updated, git builds kicked off. When I checked the git polling log, I saw a message (it's gone now, darn it) that said it was rebuilding to get workspace for git polling.

      A workspace shouldn't be needed.

      I'm unclear what git needs for this, but if you're tracking only a single branch (master) then you just need the HEAD and can compare the SHA1s.

      If it technically really really needs a git checkout, then I'd prefer if they were kept on an assigned host (the jenkins server in my case, master) instead of using the workspaces for this check. I'd want this for a couple reasons:

      • Slaves come and go. Rebuilding all my projects because a slave went down is unproductive.
      • Occasionally, I need to go into a slave and monkey with a workspace to troubleshoot a weird problem. I don't want that to impact polling.
      • It's a waste of space on the slaves. I'd rather control where the space is wasted.

      Ciao!

        Attachments

          Issue Links

            Activity

            docwhat Christian Höltje created issue -
            Hide
            docwhat Christian Höltje added a comment -

            Okay! I just figured out this bug is what causes massive amounts of builds to kick off whenever a slave goes down. It is also why all builds kick off when I restart Jenkins!

            Lets assume we have a set of jobs, J, that require slaves of label L. If all slaves of label L go down, then there are no workspaces available for J, therefore the git plugin kicks off builds for all jobs J to get a workspace.

            In addition, it seems that sometimes on restart the git plugin kicks off before the slaves come online causing all jobs on the system to kick off!

            Finally, this also breaks the "only bring up slave when needed" functionality. I imagine it breaks the "provision a slave automatically" feature as well.

            This really needs to be fixed. It means any maintenance task renders the build system busy for hours (we have 100s of builds, some of which are very long). It also needlessly bumps version numbers for no obvious reason which causes confusion.

            Show
            docwhat Christian Höltje added a comment - Okay! I just figured out this bug is what causes massive amounts of builds to kick off whenever a slave goes down. It is also why all builds kick off when I restart Jenkins! Lets assume we have a set of jobs, J, that require slaves of label L. If all slaves of label L go down, then there are no workspaces available for J, therefore the git plugin kicks off builds for all jobs J to get a workspace. In addition, it seems that sometimes on restart the git plugin kicks off before the slaves come online causing all jobs on the system to kick off! Finally, this also breaks the "only bring up slave when needed" functionality. I imagine it breaks the "provision a slave automatically" feature as well. This really needs to be fixed. It means any maintenance task renders the build system busy for hours (we have 100s of builds, some of which are very long). It also needlessly bumps version numbers for no obvious reason which causes confusion.
            Hide
            spyder Andrew Herron added a comment -

            I've been experimenting with a master/slave Jenkins setup to combine our 4 existing Hudson servers into one, but this issue is going to make it untenable. I don't have an easy way to count but there'd easily be more than 50 builds.

            In my case, the Jenkins master has 0 executors. Any git project, no matter what the slave label target is, hits this problem on my server.

            Interestingly the projects still in SVN don't have this issue; they appear on the build executor list to poll, but when there are no changes the build doesn't trigger. Looks like the git plugin doesn't check for changes until after the build has begun, meaning it would have to cancel if there are no changes.

            Show
            spyder Andrew Herron added a comment - I've been experimenting with a master/slave Jenkins setup to combine our 4 existing Hudson servers into one, but this issue is going to make it untenable. I don't have an easy way to count but there'd easily be more than 50 builds. In my case, the Jenkins master has 0 executors. Any git project, no matter what the slave label target is, hits this problem on my server. Interestingly the projects still in SVN don't have this issue; they appear on the build executor list to poll, but when there are no changes the build doesn't trigger. Looks like the git plugin doesn't check for changes until after the build has begun, meaning it would have to cancel if there are no changes.
            Hide
            ndeloof Nicolas De Loof added a comment -

            Partially fixed by https://github.com/jenkinsci/git-plugin/commit/c750b63ecd527a5721d9d8ee426bff97dd0e7a75
            using git ls-remote thanks to onemanbucket contribution
            (only supports simple cases : single branch, single repo)

            Show
            ndeloof Nicolas De Loof added a comment - Partially fixed by https://github.com/jenkinsci/git-plugin/commit/c750b63ecd527a5721d9d8ee426bff97dd0e7a75 using git ls-remote thanks to onemanbucket contribution (only supports simple cases : single branch, single repo)
            Hide
            spyder Andrew Herron added a comment -

            Cool!

            I thought the git plugin only supported single branch single repo cases in general though

            Show
            spyder Andrew Herron added a comment - Cool! I thought the git plugin only supported single branch single repo cases in general though
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            No, it lets you specify a wildcard to pick branches to build.

            In a few releases, we should get rid of the checkbox, too. Excerpt from the IRC log:

            (02:31:03 PM) kohsuke: Any reason this needs to be a checkbox?
            (02:31:16 PM) One-Man-Bucket: i was thinking it could be the default behaviour in the simple use case
            (02:31:18 PM) abayer: kohsuke: I'd like it to not be the default behavior quite yet.
            (02:31:30 PM) kohsuke: I mean, shouldn't it just work automatically?
            (02:31:57 PM) abayer: Actually, I'd be good with having the readResolve make it off and the default for new projects to be on.
            (02:32:05 PM) One-Man-Bucket: as i wrote int he pull request, jgit has ls-remote since a couple of months
            (02:32:22 PM) kohsuke: I think we already have enough switches and knobs in Git configuration
            (02:32:28 PM) kohsuke: IMHO, that is.
            (02:32:45 PM) abayer: One-Man-Bucket: I'm actually meeting with ksawicki of github tomorrow to start talking about gutting/rewriting the git plugin from the ground up, to mainly use jgit among other things. =)
            (02:32:59 PM) One-Man-Bucket: abayer: oh, cool
            (02:33:41 PM) abayer: I'm just wary of potential regressions in upgrades due to having introduced gobs of them myself. =)
            (02:34:13 PM) One-Man-Bucket: abayer: :)
            (02:34:42 PM) kohsuke: OK, I suppose you can take away the checkbox later.
            (02:35:04 PM) One-Man-Bucket: one perk with the checkbox is that you get feedback if the configuration won't support remote poll
            (02:35:14 PM) One-Man-Bucket: which is important for our usecase
            (02:37:06 PM) kohsuke: One-Man-Bucket: by "it just works" I mean switching to ls-remote when and only when you can.
            (02:37:39 PM) kohsuke: IIUC, provided that it works, there's no downside in using ls-remote.
            (02:37:57 PM) One-Man-Bucket: kohsuke: fair enough
            (02:38:38 PM) kohsuke: And also, IIUC, there's no inherent limitation in git ls-remote that you can't support multiple branches, right?
            (02:38:49 PM) kohsuke: It's just that we haven't implemented that yet.
            (02:39:13 PM) One-Man-Bucket: right, it should work for multiple branches as well
            (02:39:37 PM) kohsuke: Anyway, I guess we aren't eliminating options --- as long as folks are willing to drop this checkbox at a later point, I'm happy.
            (02:39:39 PM) abayer: Would it work with branch wildcards?
            (02:39:58 PM) kohsuke: Yeah, you just list all the remote branches and check against the wildcard you got
            (02:40:12 PM) abayer: kohsuke: That's fine by me. One release to make sure it doesn't barf, and in the meantime have an escape hatch.
            (02:40:26 PM) kohsuke: Yes.
            (02:40:57 PM) kohsuke: ("eliminating options" is kind of funny here because we are trying not to eliminate the option of eliminating the option (= the checkbox)
            
            Show
            kohsuke Kohsuke Kawaguchi added a comment - No, it lets you specify a wildcard to pick branches to build. In a few releases, we should get rid of the checkbox, too. Excerpt from the IRC log: (02:31:03 PM) kohsuke: Any reason this needs to be a checkbox? (02:31:16 PM) One-Man-Bucket: i was thinking it could be the default behaviour in the simple use case (02:31:18 PM) abayer: kohsuke: I'd like it to not be the default behavior quite yet. (02:31:30 PM) kohsuke: I mean, shouldn't it just work automatically? (02:31:57 PM) abayer: Actually, I'd be good with having the readResolve make it off and the default for new projects to be on. (02:32:05 PM) One-Man-Bucket: as i wrote int he pull request, jgit has ls-remote since a couple of months (02:32:22 PM) kohsuke: I think we already have enough switches and knobs in Git configuration (02:32:28 PM) kohsuke: IMHO, that is. (02:32:45 PM) abayer: One-Man-Bucket: I'm actually meeting with ksawicki of github tomorrow to start talking about gutting/rewriting the git plugin from the ground up, to mainly use jgit among other things. =) (02:32:59 PM) One-Man-Bucket: abayer: oh, cool (02:33:41 PM) abayer: I'm just wary of potential regressions in upgrades due to having introduced gobs of them myself. =) (02:34:13 PM) One-Man-Bucket: abayer: :) (02:34:42 PM) kohsuke: OK, I suppose you can take away the checkbox later. (02:35:04 PM) One-Man-Bucket: one perk with the checkbox is that you get feedback if the configuration won't support remote poll (02:35:14 PM) One-Man-Bucket: which is important for our usecase (02:37:06 PM) kohsuke: One-Man-Bucket: by "it just works" I mean switching to ls-remote when and only when you can. (02:37:39 PM) kohsuke: IIUC, provided that it works, there's no downside in using ls-remote. (02:37:57 PM) One-Man-Bucket: kohsuke: fair enough (02:38:38 PM) kohsuke: And also, IIUC, there's no inherent limitation in git ls-remote that you can't support multiple branches, right? (02:38:49 PM) kohsuke: It's just that we haven't implemented that yet. (02:39:13 PM) One-Man-Bucket: right, it should work for multiple branches as well (02:39:37 PM) kohsuke: Anyway, I guess we aren't eliminating options --- as long as folks are willing to drop this checkbox at a later point, I'm happy. (02:39:39 PM) abayer: Would it work with branch wildcards? (02:39:58 PM) kohsuke: Yeah, you just list all the remote branches and check against the wildcard you got (02:40:12 PM) abayer: kohsuke: That's fine by me. One release to make sure it doesn't barf, and in the meantime have an escape hatch. (02:40:26 PM) kohsuke: Yes. (02:40:57 PM) kohsuke: ("eliminating options" is kind of funny here because we are trying not to eliminate the option of eliminating the option (= the checkbox)
            Hide
            spyder Andrew Herron added a comment -

            I've just updated to the new version of the git plugin, and I can confirm the fix works. Shut down all of my slaves and there are no jobs queued

            I really would've preferred it to be on by default, but we have our job configs in SCM so it was a simple search/replace to turn it on for every build.

            Show
            spyder Andrew Herron added a comment - I've just updated to the new version of the git plugin, and I can confirm the fix works. Shut down all of my slaves and there are no jobs queued I really would've preferred it to be on by default, but we have our job configs in SCM so it was a simple search/replace to turn it on for every build.
            Hide
            spyder Andrew Herron added a comment -

            I'm seeing some strange errors in my SCM polling logs now... These errors don't show in the UI, and don't seem to impact builds, but they're still concerning.

            I have 3 long builds showing an error while they are building - this doesn't happen on other active builds though:

            (server name removed)

            [poll] Last Built Revision: Revision a03190f9886de492a997a72c6525d65424618641 (origin/master)
            ERROR: Failed to record SCM polling
            hudson.plugins.git.GitException: Error performing command: git ls-remote -h git@<server> master
            Command "git ls-remote -h git@<server> master" returned status code 128: error: cannot run ssh: No such file or directory
            fatal: unable to fork
            
            	at hudson.plugins.git.GitAPI.launchCommandIn(GitAPI.java:764)
            	at hudson.plugins.git.GitAPI.launchCommand(GitAPI.java:729)
            	at hudson.plugins.git.GitAPI.getHeadRev(GitAPI.java:1043)
            	at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:624)
            	at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:355)
            	at hudson.scm.SCM.poll(SCM.java:372)
            	at hudson.model.AbstractProject.poll(AbstractProject.java:1324)
            	at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:420)
            	at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:449)
            	at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
            	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
            	at java.lang.Thread.run(Thread.java:636)
            Caused by: hudson.plugins.git.GitException: Command "git ls-remote -h git@<server> master" returned status code 128: error: cannot run ssh: No such file or directory
            fatal: unable to fork
            
            	at hudson.plugins.git.GitAPI.launchCommandIn(GitAPI.java:759)
            	... 15 more
            

            Builds that used the ** branch specifier (we want our builds to all be master-only, this has exposed a few that aren't) have this error in their polling log:

            Started on 09/09/2011 5:10:18 PM
            Using strategy: Default
            [poll] Last Build : #38
            [poll] Last Built Revision: Revision b9af602f4250b74fbf026a92716083d9b294121e (origin/master)
            ERROR: Failed to record SCM polling
            java.lang.NullPointerException
            	at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:657)
            	at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:355)
            	at hudson.scm.SCM.poll(SCM.java:372)
            	at hudson.model.AbstractProject.poll(AbstractProject.java:1324)
            	at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:420)
            	at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:449)
            	at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
            	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
            	at java.lang.Thread.run(Thread.java:636)
            

            Happy to help track these down if you need me to.

            Show
            spyder Andrew Herron added a comment - I'm seeing some strange errors in my SCM polling logs now... These errors don't show in the UI, and don't seem to impact builds, but they're still concerning. I have 3 long builds showing an error while they are building - this doesn't happen on other active builds though: (server name removed) [poll] Last Built Revision: Revision a03190f9886de492a997a72c6525d65424618641 (origin/master) ERROR: Failed to record SCM polling hudson.plugins.git.GitException: Error performing command: git ls-remote -h git@<server> master Command "git ls-remote -h git@<server> master" returned status code 128: error: cannot run ssh: No such file or directory fatal: unable to fork at hudson.plugins.git.GitAPI.launchCommandIn(GitAPI.java:764) at hudson.plugins.git.GitAPI.launchCommand(GitAPI.java:729) at hudson.plugins.git.GitAPI.getHeadRev(GitAPI.java:1043) at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:624) at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:355) at hudson.scm.SCM.poll(SCM.java:372) at hudson.model.AbstractProject.poll(AbstractProject.java:1324) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:420) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:449) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang. Thread .run( Thread .java:636) Caused by: hudson.plugins.git.GitException: Command "git ls-remote -h git@<server> master" returned status code 128: error: cannot run ssh: No such file or directory fatal: unable to fork at hudson.plugins.git.GitAPI.launchCommandIn(GitAPI.java:759) ... 15 more Builds that used the ** branch specifier (we want our builds to all be master-only, this has exposed a few that aren't) have this error in their polling log: Started on 09/09/2011 5:10:18 PM Using strategy: Default [poll] Last Build : #38 [poll] Last Built Revision: Revision b9af602f4250b74fbf026a92716083d9b294121e (origin/master) ERROR: Failed to record SCM polling java.lang.NullPointerException at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:657) at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:355) at hudson.scm.SCM.poll(SCM.java:372) at hudson.model.AbstractProject.poll(AbstractProject.java:1324) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:420) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:449) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang. Thread .run( Thread .java:636) Happy to help track these down if you need me to.
            Hide
            spyder Andrew Herron added a comment - - edited

            ok, "ssh: no such file or directory" isn't related to build length... still happening after the builds finished.

            Looks like it's related to the slave node OS; it's happening on all builds that are set to only use our windows slaves. Turning off the fast remote polling for those builds stops the error.

            Show
            spyder Andrew Herron added a comment - - edited ok, "ssh: no such file or directory" isn't related to build length... still happening after the builds finished. Looks like it's related to the slave node OS; it's happening on all builds that are set to only use our windows slaves. Turning off the fast remote polling for those builds stops the error.
            Hide
            ustuehler Uwe Stuehler added a comment -

            I think this issue is related to JENKINS-9596, at least on a high level.

            Show
            ustuehler Uwe Stuehler added a comment - I think this issue is related to JENKINS-9596 , at least on a high level.
            ustuehler Uwe Stuehler made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-9596 [ JENKINS-9596 ]
            Hide
            spyder Andrew Herron added a comment -

            I just updated to Jenkins 1.447 with git plugin 1.1.15 and decided to try fast polling on the windows builds again. It works now

            Show
            spyder Andrew Herron added a comment - I just updated to Jenkins 1.447 with git plugin 1.1.15 and decided to try fast polling on the windows builds again. It works now
            Hide
            marc_guenther Marc Günther added a comment -

            The second exception in Andrews comment from 09/Sep/11 is caused by JENKINS-10880

            Show
            marc_guenther Marc Günther added a comment - The second exception in Andrews comment from 09/Sep/11 is caused by JENKINS-10880
            marc_guenther Marc Günther made changes -
            Link This issue is related to JENKINS-10880 [ JENKINS-10880 ]
            Hide
            jhansche Joe Hansche added a comment - - edited

            Nicolas De Loof: What is the reason behind restricting the option to the simple case of single repo, single branch? git ls-remote can retrieve the full set of branches, and revisions for each, and then compare those branch revisions against the "last built revision" for each branch it is configured for. If any of them are different, then a build would be scheduled – this is no different than what already happens with workspace polling, right?

            Another thing I don't understand, is how does it determine that it's not a single repo/single branch? That is, I'm using the multiple-scms plugin, and apparently that is causing the ls-remote option to be disabled automatically. But each instance of the GitSCM in my project contains only a single repo and a single branch, so why would that be failing? Multiple-SCMs should be able to isolate each of those separately, without considering it any more complex than a single repo, single branch.

            This behavior is a major problem for us, as we have the same problem that Christian reports in the first comment. We try to keep our master server free of builds, so that it can focus on its main task of "being the master." That means on restart, all of our jobs (well over 100) get scheduled immediately on startup, even though the slaves are actually coming online only seconds later. That is a huge waste of resources, and at times it actually causes the entire cluster to become unstable – simply as a result of starting up!

            I'll look at modifying the ls-remote patch to see if I can get it to work with multiple branches, and also to find out why it behaves like this in the first place (since at least in my case, I actually don't have multiple branches). I just wanted to see if anyone else knows of some reason why what I said above could not work?

            EDIT: bah, nevermind :/ I just realized what was causing it now... I was so focused on the "single branch" criteria, that I didn't think to look at excluded regions and users, which many of my projects use to prevent re-triggering a build when a Jenkins job is responsible for auto publishing commits (publishing things like minified JavaScript and CSS, and newly compiled Flash (swf) files). Removed those exclusions, and it works. I agree with the other feature request though, that this should be caught via the frontend before the user even clicks save, so that at least he knows why it won't work.

            Show
            jhansche Joe Hansche added a comment - - edited Nicolas De Loof : What is the reason behind restricting the option to the simple case of single repo, single branch? git ls-remote can retrieve the full set of branches, and revisions for each, and then compare those branch revisions against the "last built revision" for each branch it is configured for. If any of them are different, then a build would be scheduled – this is no different than what already happens with workspace polling, right? Another thing I don't understand, is how does it determine that it's not a single repo/single branch? That is, I'm using the multiple-scms plugin, and apparently that is causing the ls-remote option to be disabled automatically. But each instance of the GitSCM in my project contains only a single repo and a single branch, so why would that be failing? Multiple-SCMs should be able to isolate each of those separately, without considering it any more complex than a single repo, single branch. This behavior is a major problem for us, as we have the same problem that Christian reports in the first comment. We try to keep our master server free of builds, so that it can focus on its main task of "being the master." That means on restart, all of our jobs (well over 100) get scheduled immediately on startup, even though the slaves are actually coming online only seconds later. That is a huge waste of resources, and at times it actually causes the entire cluster to become unstable – simply as a result of starting up! I'll look at modifying the ls-remote patch to see if I can get it to work with multiple branches, and also to find out why it behaves like this in the first place (since at least in my case, I actually don't have multiple branches). I just wanted to see if anyone else knows of some reason why what I said above could not work? EDIT: bah, nevermind :/ I just realized what was causing it now... I was so focused on the "single branch" criteria, that I didn't think to look at excluded regions and users, which many of my projects use to prevent re-triggering a build when a Jenkins job is responsible for auto publishing commits (publishing things like minified JavaScript and CSS, and newly compiled Flash (swf) files). Removed those exclusions, and it works. I agree with the other feature request though, that this should be caught via the frontend before the user even clicks save, so that at least he knows why it won't work.
            Hide
            wgracelee wgracelee added a comment -

            We're seeing this message in (git) polling log, and we're using git plugin 1.1.23.

            Polling Log
            View as plain text

            This page captures the polling log that triggered this build.

            Started on Oct 15, 2012 11:24:38 PM
            Workspace is offline.
            Scheduling a new build to get a workspace.
            Done. Took 4 ms
            Changes found

            Show
            wgracelee wgracelee added a comment - We're seeing this message in (git) polling log, and we're using git plugin 1.1.23. Polling Log View as plain text This page captures the polling log that triggered this build. Started on Oct 15, 2012 11:24:38 PM Workspace is offline. Scheduling a new build to get a workspace. Done. Took 4 ms Changes found
            ringerc Craig Ringer made changes -
            Link This issue is related to JENKINS-9067 [ JENKINS-9067 ]
            Hide
            inger Gergely Nagy added a comment -

            Seeing this with 1.1.25, with a single repo & single branch (so it should work according to the above).

            Started on 24-Jan-2013 00:24:57
            No workspace is available, so can't check for updates.
            Scheduling a new build to get a workspace.
            Done. Took 1 ms
            Changes found

            Show
            inger Gergely Nagy added a comment - Seeing this with 1.1.25, with a single repo & single branch (so it should work according to the above). Started on 24-Jan-2013 00:24:57 No workspace is available, so can't check for updates. Scheduling a new build to get a workspace. Done. Took 1 ms Changes found
            Hide
            yakobe jake bishop added a comment - - edited

            Any news on this? I can turn on fast remote polling to get around this. But that is not really a solution because then I can only use one branch (without wildcards).

            Show
            yakobe jake bishop added a comment - - edited Any news on this? I can turn on fast remote polling to get around this. But that is not really a solution because then I can only use one branch (without wildcards).
            Hide
            ndeloof Nicolas De Loof added a comment -

            ls-remote is the default on 2.0 for "single branch" case.
            investigating how to use for general use

            Show
            ndeloof Nicolas De Loof added a comment - ls-remote is the default on 2.0 for "single branch" case. investigating how to use for general use
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Nicolas De Loof
            Path:
            src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
            src/main/java/org/jenkinsci/plugins/gitclient/GitClient.java
            src/main/java/org/jenkinsci/plugins/gitclient/JGitAPIImpl.java
            src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java
            src/test/java/org/jenkinsci/plugins/gitclient/GitAPITestCase.java
            http://jenkins-ci.org/commit/git-client-plugin/649c08617ff01191028e2e744013ed64e8a14431
            Log:
            JENKINS-10131 ls-remote for general usage to lookup all branches

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java src/main/java/org/jenkinsci/plugins/gitclient/GitClient.java src/main/java/org/jenkinsci/plugins/gitclient/JGitAPIImpl.java src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java src/test/java/org/jenkinsci/plugins/gitclient/GitAPITestCase.java http://jenkins-ci.org/commit/git-client-plugin/649c08617ff01191028e2e744013ed64e8a14431 Log: JENKINS-10131 ls-remote for general usage to lookup all branches
            ndeloof Nicolas De Loof made changes -
            Issue Type Bug [ 1 ] Improvement [ 4 ]
            Hide
            marc_guenther Marc Günther added a comment -

            I don't agree that this is an improvement. It's a genuine bug, and at the moment there is not even a real workaround for it.

            For the multiple branch case, we ended up using a conditional buildstep, which compares GIT_COMMIT and GIT_PREVIOUS_COMMIT. But then you can't do manual builds anymore, because there is no way to find out the build cause inside of a build....

            Show
            marc_guenther Marc Günther added a comment - I don't agree that this is an improvement. It's a genuine bug, and at the moment there is not even a real workaround for it. For the multiple branch case, we ended up using a conditional buildstep, which compares GIT_COMMIT and GIT_PREVIOUS_COMMIT. But then you can't do manual builds anymore, because there is no way to find out the build cause inside of a build....
            ojacobson Owen Jacobson made changes -
            Link This issue is related to JENKINS-20607 [ JENKINS-20607 ]
            Hide
            ojacobson Owen Jacobson added a comment -

            It's Worse Than You Think.

            The ls-remote polling, as written, makes a hard build/no-build decision based on the git ls-remote output only. This interacts ... poorly with the GitSCM plugin features that filter commits (UserExclusion and so on), since they need to look at the individual commits to make their own go/no-go decisions - but that can't be done remotely.

            Forcing workspace polling fixes that (and I've got a related PR open for JENKINS-20607 anyways) but reintroduces the problem Christian et al are complaining about. I suspect the real solution is to always do the polling on the master, possibly outside the workspace, and have the build runs re-fetch and check out the correct commit.

            From working on JENKINS-20607 I'm well aware of how much work that is.

            Show
            ojacobson Owen Jacobson added a comment - It's Worse Than You Think. The ls-remote polling, as written, makes a hard build/no-build decision based on the git ls-remote output only. This interacts ... poorly with the GitSCM plugin features that filter commits ( UserExclusion and so on), since they need to look at the individual commits to make their own go/no-go decisions - but that can't be done remotely. Forcing workspace polling fixes that (and I've got a related PR open for JENKINS-20607 anyways) but reintroduces the problem Christian et al are complaining about. I suspect the real solution is to always do the polling on the master, possibly outside the workspace, and have the build runs re-fetch and check out the correct commit. From working on JENKINS-20607 I'm well aware of how much work that is.
            Hide
            markewaite Mark Waite added a comment -

            The 1.6.2 release of git-client-plugin fixed a bug in ls-remote polling for the command line git implementation. The bug would have manifest itself as having ls-remote based polling show that there were changes when there were no changes.

            Refer to https://github.com/jenkinsci/git-client-plugin/commit/e5e75d5a100905c0d5b332bbde3642e96ffe10a9 for more details on the bug.

            That fix does not change the behavior described by Owen Jacobson. Remote polling does not apply commit filtering, so if you're using inclusion or exclusion filters in your job definition, you can't use remote polling.

            Show
            markewaite Mark Waite added a comment - The 1.6.2 release of git-client-plugin fixed a bug in ls-remote polling for the command line git implementation. The bug would have manifest itself as having ls-remote based polling show that there were changes when there were no changes. Refer to https://github.com/jenkinsci/git-client-plugin/commit/e5e75d5a100905c0d5b332bbde3642e96ffe10a9 for more details on the bug. That fix does not change the behavior described by Owen Jacobson. Remote polling does not apply commit filtering, so if you're using inclusion or exclusion filters in your job definition, you can't use remote polling.
            Hide
            drdt Don Ross added a comment -

            Now that the default behavior of the plug-in is to use remote polling, does this mean we need to not use exclusion filters?

            Show
            drdt Don Ross added a comment - Now that the default behavior of the plug-in is to use remote polling, does this mean we need to not use exclusion filters?
            Hide
            markewaite Mark Waite added a comment - - edited

            Don Ross you may continue to use exclusion filters and inclusion filters, but jobs which use them must be configured to "Force polling using workspace".

            Effectively, you're providing the compatibility with past behavior by forcing polling to use the workspace. Had you enabled fast remote polling with inclusion or exclusion filtering with git plugin releases prior to 2.0, you would have seen the same types of problems as exist today. The remote polling technique does not receive enough information to evaluate inclusion or exclusion filters.

            Show
            markewaite Mark Waite added a comment - - edited Don Ross you may continue to use exclusion filters and inclusion filters, but jobs which use them must be configured to "Force polling using workspace". Effectively, you're providing the compatibility with past behavior by forcing polling to use the workspace. Had you enabled fast remote polling with inclusion or exclusion filtering with git plugin releases prior to 2.0, you would have seen the same types of problems as exist today. The remote polling technique does not receive enough information to evaluate inclusion or exclusion filters.
            Hide
            drdt Don Ross added a comment -

            Thanks, Mark. I actually never got exclusions to work in prior versions, so I wanted to try them out now I upgraded.

            I am having trouble getting forced workspace polling to work at any rate; see my comments on https://issues.jenkins-ci.org/browse/JENKINS-20427.

            Show
            drdt Don Ross added a comment - Thanks, Mark. I actually never got exclusions to work in prior versions, so I wanted to try them out now I upgraded. I am having trouble getting forced workspace polling to work at any rate; see my comments on https://issues.jenkins-ci.org/browse/JENKINS-20427 .
            Hide
            markewaite Mark Waite added a comment -

            As described in the bug report, this is resolved. The git plugin does not require a workspace for polling, and the default (as of git plugin 2.0) is to not use a workspace for polling.

            The problems described by Owen Jacobson (and others) still exist for the somewhat more advanced use cases like excluding users, excluding files, and including files, but the basic issue reported by the bug is fixed.

            Show
            markewaite Mark Waite added a comment - As described in the bug report, this is resolved. The git plugin does not require a workspace for polling, and the default (as of git plugin 2.0) is to not use a workspace for polling. The problems described by Owen Jacobson (and others) still exist for the somewhat more advanced use cases like excluding users, excluding files, and including files, but the basic issue reported by the bug is fixed.
            markewaite Mark Waite made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            markewaite Mark Waite made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 140319 ] JNJira + In-Review [ 205203 ]

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                docwhat Christian Höltje
              • Votes:
                22 Vote for this issue
                Watchers:
                27 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: