-
Bug
-
Resolution: Not A Defect
-
Major
-
None
-
* Jenkins core 2.156
* All experimental update center level plugins, including git and git-client betas
Summary
Declarative Pipeline jobs which use a set of parallel stages to concurrently download several branches of a different repository via the checkout step can fail. The appearance is that commit hashes get confused between the two repositories. The Jenkinsfile is stored in SCM, and Git Client is expecting the commit SHA for the Jenkinsfile, but instead reads the commit SHA for one of checkout steps in the pipeline itself.
Frequency
100% of the time. But this has been a real devil to try and explain...
What this test is doing
- This Jenkinsfile (which can be seen here) does a checkout of each individual branch of this repository.
- Of the five branches that get checked out, three are using jgit as their checkout tool, and two are using Default.
- Each branch that gets checked out is full of little more than a lot of ASCII gibberish. This is to make sure the checkouts take a long enough time that the pipeline stages will actually execute in parallel for more than a fraction of a second.
Steps to recreate
1. Create a new Pipeline job. Use the following settings:
- Definition: Pipeline script from SCM
- SCM: Git
- Repository URL: https://github.com/kshultzCB/empty-repo.git
- Credentials: Any credentials you already have set up for public GitHub repos should work fine. These repos are all public.
- Branches to build: version-for-github-no-submodules . This is important, the other branches will do checkout s on a different repo.
- The remaining settings are all left to default:
- Git executable: Default
- Repository browser: (Auto)
- Additional behaviors: (None set)
- Script path: Jenkinsfile
- Lightweight checkout: checked
2. Save the Pipeline
3. Click "Build Now"
4. Watch the build process via the streaming console log. The first sign of trouble looks to be a GitException, which reads:
ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from https://github.com/kshultzCB/repo_without_submodules.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:903) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1130) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1161) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1$1.call(SynchronousNonBlockingStepExecution.java:51) at hudson.security.ACL.impersonate(ACL.java:290) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1.run(SynchronousNonBlockingStepExecution.java:48) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://github.com/kshultzCB/repo_without_submodules.git +refs/heads/*:refs/remotes/origin/*" returned status code 1: stdout: stderr: warning: no common commits
5. The job continues to execute. Eventually, we land where it looks, to me, like the commit SHA for the Jenkinsfile is being confused with the commit SHA for one of the branches we are trying to check out. Take a look at this log snippet:
From https://github.com/kshultzCB/repo_without_submodules * [new branch] 1-lots-of-stuff -> origin/1-lots-of-stuff * [new branch] 2-lots-of-stuff -> origin/2-lots-of-stuff * [new branch] 3-lots-of-stuff -> origin/3-lots-of-stuff * [new branch] 4-lots-of-stuff -> origin/4-lots-of-stuff error: Ref refs/remotes/origin/master is at 780de2489767b850181b1eff22051633ecc376e7 but expected 52fd1c9cf81f1f38e4cca47fb4f1c995967d6201 ! 52fd1c9...780de24 master -> origin/master (unable to update local ref) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2294) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1881) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:83) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:468) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) ... 4 more
This is the line that's interesting:
Error: Ref refs/remotes/origin/master is at 780de2489767b850181b1eff22051633ecc376e7 but expected 52fd1c9cf81f1f38e4cca47fb4f1c995967d6201 ! 52fd1c9...780de24 master -> origin/master (unable to update local ref)
It took a lot of scratching my head, but here's what's going on.
- 780de24 is a commit in repo_without_submodules. That's the repo which is getting checked out within the Jenkinsfile.
- 52fd1c9 is a commit in empty_repo. That's the repo where the Jenkinsfile itself is stored.
A complete log from running this test can be seen here.
I've set the component to both git-plugin and git-client-plugin, since there are errors shown from both.
