-
Bug
-
Resolution: Cannot Reproduce
-
Major
-
None
-
amazon linux 2014.03 slaves
jenkins 1.562 on amazon linux 2014.03
ec2 plugin 1.21
Slaves randomly loose ability to exec programs in the workspace. This usually manifests with either git or mvn not being found. Once in this state, running a simple "which" command via the script console shows that the program can not be found, though it is definitely there.
println "which /usr/bin/git".execute().text
It does not matter if the command is called via absolute path, it will still not be found.
Example stack where git can't be found. Similar stack presents for mvn if I switch to jgit once in this state.
hudson.plugins.git.GitException: Could not init /home/ec2-user/jenkins/workspace/job
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$4.execute(CliGitAPIImpl.java:476)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:379)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:328)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: hudson.plugins.git.GitException: Error performing command: /usr/bin/git init /home/ec2-user/jenkins/workspace/job
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1308)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1277)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1273)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1078)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$4.execute(CliGitAPIImpl.java:474)
... 11 more
Caused by: java.io.IOException: Cannot run program "/usr/bin/git" (in directory "/home/ec2-user/jenkins/workspace/job"): error=2, No such file or directory
at java.lang.ProcessBuilder.start(Unknown Source)
at hudson.Proc$LocalProc.<init>(Proc.java:244)
at hudson.Proc$LocalProc.<init>(Proc.java:216)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:775)
at hudson.Launcher$ProcStarter.start(Launcher.java:355)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1297)
... 15 more
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 21 more
ERROR: null
Once this happens, the only recourse is to delete the slave and let the plugin provision a new instance from the base AMI.