In order to test for another bug, I created a farm (multi-configuration) job that executes 8 times per minute.

It runs on four of my windows xp slaves, and a slave which runs on the master(redhat).

There are seven trigger jobs (trigger [1-7]) which all trigger the above, they do nothing but sleep 3 seconds.

All eight of the above jobs (find a tar of their config.xml files attached) are configured to run only on "master", which has only 1 executor, thereby single threading the execution.

All of the windows xp slaves have cygwin installed.

The job runs fine on its own, without failure, when the triggers did not exist.

With the triggers, it takes about ten minutes or so (75-150 builds) for a failure, and it is not always on the same machine, the stack trace is:

Started by upstream project "farm env" build number 2330
Building remotely on xpbuild1_vm_sandbox
$ C:\cygwin\bin\cygpath -w /bin/sh
[xpbuild1_vm_sandbox] $ /bin/sh -xe "C:\Documents and Settings\hudson\hudson124072745410030835.sh"
The system cannot find the path specified
FATAL: command execution failed
java.io.IOException: Cannot run program "/bin/sh" (in directory "c:\hudson_sandbox\workspace\farm env\label\xp_vm_sandbox"): CreateProcess error=3, The system cannot find the path specified
at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
at hudson.Proc$LocalProc.<init>(Proc.java:192)
at hudson.Proc$LocalProc.<init>(Proc.java:164)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:638)
at hudson.Launcher$ProcStarter.start(Launcher.java:273)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:793)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:767)
at hudson.remoting.UserRequest.perform(UserRequest.java:114)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: CreateProcess error=3, The system cannot find the path specified
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(ProcessImpl.java:81)
at java.lang.ProcessImpl.start(ProcessImpl.java:30)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
... 15 more
Checking console output
Notifying upstream projects of job completion
Finished: FAILURE