It seems like PR:18 introduced a race-condition when you have several builds of the same job starting at the same time.
Looking at the code in the pull request:
We changed from using acquire to using allocate, which is non-blocking and instead append a suffix to the path if needed. The only problem with the change is that we still use the base directory given to allocate and ignoring the fact that we might have allocated another directory. Instead we should use lease.path, which is what we actually leased!
So the code should probably be in the lines of: