-
Bug
-
Resolution: Not A Defect
-
Minor
-
None
-
Jenkins ver. 2.46.2, Pipeline 2.5
I'm building a Jenkins job that will run all my staging tests continuously, but not all at once (they rely on shared hardware). So, I'm creating parallel jobs, with a semaphore to ensure that only a limited amount are run at once.
Here's a simplified version of my pipeline that reproduces the issue:
import java.util.concurrent.Semaphore def run(job) { return { this.limiter.acquire(); try { println "running ${job}" build job println "finished ${job}" } finally { this.limiter.release(); } } } def getJobs() { def allJobs = Jenkins.getInstance().getJobNames() def stagingJobs = [] for(String job : allJobs) { if (job.startsWith("staging/temp")) { stagingJobs.add(job) } } println "${stagingJobs.size()} jobs were found." return stagingJobs } this.limiter = new Semaphore(2) def jobs = [:] for (job in getJobs()) { jobs[job] = run(job) } parallel jobs
When I run without the semaphores, everything works fine. But with the code above, I get nothing outputted except:
[Pipeline] echo
6 jobs were found.
[Pipeline] parallel
[Pipeline] [staging/temp1] { (Branch: staging/temp1)
[Pipeline] [staging/temp2] { (Branch: staging/temp2)
[Pipeline] [staging/temp3] { (Branch: staging/temp3)
[Pipeline] [staging/temp4] { (Branch: staging/temp4)
[Pipeline] [staging/temp5] { (Branch: staging/temp5)
[Pipeline] [staging/temp6] { (Branch: staging/temp6)
If I view the pipeline steps, I can see the first two jobs start, and their log messages output. However, it seems like the runner never receives a notification that the staging jobs finish. As a result, the semaphore never releases and the other 4 jobs never manage to start. Here's a thread dump mid test, after the downstream builds have definitely finished:
Thread #7 at DSL.build(unsure what happened to downstream build) at WorkflowScript.run(WorkflowScript:9) at DSL.parallel(Native Method) at WorkflowScript.run(WorkflowScript:38) Thread #8 at DSL.build(unsure what happened to downstream build) at WorkflowScript.run(WorkflowScript:9) Thread #11 at WorkflowScript.run(WorkflowScript:6) Thread #12 at WorkflowScript.run(WorkflowScript:6)
Eventually it times out with several java.lang.InterruptedException errors.
Is there a bettery way to restrict this? I would rather avoid spinning up nodes for what amounts to a simple test runner.
