Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-25921

Random RejectedExecutionException during tests

    Details

    • Similar Issues:

      Description

      Oftentimes tests fail with

      java.util.concurrent.RejectedExecutionException: null
      	at hudson.remoting.SingleLaneExecutorService.execute(SingleLaneExecutorService.java:99)
      	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
      	at com.google.common.util.concurrent.ForwardingExecutorService.submit(ForwardingExecutorService.java:105)
      	at jenkins.util.InterceptingExecutorService.submit(InterceptingExecutorService.java:39)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.scheduleRun(CpsThreadGroup.java:172)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.waitForSuspension(CpsFlowExecution.java:524)
      	at org.jenkinsci.plugins.workflow.SingleJobTestBase.waitForWorkflowToSuspend(SingleJobTestBase.java:108)
      	at org.jenkinsci.plugins.workflow.SingleJobTestBase.waitForWorkflowToComplete(SingleJobTestBase.java:95)
      	at org.jenkinsci.plugins.workflow.steps.input.InputStepRestartTest$2.evaluate(InputStepRestartTest.java:63)
      

      The usual meaning of "shutdown" here, that Jenkins is shutting down, does not seem to apply, since we are in the middle of a story.

      I suspect that CpsThreadGroup.scheduleRun is to blame. It calls runner.shutdown() sometimes. It does catch RejectedExecutionException (from cc4eca7), but in the nested call to submit which was observed to throw this exception earlier, not the one that is throwing the exception here.

      The call to shutdown dates to 42dbfb1 in the single-thread branch, which says only

      shutdown the executor pool when CpsThreadGroup is done

      Is this still necessary and appropriate after 289c9b0 (collection of CpsThread)?

      It is possible the flaw is in CpsFlowExecution.waitForSuspension, as a TODO comment there suggests. Can there be a race condition whereby programPromise is momentarily not null, but then the flow ends immediately afterwards? For that matter, is the comment

      the execution has already finished

      even correct, given that there appears to be no code which would set programPromise back to null? If I am right, then this comment is misleading, quietly returning when it is null is possibly wrong, and g.scheduleRun().get() ought to be returning without throwing an exception if the flow is already shut down.

        Attachments

          Activity

          Hide
          jglick Jesse Glick added a comment -

          In fact adding an extra call to waitForWorkflowToComplete triggers the failure, confirming my hypothesis.

          Show
          jglick Jesse Glick added a comment - In fact adding an extra call to waitForWorkflowToComplete triggers the failure, confirming my hypothesis.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsThreadGroup.java
          http://jenkins-ci.org/commit/workflow-cps-plugin/fb0ee2a6c8ce84137b2fbacea52c14a601678c2e
          Log:
          [FXED JENKINS-25921] scheduleRun should not throw RejectedExecutionException simply because the flow is complete.
          Originally-Committed-As: 121d07f1e182f3d17a9257de79dcadb0cb5d67f5

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsThreadGroup.java http://jenkins-ci.org/commit/workflow-cps-plugin/fb0ee2a6c8ce84137b2fbacea52c14a601678c2e Log: [FXED JENKINS-25921] scheduleRun should not throw RejectedExecutionException simply because the flow is complete. Originally-Committed-As: 121d07f1e182f3d17a9257de79dcadb0cb5d67f5

            People

            • Assignee:
              jglick Jesse Glick
              Reporter:
              jglick Jesse Glick
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: