Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22483

Job execution deadlock on safeRestart

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      • Start a multijob build with multiple phases
      • Plan a safe restart (URL/safeRestart)
      • The multijob subjobs/phase will finish
      • The multijob itself will not finish as the next subjob is scheduled
        • But the subjob is not started as "jenkins is preparing to shut down"

        Attachments

          Activity

          Hide
          patn03 Patrick Nielsen added a comment -

          I'm able to produce this issue using multijob plugin and trying to do a safeShutdown api and get a deadlock on waiting for jenkins to shutdown.

          Jobs required to reproduce, The child Jobs are serial not multiphase.

          • Parent Job
          • Child 1 Job
          • Child 2 Job

          What happens is parent job is running during the time the URL API call is made. Jenkins behavior is let all jobs finish and queue any future jobs. This allows the childjob 1 or 2 to get in the queue. The parent job is holding on to the child job calls waiting for them to complete but that won't happen since they are in queue.

          Show
          patn03 Patrick Nielsen added a comment - I'm able to produce this issue using multijob plugin and trying to do a safeShutdown api and get a deadlock on waiting for jenkins to shutdown. Jobs required to reproduce, The child Jobs are serial not multiphase. Parent Job Child 1 Job Child 2 Job What happens is parent job is running during the time the URL API call is made. Jenkins behavior is let all jobs finish and queue any future jobs. This allows the childjob 1 or 2 to get in the queue. The parent job is holding on to the child job calls waiting for them to complete but that won't happen since they are in queue.
          Hide
          da5id David Hows added a comment -

          I was also able to reproduce this.

          Process was:

          1. Install multiJob
          2. Create a multi-job workflow with several tasks within it
          3. Launch the mutlijob and wait till the first task is underway but not finished
          4. Run safeShutdown
          5. Wait till the first task of the multiJob is finished
          6. The second task of the mutliJob will be queued, and the multiJob itself will remain executing waiting for its sub-jobs (that cannot start) to finish
          Show
          da5id David Hows added a comment - I was also able to reproduce this. Process was: Install multiJob Create a multi-job workflow with several tasks within it Launch the mutlijob and wait till the first task is underway but not finished Run safeShutdown Wait till the first task of the multiJob is finished The second task of the mutliJob will be queued, and the multiJob itself will remain executing waiting for its sub-jobs (that cannot start) to finish
          Hide
          rk R K added a comment -

          For this use case, try the Lenient Shutdown plugin. It tries to consider triggered jobs and blocks only other jobs, so eventually Jenkins gets into a state where it can be restarted. (Unfortunately, the public version of it does not allow to trigger a restart when everything has finished.)

          Show
          rk R K added a comment - For this use case, try the Lenient Shutdown plugin. It tries to consider triggered jobs and blocks only other jobs, so eventually Jenkins gets into a state where it can be restarted. (Unfortunately, the public version of it does not allow to trigger a restart when everything has finished.)

            People

            • Assignee:
              Unassigned
              Reporter:
              jk Jan Klass
            • Votes:
              4 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated: