JenkinsIcon indicating the project type

Jenkins

Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38316

Entering, then cancelling, quiet mode causes builds to hang

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Duplicate
    • Component/s: workflow-cps-plugin
    • Labels:
      None
    • Environment:
      jenkins:latest docker image
    • Similar Issues:
    • Epic Link:

      Description

      This problem can be reproduced with a simple pipeline job that does nothing but echo and sleep in a loop.

      1) Start a run of the pipeline job, the job produces output in the Console Output as I expect
      2) Put Jenkins in "Preparing for Shutdown" mode. The jobs stop producing output. That's somewhat surprising, as runs that were started before entering Shutdown Mode are usually allowed to complete.
      3) Cancel "Shutdown Mode." The job is still "stuck" and doesn't produce any output. The job also cannot be canceled except by a "hard kill," i.e. going through the "Forcibly terminate running steps" and "Forcibly kill entire build" links

      The behavior in #2 is surprising, but not really a problem for me. However, the fact that entering and leaving shutdown mode completely hoses running jobs is really bad. This is compounded by an init.groovy script we use that places Jenkins in shutdown mode at startup.

        Attachments

          Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-32015 Stop Groovy code execution during quietingDown

          • Major - Major loss of function.
          • Resolved
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-34256 Preparing Jenkins For Shutdown Hangs Running Pipelines

          • Major - Major loss of function.
          • In Review
          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-49365 Resuming of pipelines vs. docker.image(...).inside(...)?

          • Blocker - Blocks development and/or testing work, production could not run.
          • Reopened

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-34256 Preparing Jenkins For Shutdown Hangs Running Pipelines

          • Major - Major loss of function.
          • In Review

            Activity

            Ascending order - Click to sort in descending order
            11 older comments
            Hide
            Permalink
            reinholdfuereder Reinhold Füreder added a comment -

            ad #2 pipeline stops/freezes instead of expected and documented (e.g. also in UI "Manage Jenkins > Prepare for Shutdown") finishing of job:

            • this is surprising and unexpected considering also the documentation, but of course makes sense as described by Jesse.
            • however, would it make sense to change that to e.g. finish execution of currently active stage like commented in JENKINS-49365 – and somehow signal that in the UI similar to current "(part)" suffix in "build executor status" to allow for gentle shutdown at well-defined points? And therefore, as mentioned in JENKINS-49365, also when docker.image(...).inside(...) is used...

            ad #3 pipeline does not wake up after canceling the shutdown mode:

            • yes, I can confirm: "pause" (also) after cancelling followed by "resume" works and would be an IMHO acceptable workaround – although I really would not mind if that would be "just" fixed, naively assuming that it is not that difficult to do?

            While (resumable, i.e. non-docker.image(...).inside(...)-dependent) pipelines resume fine after a restart, there is a remainder in "build executor status" that (after the pipeline has finished) requires the workaround to abort it (the finished pipeline) by pressing the 'x' that in turn then asks "Are you sure you want to abort null?":

            Show
            reinholdfuereder Reinhold Füreder added a comment - ad #2 pipeline stops/freezes instead of expected and documented (e.g. also in UI "Manage Jenkins > Prepare for Shutdown") finishing of job: this is surprising and unexpected considering also the documentation, but of course makes sense as described by Jesse. however, would it make sense to change that to e.g. finish execution of currently active stage like commented in JENKINS-49365 – and somehow signal that in the UI similar to current "(part)" suffix in "build executor status" to allow for gentle shutdown at well-defined points? And therefore, as mentioned in JENKINS-49365 , also when docker.image(...).inside(...) is used... ad #3 pipeline does not wake up after canceling the shutdown mode: yes, I can confirm: "pause" (also) after cancelling followed by "resume" works and would be an IMHO acceptable workaround – although I really would not mind if that would be "just" fixed, naively assuming that it is not that difficult to do? While (resumable, i.e. non- docker.image(...).inside(...) -dependent) pipelines resume fine after a restart, there is a remainder in "build executor status" that (after the pipeline has finished) requires the workaround to abort it (the finished pipeline) by pressing the 'x' that in turn then asks "Are you sure you want to abort null?":
            Hide
            Permalink
            svanoort Sam Van Oort added a comment -

            Closing in preference to the pre-existing issue that has the same result (sigh, still not resolved yet).

            Show
            svanoort Sam Van Oort added a comment - Closing in preference to the pre-existing issue that has the same result (sigh, still not resolved yet).
            Hide
            Permalink
            taylor01 David Taylor added a comment -

            Any updates would be very appreciated. This bug is causing a lot of problems for our build system

            Show
            taylor01 David Taylor added a comment - Any updates would be very appreciated. This bug is causing a lot of problems for our build system
            Hide
            Permalink
            totoroliu Rick Liu added a comment -

            This still happened to Jenkins v*2.138.4* LTS

            Show
            totoroliu Rick Liu added a comment - This still happened to Jenkins v*2.138.4* LTS
            Hide
            Permalink
            jglick Jesse Glick added a comment -

            (incorrect resolution)

            Show
            jglick Jesse Glick added a comment - (incorrect resolution)

              People

              • Assignee:
                Unassigned
                Reporter:
                srwalter Steven Walter
              • Votes:
                20 Vote for this issue
                Watchers:
                26 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: