Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38316

Entering, then cancelling, quiet mode causes builds to hang

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: workflow-cps-plugin
    • Labels:
      None
    • Environment:
      jenkins:latest docker image
    • Similar Issues:

      Description

      This problem can be reproduced with a simple pipeline job that does nothing but echo and sleep in a loop.

      1) Start a run of the pipeline job, the job produces output in the Console Output as I expect
      2) Put Jenkins in "Preparing for Shutdown" mode. The jobs stop producing output. That's somewhat surprising, as runs that were started before entering Shutdown Mode are usually allowed to complete.
      3) Cancel "Shutdown Mode." The job is still "stuck" and doesn't produce any output. The job also cannot be canceled except by a "hard kill," i.e. going through the "Forcibly terminate running steps" and "Forcibly kill entire build" links

      The behavior in #2 is surprising, but not really a problem for me. However, the fact that entering and leaving shutdown mode completely hoses running jobs is really bad. This is compounded by an init.groovy script we use that places Jenkins in shutdown mode at startup.

        Attachments

          Issue Links

            Activity

            Hide
            yasko Yasen Terziivanov added a comment -

            Also a problem in Jenkins: 2.89.4, Pipeline: 2.5 

            Show
            yasko Yasen Terziivanov added a comment - Also a problem in Jenkins: 2.89.4, Pipeline: 2.5 
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            ad #2 pipeline stops/freezes instead of expected and documented (e.g. also in UI "Manage Jenkins > Prepare for Shutdown") finishing of job:

            • this is surprising and unexpected considering also the documentation, but of course makes sense as described by Jesse.
            • however, would it make sense to change that to e.g. finish execution of currently active stage like commented in JENKINS-49365 – and somehow signal that in the UI similar to current "(part)" suffix in "build executor status" to allow for gentle shutdown at well-defined points? And therefore, as mentioned in JENKINS-49365, also when docker.image(...).inside(...) is used...

            ad #3 pipeline does not wake up after canceling the shutdown mode:

            • yes, I can confirm: "pause" (also) after cancelling followed by "resume" works and would be an IMHO acceptable workaround – although I really would not mind if that would be "just" fixed, naively assuming that it is not that difficult to do?

            While (resumable, i.e. non-docker.image(...).inside(...)-dependent) pipelines resume fine after a restart, there is a remainder in "build executor status" that (after the pipeline has finished) requires the workaround to abort it (the finished pipeline) by pressing the 'x' that in turn then asks "Are you sure you want to abort null?":

            Show
            reinholdfuereder Reinhold Füreder added a comment - ad #2 pipeline stops/freezes instead of expected and documented (e.g. also in UI "Manage Jenkins > Prepare for Shutdown") finishing of job: this is surprising and unexpected considering also the documentation, but of course makes sense as described by Jesse. however, would it make sense to change that to e.g. finish execution of currently active stage like commented in JENKINS-49365 – and somehow signal that in the UI similar to current "(part)" suffix in "build executor status" to allow for gentle shutdown at well-defined points? And therefore, as mentioned in JENKINS-49365 , also when docker.image(...).inside(...) is used... ad #3 pipeline does not wake up after canceling the shutdown mode: yes, I can confirm: "pause" (also) after cancelling followed by "resume" works and would be an IMHO acceptable workaround – although I really would not mind if that would be "just" fixed, naively assuming that it is not that difficult to do? While (resumable, i.e. non- docker.image(...).inside(...) -dependent) pipelines resume fine after a restart, there is a remainder in "build executor status" that (after the pipeline has finished) requires the workaround to abort it (the finished pipeline) by pressing the 'x' that in turn then asks "Are you sure you want to abort null?":
            Hide
            svanoort Sam Van Oort added a comment -

            Closing in preference to the pre-existing issue that has the same result (sigh, still not resolved yet).

            Show
            svanoort Sam Van Oort added a comment - Closing in preference to the pre-existing issue that has the same result (sigh, still not resolved yet).
            Hide
            taylor01 David Taylor added a comment -

            Any updates would be very appreciated. This bug is causing a lot of problems for our build system

            Show
            taylor01 David Taylor added a comment - Any updates would be very appreciated. This bug is causing a lot of problems for our build system
            Hide
            totoroliu Rick Liu added a comment -

            This still happened to Jenkins v*2.138.4* LTS

            Show
            totoroliu Rick Liu added a comment - This still happened to Jenkins v*2.138.4* LTS

              People

              • Assignee:
                Unassigned
                Reporter:
                srwalter Steven Walter
              • Votes:
                20 Vote for this issue
                Watchers:
                26 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: