Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34256

Preparing Jenkins For Shutdown Hangs Running Pipelines

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Start a couple long-running pipelines with

      node

      { sleep 100 }

      Queue up a few more jobs. Go to "manage jenkins" and "prepare for shutdown."

      Now pipeline jobs that would finish and unenqueue never finish and have to manually be killed (which does work). Freestyle jobs complete normally. Queued jobs aren't run, so that part of prepare-for-shutdown works.

      Even stranger: upon killing and restarting with Ctrl+C, we get this lovely conundrum:

      Those pipeline builds won't show up in the build queue on the main screen.

      Checks to do:

      • Regression in core?
      • Regression in pipeline?
      • does /safeRestart or /restart trigger it?

        Attachments

          Issue Links

            Activity

            Hide
            svanoort Sam Van Oort added a comment -

            Mike Kozell That specific case sounds a lot like a gremlin we've been chasing on and off for quite a while. I'm assigning this to Jose Blas Camacho Taboada to investigate.

            I think what you report may be independent of what was discussed here though which is probably the root cause of the issue: https://issues.jenkins-ci.org/browse/JENKINS-34256?focusedCommentId=336282&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-336282 through https://issues.jenkins-ci.org/browse/JENKINS-34256?focusedCommentId=332080&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-332080

            Show
            svanoort Sam Van Oort added a comment - Mike Kozell That specific case sounds a lot like a gremlin we've been chasing on and off for quite a while. I'm assigning this to Jose Blas Camacho Taboada to investigate. I think what you report may be independent of what was discussed here though which is probably the root cause of the issue: https://issues.jenkins-ci.org/browse/JENKINS-34256?focusedCommentId=336282&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-336282 through https://issues.jenkins-ci.org/browse/JENKINS-34256?focusedCommentId=332080&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-332080
            Hide
            jeubank Josiah Eubank added a comment -

            Also experiencing this new since around 2.140 in all pipeline jobs.  These jobs are on lowest durability setting.

             

            Previously had no issue holding the queue by "preparing for shutdown" and the currently running jobs would finish.  Now have to force Jenkins to restart to get rid of the jobs.

            Show
            jeubank Josiah Eubank added a comment - Also experiencing this new since around 2.140 in all pipeline jobs.  These jobs are on lowest durability setting.   Previously had no issue holding the queue by "preparing for shutdown" and the currently running jobs would finish.  Now have to force Jenkins to restart to get rid of the jobs.
            Hide
            narenji Ali Narenji added a comment -

            We have the same problem on Jenkins 2.138.2.

            Is there any time estimation for resolving the issue?

            Show
            narenji Ali Narenji added a comment - We have the same problem on Jenkins 2.138.2. Is there any time estimation for resolving the issue?
            Hide
            hgholami Hamid Gholami added a comment -

            Any update?

            We have same issue on Jenkins.

            Show
            hgholami Hamid Gholami added a comment - Any update? We have same issue on Jenkins.
            Hide
            tknerr Torben Knerr added a comment - - edited

            Same issue here with Jenkins LTS 2.150.2

            I'm seeing this with pipeline durability set to "PERFORMANCE_OPTIMIZED" in the global configuration.

            Sam Van Oort re-reading your comment here https://issues.jenkins-ci.org/browse/JENKINS-34256?focusedCommentId=332080&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-332080 I am wondering why the currently executing pipeline should actually halt by design – wouldn't it be more intuitive if any running pipelines just complete (as it was the case with freestyle jobs earlier)

            Show
            tknerr Torben Knerr added a comment - - edited Same issue here with Jenkins LTS 2.150.2 I'm seeing this with pipeline durability set to "PERFORMANCE_OPTIMIZED" in the global configuration. Sam Van Oort re-reading your comment here https://issues.jenkins-ci.org/browse/JENKINS-34256?focusedCommentId=332080&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-332080 I am wondering why the currently executing pipeline should actually halt by design – wouldn't it be more intuitive if any running pipelines just complete (as it was the case with freestyle jobs earlier)

              People

              • Assignee:
                Unassigned
                Reporter:
                svanoort Sam Van Oort
              • Votes:
                28 Vote for this issue
                Watchers:
                46 Start watching this issue

                Dates

                • Created:
                  Updated: