Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45758

Pipeline sh, bat & sleep steps hang and executors don't release.

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Incomplete
    • Component/s: pipeline
    • Environment:
      Jenkins v2.69, running on Ubuntu 16.04LTS, with agents running Windows 10 and Ubuntu. Pipeline v2.5, Durable Task v1.14
    • Similar Issues:

      Description

      Jenkins intermittently enters a state where:

      1. Any started pipeline jobs hang indefinitely on sh, bat, and sleep steps regardless of if they are executed on master or an agent, Unix or Windows systems
      2. Pipelines without sh/bat/sleep steps complete, but do not release their executors when completed
      3. Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works
      4. Jenkins is otherwise responsive
      5. *Update* Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released.

      The following Pipeline script with "sleep"

      node('NodeName') {
        echo 'Hello World'
        sleep(1)
      }
      

      results in hanging at the sleep step... 

      [Pipeline] node
      Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang
      [Pipeline] {
      [Pipeline] echo
      Hello World
      [Pipeline] sleep
      Sleeping for 1 sec
      

      This pipeline script completes, 

      node('NodeName') {
        echo 'Hello World'
      }

      but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted.

      *Update* Multiple ThreadDumps attached from described Jenkins state.

        Attachments

        1. ThreadDump-Agent.txt
          0.2 kB
        2. ThreadDump2.txt
          645 kB
        3. ThreadDump.txt
          736 kB

          Activity

          Hide
          jglick Jesse Glick added a comment -

          No idea offhand. Would need steps to reproduce from scratch.

          Show
          jglick Jesse Glick added a comment - No idea offhand. Would need steps to reproduce from scratch.

            People

            • Assignee:
              Unassigned
              Reporter:
              anshuarya Anshu Arya
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: