Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27476

Plugin casue deadlock on Jenkins LTS 1.596.1

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Duplicate
    • Component/s: durable-task-plugin
    • Labels:
      None
    • Environment:
      RHEL 6.x
      Jenkins LTS 1.596.1
      Durable Task Plugin 1.4
    • Similar Issues:

      Description

      Our Jenkins instance are getting locked up every day.
      It seems like this is due to the durable task plugin.

      Usins JConsole and connecting to the running java process I find dadlocks and gets this stacktrace:

      Name: Computer.threadPoolForRemoting [#179]
      State: BLOCKED on hudson.slaves.RetentionStrategy$Demand@1455cecd owned by: jenkins.util.Timer [#1]
      Total blocked: 26  Total waited: 522
      
      Stack trace: 
      hudson.slaves.RetentionStrategy$Demand.check(RetentionStrategy.java:212)
      hudson.slaves.RetentionStrategy$Demand.check(RetentionStrategy.java:172)
      hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:661)
      hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120)
      hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180)
         - locked java.lang.Object@68ed76f9
      jenkins.model.Jenkins.updateComputerList(Jenkins.java:1218)
      jenkins.model.Jenkins.setNodes(Jenkins.java:1714)
      jenkins.model.Jenkins.removeNode(Jenkins.java:1709)
         - locked hudson.model.Hudson@794217b7
      hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:65)
      org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy$1.run(OnceRetentionStrategy.java:125)
         - locked hudson.model.Queue@5a25192e
      jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      java.util.concurrent.FutureTask.run(FutureTask.java:166)
      java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      java.lang.Thread.run(Thread.java:722)
      

      We are running the Jenkins LTS version 1.596.1 and Durable Task Plugin 1.4.
      We also had this problem with Durable Task plugin 1.3.

      Running Durable Task plugin 1.2 on Jenkins LTS 1.580.3 seemd to work OK.

        Attachments

          Issue Links

            Activity

            pablaasmo Per Arnold Blaasmo created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Assignee Jesse Glick [ jglick ] stephenconnolly [ stephenconnolly ]
            pablaasmo Per Arnold Blaasmo made changes -
            Link This issue depends on JENKINS-27565 [ JENKINS-27565 ]
            imakowski Ireneusz Makowski made changes -
            Labels lts-candidate
            jglick Jesse Glick made changes -
            Labels lts-candidate
            jglick Jesse Glick made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Duplicate [ 3 ]
            stephenconnolly Stephen Connolly made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 161684 ] JNJira + In-Review [ 208541 ]

              People

              • Assignee:
                stephenconnolly Stephen Connolly
                Reporter:
                pablaasmo Per Arnold Blaasmo
              • Votes:
                2 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: