-
Bug
-
Resolution: Fixed
-
Major
-
None
-
core 2.176.2
kubernetes 2.18.1
durable task plugin 1.30
Jenkins master freeze on hudson.model.Queue.withLock.
At certain moment all jobs have been stucked in waiting state, locked by java.util.concurrent.locks.ReentrantLock without any progress for 20 minutes until jenkins was restarted:
"Computer.threadPoolForRemoting [#26557] / waiting for JNLP4-connect connection from 10.116.113.11/10.116.113.11:35228 id=20559800" #9908709 daemon prio=5 os_prio=0 tid=0x00007f30602ca000 nid=0x4609 in Object.wait() [0x00007f3120f8b000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at hudson.remoting.Request.call(Request.java:177) - locked <0x00000004ea5957f0> (a hudson.remoting.UserRequest) at hudson.remoting.Channel.call(Channel.java:956) at org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave._terminate(KubernetesSlave.java:263) at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:67) at org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy$1$1.run(OnceRetentionStrategy.java:128) at hudson.model.Queue._withLock(Queue.java:1379) at hudson.model.Queue.withLock(Queue.java:1256) at org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy$1.run(OnceRetentionStrategy.java:123) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - <0x00000002c0779d00> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) - <0x00000004ea595920> (a java.util.concurrent.ThreadPoolExecutor$Worker)