Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44597

ConcurrentModificationException on releasing lock

    Details

    • Type: Bug
    • Status: Reopened (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Labels:
      None
    • Environment:
      Jenkins ver. 2.46.2 (LTS)
      Java 1.8.0_131-b11 (Oracle)
      Lockable Resources plugin 2.0
      Pipeline 2.5
    • Similar Issues:
    • Released As:
      lockable-resources 2.1

      Description

      We are using the lockable resources plugin extensively in our Jenkins Pipeline Scripts, today one of our Builds failed altough the actual build process was successful. The following error is shown in the log.

      [lockable-resources] released lock on [LOCK]
      java.util.ConcurrentModificationException
      	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
      	at java.util.ArrayList$Itr.next(ArrayList.java:851)
      	at org.jenkins.plugins.lockableresources.LockableResourcesManager.getNextQueuedContext(LockableResourcesManager.java:445)
      	at org.jenkins.plugins.lockableresources.LockableResourcesManager.unlockNames(LockableResourcesManager.java:351)
      	at org.jenkins.plugins.lockableresources.LockStepExecution$Callback.finished(LockStepExecution.java:84)
      	at org.jenkinsci.plugins.workflow.steps.BodyExecutionCallback$TailCall.onSuccess(BodyExecutionCallback.java:114)
      	at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$SuccessAdapter.receive(CpsBodyExecution.java:362)
      	at com.cloudbees.groovy.cps.Outcome.resumeFrom(Outcome.java:73)
      	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:146)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
      	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
      	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:165)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:330)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230)
      	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
      	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:748)

      We are holding onto best practices for locks, to not use them from within node blocks, but outside so no executor is blocked as long as we wait for the lock:

      lock(label:'lock-group'){
          node {
              // here comes the build
          }
      }

      hope that helps.

      Our build infrastructure consists of two servers (a master with 1 executor) and a slave (with 3 executors).

      One thing we also noticed is that locking and unlocking (even when using the UI) is often super slow (taking up to a minute), but that may be another problem.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dageissl Daniel Gei├čler
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: