Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58671

Concurrency problems with SVN workflow libs => maybe upgrade MapDB

    Details

    • Similar Issues:

      Description

      Initial Docker based Jenkins startup based on JCasC and job-dsl and pipeline jobs auto-scheduled (via job-dsl 'queue' command) right after their creation that all use a shared Jenkins pipeline library in SVN.

      • This did not hurt and led to successful pipeline job executions
        2019-07-26 08:16:12.952+0000 [id=150]   WARNING j.s.i.s.SVNRepositoryView#<init>: failing to make/load /var/jenkins_home/caches/svn-fa86427e-e420-4d14-a4fd-9ab1bdfcde57.db
        java.lang.AssertionError: unknown trans log instruction '-58' at log offset: 176
                at org.mapdb.StoreWAL.replayLogFile(StoreWAL.java:827)
                at org.mapdb.StoreWAL.commit(StoreWAL.java:606)
                at org.mapdb.EngineWrapper.commit(EngineWrapper.java:94)
                at org.mapdb.DBMaker.makeEngine(DBMaker.java:838)
                at org.mapdb.DBMaker.make(DBMaker.java:701)
                at jenkins.scm.impl.subversion.SVNRepositoryView.<init>(SVNRepositoryView.java:105)
                at jenkins.scm.impl.subversion.SubversionSCMSource.openSession(SubversionSCMSource.java:338)
                at jenkins.scm.impl.subversion.SubversionSCMSource.retrieve(SubversionSCMSource.java:294)
                at jenkins.scm.api.SCMSource.fetch(SCMSource.java:636)
                at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.lambda$retrieve$0(SCMSourceRetriever.java:89)
                at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrySCMOperation(SCMSourceRetriever.java:104)
                at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrieve(SCMSourceRetriever.java:89)
                at org.jenkinsci.plugins.workflow.libs.LibraryAdder.retrieve(LibraryAdder.java:157)
                at org.jenkinsci.plugins.workflow.libs.LibraryAdder.add(LibraryAdder.java:138)
                at org.jenkinsci.plugins.workflow.libs.LibraryDecorator$1.call(LibraryDecorator.java:125)
                at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1065)
                at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
                at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
                at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
                at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
                at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
                at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
                at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
                at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
                at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
                at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:561)
                at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:522)
                at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
                at hudson.model.ResourceController.execute(ResourceController.java:97)
                at hudson.model.Executor.run(Executor.java:429)
        
      • This, by contrast, did hurt and led to failing builds:
        2019-07-26 08:01:00.838+0000 [id=150]   WARNING j.s.i.s.SVNRepositoryView#<init>: failing to make/load /var/jenkins_home/caches/svn-fa86427e-e420-4d14-a4fd-9ab1bdfcde57.db
        java.io.EOFException
                at org.mapdb.Volume$FileChannelVol.readFully(Volume.java:947)
                at org.mapdb.Volume$FileChannelVol.getByte(Volume.java:997)
        Caused: java.io.IOError
                at org.mapdb.Volume$FileChannelVol.getByte(Volume.java:1000)
                at org.mapdb.StoreWAL.replayLogFile(StoreWAL.java:797)
                at org.mapdb.StoreWAL.commit(StoreWAL.java:606)
                at org.mapdb.EngineWrapper.commit(EngineWrapper.java:94)
                at org.mapdb.BTreeMap.preinitCatalog(BTreeMap.java:176)
                at org.mapdb.DB.reinit(DB.java:88)
                at org.mapdb.DB.<init>(DB.java:83)
                at org.mapdb.DBMaker.make(DBMaker.java:704)
                at jenkins.scm.impl.subversion.SVNRepositoryView.<init>(SVNRepositoryView.java:105)
                at jenkins.scm.impl.subversion.SubversionSCMSource.openSession(SubversionSCMSource.java:338)
                at jenkins.scm.impl.subversion.SubversionSCMSource.retrieve(SubversionSCMSource.java:294)
                at jenkins.scm.api.SCMSource.fetch(SCMSource.java:636)
                at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.lambda$retrieve$0(SCMSourceRetriever.java:89)
                at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrySCMOperation(SCMSourceRetriever.java:104)
                at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrieve(SCMSourceRetriever.java:89)
                at org.jenkinsci.plugins.workflow.libs.LibraryAdder.retrieve(LibraryAdder.java:157)
                at org.jenkinsci.plugins.workflow.libs.LibraryAdder.add(LibraryAdder.java:138)
                at org.jenkinsci.plugins.workflow.libs.LibraryDecorator$1.call(LibraryDecorator.java:125)
                at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1065)
                at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
                at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
                at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
                at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
                at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
                at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
                at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
                at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
                at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
                at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:561)
                at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:522)
                at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
                at hudson.model.ResourceController.execute(ResourceController.java:97)
                at hudson.model.Executor.run(Executor.java:429)
        ...
        2019-07-26 08:01:00.872+0000 [id=148]   INFO    o.j.p.workflow.job.WorkflowRun#finish: Sandbox/ACME.buildDockerImage #1 completed: FAILURE
        2019-07-26 08:01:00.872+0000 [id=146]   INFO    o.j.p.workflow.job.WorkflowRun#finish: Sandbox/ACMD.stage #1 completed: FAILURE
        
        • Re-triggering the failing builds worked

      I guess the used org.mapdb v1.0.9 is to blame: there are 3.x releases and quite a few (resolved and unresolved) concurrency issues...

      Alternatively, I think, one might also be able to avoid the concurrency problem by locking the cache file creation by oneself in the usage of org.mapdb.DBMaker, that is in jenkins.scm.impl.subversion.SVNRepositoryView.<init>?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              reinholdfuereder Reinhold F├╝reder
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: