Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41880

ConcurrentModificationException while creating docker image fingerprints

    Details

    • Similar Issues:

      Description

      I'm using parallel in my Jenkinsfile around docker.build(...) operator to improve build speed.

      Sometimes I'm getting the following exception during the build:

      java.lang.RuntimeException: Failed to serialize hudson.model.Fingerprint#facets for class hudson.model.Fingerprint
      	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
      	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
      	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
      	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
      	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
      	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
      	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
      	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
      	at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
      	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
      	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
      	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
      	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
      	at hudson.XmlFile.write(XmlFile.java:170)
      	at hudson.model.Fingerprint.save(Fingerprint.java:1301)
      	at hudson.model.Fingerprint.save(Fingerprint.java:1245)
      	at hudson.BulkChange.commit(BulkChange.java:98)
      	at org.jenkinsci.plugins.docker.commons.fingerprint.DockerFingerprints.addFromFacet(DockerFingerprints.java:278)
      	at org.jenkinsci.plugins.docker.workflow.FromFingerprintStep$Execution.run(FromFingerprintStep.java:119)
      	at org.jenkinsci.plugins.docker.workflow.FromFingerprintStep$Execution.run(FromFingerprintStep.java:75)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
      	at hudson.security.ACL.impersonate(ACL.java:221)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.RuntimeException: Failed to serialize org.jenkinsci.plugins.docker.commons.fingerprint.DockerDescendantFingerprintFacet#descendantImageIds for class org.jenkinsci.plugins.docker.commons.fingerprint.DockerDescendantFingerprintFacet
      	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
      	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
      	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
      	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
      	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
      	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
      	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
      	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
      	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)
      	at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)
      	at hudson.util.PersistedList$ConverterImpl.marshal(PersistedList.java:237)
      	at hudson.util.XStream2$AssociatedConverterImpl.marshal(XStream2.java:360)
      	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
      	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
      	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
      	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
      	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
      	... 27 more
      Caused by: java.util.ConcurrentModificationException
      	at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211)
      	at java.util.TreeMap$KeyIterator.next(TreeMap.java:1265)
      	at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)
      	at com.thoughtworks.xstream.converters.collections.TreeSetConverter.marshal(TreeSetConverter.java:55)
      	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
      	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
      	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
      	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
      	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
      	... 43 more
      

        Attachments

          Issue Links

            Activity

            Hide
            coftysweet coftysweet added a comment - - edited

            I have the same issue. It seems to occur on retriggering the same build multiple times (one after another) i.e. the docker images created should be identical.

            I also build images in parallel and use docker.build().
            java.lang.RuntimeException: Failed to serialize hudson.model.Fingerprint#facets for class hudson.model.Fingerprint
            at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
            at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
            at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
            at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
            at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
            at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
            at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
            at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
            at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
            at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
            at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
            at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
            at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
            at hudson.XmlFile.write(XmlFile.java:170)
            at hudson.model.Fingerprint.save(Fingerprint.java:1301)
            at hudson.model.Fingerprint.save(Fingerprint.java:1245)
            at hudson.BulkChange.commit(BulkChange.java:98)
            at org.jenkinsci.plugins.docker.commons.fingerprint.DockerFingerprints.addFromFacet(DockerFingerprints.java:278)
            at org.jenkinsci.plugins.docker.workflow.FromFingerprintStep$Execution.run(FromFingerprintStep.java:119)
            at org.jenkinsci.plugins.docker.workflow.FromFingerprintStep$Execution.run(FromFingerprintStep.java:75)
            at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
            at hudson.security.ACL.impersonate(ACL.java:260)
            at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
            Caused by: java.lang.RuntimeException: Failed to serialize org.jenkinsci.plugins.docker.commons.fingerprint.DockerDescendantFingerprintFacet#descendantImageIds for class org.jenkinsci.plugins.docker.commons.fingerprint.DockerDescendantFingerprintFacet
            at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
            at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
            at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
            at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
            at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
            at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
            at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
            at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
            at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)
            at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)
            at hudson.util.PersistedList$ConverterImpl.marshal(PersistedList.java:237)
            at hudson.util.XStream2$AssociatedConverterImpl.marshal(XStream2.java:363)
            at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
            at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
            at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
            at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
            at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
            ... 27 more
            Caused by: java.util.ConcurrentModificationException
            at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211)
            at java.util.TreeMap$KeyIterator.next(TreeMap.java:1265)
            at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)
            at com.thoughtworks.xstream.converters.collections.TreeSetConverter.marshal(TreeSetConverter.java:55)
            at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
            at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
            at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
            at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
            at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
            ... 43 more
            Finished: FAILURE

            Show
            coftysweet coftysweet added a comment - - edited I have the same issue. It seems to occur on retriggering the same build multiple times (one after another) i.e. the docker images created should be identical. I also build images in parallel and use docker.build(). java.lang.RuntimeException: Failed to serialize hudson.model.Fingerprint#facets for class hudson.model.Fingerprint at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256) at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224) at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138) at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209) at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43) at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82) at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37) at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026) at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015) at com.thoughtworks.xstream.XStream.toXML(XStream.java:988) at hudson.XmlFile.write(XmlFile.java:170) at hudson.model.Fingerprint.save(Fingerprint.java:1301) at hudson.model.Fingerprint.save(Fingerprint.java:1245) at hudson.BulkChange.commit(BulkChange.java:98) at org.jenkinsci.plugins.docker.commons.fingerprint.DockerFingerprints.addFromFacet(DockerFingerprints.java:278) at org.jenkinsci.plugins.docker.workflow.FromFingerprintStep$Execution.run(FromFingerprintStep.java:119) at org.jenkinsci.plugins.docker.workflow.FromFingerprintStep$Execution.run(FromFingerprintStep.java:75) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47) at hudson.security.ACL.impersonate(ACL.java:260) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: Failed to serialize org.jenkinsci.plugins.docker.commons.fingerprint.DockerDescendantFingerprintFacet#descendantImageIds for class org.jenkinsci.plugins.docker.commons.fingerprint.DockerDescendantFingerprintFacet at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256) at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224) at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138) at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209) at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88) at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64) at hudson.util.PersistedList$ConverterImpl.marshal(PersistedList.java:237) at hudson.util.XStream2$AssociatedConverterImpl.marshal(XStream2.java:363) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84) at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265) at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252) ... 27 more Caused by: java.util.ConcurrentModificationException at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211) at java.util.TreeMap$KeyIterator.next(TreeMap.java:1265) at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73) at com.thoughtworks.xstream.converters.collections.TreeSetConverter.marshal(TreeSetConverter.java:55) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84) at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265) at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252) ... 43 more Finished: FAILURE
            Hide
            crummy malcolm crum added a comment -

            I've got the same error, same parallel builds.

            Show
            crummy malcolm crum added a comment - I've got the same error, same parallel builds.
            Hide
            bjoernhaeuser Björn Häuser added a comment -

            Not sure if this is the right approach to the problem: https://github.com/jenkinsci/docker-commons-plugin/pull/69

            Just wanted to mention.

            Show
            bjoernhaeuser Björn Häuser added a comment - Not sure if this is the right approach to the problem: https://github.com/jenkinsci/docker-commons-plugin/pull/69 Just wanted to mention.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Björn Häuser
            Path:
            src/main/java/org/jenkinsci/plugins/docker/commons/fingerprint/DockerFingerprintAction.java
            src/main/java/org/jenkinsci/plugins/docker/commons/fingerprint/DockerFingerprints.java
            http://jenkins-ci.org/commit/docker-commons-plugin/63f655f11c9be6e79a5f9979dbfa3c2984f3ad2d
            Log:
            Synchronise on project

            I am not sure this makes any sense. I am just trying to solve this
            issue: https://issues.jenkins-ci.org/browse/JENKINS-41880

            My thoughts:

            This is likely caused by changing the finger prints, the finger prints
            are scoped to the project (right?). If this is the case synchronising on
            the project could make sense. Also `addFromFacet` should not be a
            performance killer and rather quick.

            The alternative would be for me to synchronise the complete access to the method.

            WDYT?

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Björn Häuser Path: src/main/java/org/jenkinsci/plugins/docker/commons/fingerprint/DockerFingerprintAction.java src/main/java/org/jenkinsci/plugins/docker/commons/fingerprint/DockerFingerprints.java http://jenkins-ci.org/commit/docker-commons-plugin/63f655f11c9be6e79a5f9979dbfa3c2984f3ad2d Log: Synchronise on project I am not sure this makes any sense. I am just trying to solve this issue: https://issues.jenkins-ci.org/browse/JENKINS-41880 My thoughts: This is likely caused by changing the finger prints, the finger prints are scoped to the project (right?). If this is the case synchronising on the project could make sense. Also `addFromFacet` should not be a performance killer and rather quick. The alternative would be for me to synchronise the complete access to the method. WDYT?
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/org/jenkinsci/plugins/docker/commons/fingerprint/DockerFingerprints.java
            http://jenkins-ci.org/commit/docker-commons-plugin/c38b852d4a9ebff98a0184748396959cdaeba129
            Log:
            Merge pull request #69 from bjoernhaeuser/synchronise-on-project

            JENKINS-41880 Synchronise on Fingerprint

            Compare: https://github.com/jenkinsci/docker-commons-plugin/compare/e00c616a8476...c38b852d4a9e
            *NOTE:* This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

            Functionality will be removed from GitHub.com on January 31st, 2019.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/docker/commons/fingerprint/DockerFingerprints.java http://jenkins-ci.org/commit/docker-commons-plugin/c38b852d4a9ebff98a0184748396959cdaeba129 Log: Merge pull request #69 from bjoernhaeuser/synchronise-on-project JENKINS-41880 Synchronise on Fingerprint Compare: https://github.com/jenkinsci/docker-commons-plugin/compare/e00c616a8476...c38b852d4a9e * NOTE: * This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

              People

              • Assignee:
                Unassigned
                Reporter:
                pshirshov P S
              • Votes:
                5 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: