Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50019

java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We intermittently see nodes fail during Git checkout with a traceback that looks like this:

       
      *09:45:25* java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer
      *09:45:25* 	at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29)
      *09:45:25* 	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:71)
      *09:45:25* 	at jdk.internal.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
      *09:45:25* 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      *09:45:25* 	at java.lang.reflect.Method.invoke(Method.java:564)
      *09:45:25* 	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:922)
      *09:45:25* 	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:896)
      *09:45:25* 	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
      *09:45:25* 	at hudson.remoting.UserRequest.perform(UserRequest.java:207)
      *09:45:25* 	at hudson.remoting.UserRequest.perform(UserRequest.java:53)
      *09:45:25* 	at hudson.remoting.Request$2.run(Request.java:358)
      *09:45:25* 	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
      *09:45:25* 	at java.util.concurrent.FutureTask.run(FutureTask.java:264)
      *09:45:25* 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
      *09:45:25* 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
      *09:45:25* 	at hudson.remoting.Engine$1$1.run(Engine.java:98)
      *09:45:25* 	at java.lang.Thread.run(Thread.java:844)
      Full log: [https://ci.pytorch.org/jenkins/job/pytorch-builds/job/pytorch-macos-10.13-py3-build-test/2797//console]
      

      Retrying does not resolve the problem, however, subsequent builds on the same node often do succeed (for the case above, three hours later another build succeeded.)

      The error is highly reminiscent of https://issues.jenkins-ci.org/browse/JENKINS-19453 but that issue was fixed in the Jenkins 1.x series, and this is a much more modern version of Jenkins. Additionally, the error doesn't seem to be persistent (in that it's not necessary to restart the worker to resolve the problem.)

      BTW, this is not just an OS X slave problem; we've had it happen to Linux workers too (although the missing class is different): https://ci.pytorch.org/jenkins/job/pytorch-builds/job/pytorch-linux-trusty-py2.7.9-build/3214/console

      I'm not really sure how to go about making a reproducing test case. Let me know if you have any ideas.

      The other case stack trace looks like this:

      23:28:29 Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ip-172-31-82-71.ec2.internal/172.31.82.71:42464
      23:28:29 		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
      23:28:29 		at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
      23:28:29 		at hudson.remoting.Channel.call(Channel.java:908)
      23:28:29 		at hudson.FilePath.act(FilePath.java:986)
      23:28:29 		at hudson.FilePath.act(FilePath.java:975)
      23:28:29 		at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:137)
      23:28:29 		at hudson.plugins.git.GitSCM.createClient(GitSCM.java:758)
      23:28:29 		at hudson.plugins.git.GitSCM.createClient(GitSCM.java:749)
      23:28:29 		at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1117)
      23:28:29 		at hudson.scm.SCM.checkout(SCM.java:495)
      23:28:29 		at hudson.model.AbstractProject.checkout(AbstractProject.java:1202)
      23:28:29 		at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
      23:28:29 		at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
      23:28:29 		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
      23:28:29 		at hudson.model.Run.execute(Run.java:1724)
      23:28:29 		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
      23:28:29 		at hudson.model.ResourceController.execute(ResourceController.java:97)
      23:28:29 		at hudson.model.Executor.run(Executor.java:429)
      23:28:29 java.lang.NoClassDefFoundError: Could not initialize class com.sun.proxy.$Proxy10
      23:28:29 	at sun.reflect.GeneratedConstructorAccessor16.newInstance(Unknown Source)
      23:28:29 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      23:28:29 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      23:28:29 	at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:739)
      23:28:29 	at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:165)
      23:28:29 	at hudson.remoting.Channel.export(Channel.java:722)
      23:28:29 	at hudson.remoting.Channel.export(Channel.java:686)
      23:28:29 	at org.jenkinsci.plugins.gitclient.LegacyCompatibleGitAPIImpl.writeReplace(LegacyCompatibleGitAPIImpl.java:198)
      23:28:29 	at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
      23:28:29 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      23:28:29 	at java.lang.reflect.Method.invoke(Method.java:498)
      23:28:29 	at java.io.ObjectStreamClass.invokeWriteReplace(ObjectStreamClass.java:1218)
      23:28:29 	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1136)
      23:28:29 	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
      23:28:29 	at hudson.remoting.UserRequest._serialize(UserRequest.java:250)
      23:28:29 	at hudson.remoting.UserRequest.serialize(UserRequest.java:259)
      23:28:29 	at hudson.remoting.UserRequest.perform(UserRequest.java:221)
      23:28:29 	at hudson.remoting.UserRequest.perform(UserRequest.java:53)
      23:28:29 	at hudson.remoting.Request$2.run(Request.java:358)
      23:28:29 	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
      23:28:29 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      23:28:29 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      23:28:29 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      23:28:29 	at hudson.remoting.Engine$1$1.run(Engine.java:94)
      23:28:29 	at java.lang.Thread.run(Thread.java:748)
      23:28:29 Caused: java.io.IOException: Remote call on JNLP4-connect connection from ip-172-31-82-71.ec2.internal/172.31.82.71:42464 failed
      23:28:29 	at hudson.remoting.Channel.call(Channel.java:916)
      23:28:29 	at hudson.FilePath.act(FilePath.java:986)
      23:28:29 Caused: java.io.IOException: remote file operation failed: /var/lib/jenkins/workspace/pytorch-builds/pytorch-linux-trusty-py2.7.9-build at hudson.remoting.Channel@54638415:JNLP4-connect connection from ip-172-31-82-71.ec2.internal/172.31.82.71:42464
      23:28:29 	at hudson.FilePath.act(FilePath.java:993)
      23:28:29 	at hudson.FilePath.act(FilePath.java:975)
      23:28:29 	at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:137)
      23:28:29 	at hudson.plugins.git.GitSCM.createClient(GitSCM.java:758)
      23:28:29 	at hudson.plugins.git.GitSCM.createClient(GitSCM.java:749)
      23:28:29 	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1117)
      23:28:29 	at hudson.scm.SCM.checkout(SCM.java:495)
      23:28:29 	at hudson.model.AbstractProject.checkout(AbstractProject.java:1202)
      23:28:29 	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
      23:28:29 	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
      23:28:29 	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
      23:28:29 	at hudson.model.Run.execute(Run.java:1724)
      23:28:29 	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
      23:28:29 	at hudson.model.ResourceController.execute(ResourceController.java:97)
      23:28:29 	at hudson.model.Executor.run(Executor.java:429)
      

        Attachments

          Issue Links

            Activity

            Hide
            ezyang Edward Yang added a comment -

            So, with bugs like this, it is always hard to tell if your workaround has worked or not, but we are reasonably confident that making the JVM versions on your server and client line up exactly helps prevent the problem. We noticed this specifically on OS X because our workers had Java auto-update, and when we turned it off the error incidence went way down. If you are having this problem, try making sure your Java versions are the same.

            Show
            ezyang Edward Yang added a comment - So, with bugs like this, it is always hard to tell if your workaround has worked or not, but we are reasonably confident that making the JVM versions on your server and client line up exactly helps prevent the problem. We noticed this specifically on OS X because our workers had Java auto-update, and when we turned it off the error incidence went way down. If you are having this problem, try making sure your Java versions are the same.
            Hide
            jthompson Jeff Thompson added a comment -

            This looks like a communication failure between master and agent. The connection can be broken by many external causes. Commonly these have to do with networking, system, or other environmental issues. As Edward Yang noted sometimes these can be corrected by making sure versions are lined up. Sometimes it is due to a virtualization or containerization environment closing things down.
            It's difficult to suggest where you should investigate to determine the issue. However, there currently isn't enough information provided to take any action.

            Show
            jthompson Jeff Thompson added a comment - This looks like a communication failure between master and agent. The connection can be broken by many external causes. Commonly these have to do with networking, system, or other environmental issues. As Edward Yang noted sometimes these can be corrected by making sure versions are lined up. Sometimes it is due to a virtualization or containerization environment closing things down. It's difficult to suggest where you should investigate to determine the issue. However, there currently isn't enough information provided to take any action.
            Hide
            jthompson Jeff Thompson added a comment -

            Since the reporter said that this was now working for them, I'm going to close it out. Hopefully they'll continue to have success.

            Show
            jthompson Jeff Thompson added a comment - Since the reporter said that this was now working for them, I'm going to close it out. Hopefully they'll continue to have success.
            Hide
            markewaite Mark Waite added a comment -

            Fixed in git client plugin 2.7.4 thanks to PR 359 from Vincent Latombe.

            Show
            markewaite Mark Waite added a comment - Fixed in git client plugin 2.7.4 thanks to PR 359 from Vincent Latombe .
            Hide
            jtancer Jon Tancer added a comment -

            Rebooting the host fixed the problem for me.

            Show
            jtancer Jon Tancer added a comment - Rebooting the host fixed the problem for me.

              People

              • Assignee:
                Unassigned
                Reporter:
                ezyang Edward Yang
              • Votes:
                3 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: