Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46386

Agent sometimes fails to launch in custom cloud provider

    Details

    • Similar Issues:

      Description

      (previously posted about this on mailing list https://groups.google.com/forum/#!topic/jenkinsci-dev/Zte6nExHM8w was encouraged to file a ticket also)

      I'm developing a cloud provider plugin and have run into an intermittent issue with launching agents. In the computer launch log I get this series of errors:

       

      openjdk full version "1.8.0_141-b16"
       Copying slave.jar
       Launching slave agent: java -jar /tmp/slave.jar -workDir .
       Both error and output logs will be printed to ./remoting
       <===[JENKINS REMOTING CAPACITY]===>Slave.jar version: 3.10
       This is a Unix agent
       Evacuated stdout
       java.io.IOException: Remote call on testnode failed
       at hudson.remoting.Channel.call(Channel.java:847)
       at org.jenkinsci.modules.slave_installer.impl.ComputerListenerImpl.onOnline(ComputerListenerImpl.java:32)
       at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:582)
       at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:390)
       at sfoster.jenkins.plugins.onecloud.OneComputerLauncher.launch(OneComputerLauncher.java:141)
       at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:262)
       at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
       Caused by: java.lang.LinkageError: Failed to load jenkins.util.SystemProperties
       at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:385)
       at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:293)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
       at hudson.FilePath.<clinit>(FilePath.java:2363)
       at sun.misc.Unsafe.ensureClassInitialized(Native Method)
       at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
       at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156)
       at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
       at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
       at java.lang.reflect.Field.getLong(Field.java:611)
       at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1707)
       at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:72)
       at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:484)
       at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:472)
       at java.security.AccessController.doPrivileged(Native Method)
       at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:472)
       at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:369)
       at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:598)
       at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
       at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
       at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
       at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
       at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
       at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
       at hudson.remoting.UserRequest.deserialize(UserRequest.java:245)
       at hudson.remoting.UserRequest.perform(UserRequest.java:159)
       at hudson.remoting.UserRequest.perform(UserRequest.java:52)
       at hudson.remoting.Request$2.run(Request.java:336)
       at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
       at ......remote call to testnode(Native Method)
       at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1554)
       at hudson.remoting.UserResponse.retrieve(UserRequest.java:281)
       at hudson.remoting.Channel.call(Channel.java:839)
       ... 10 more
       Caused by: java.lang.NoClassDefFoundError: javax/servlet/ServletContextListener
       at java.lang.ClassLoader.defineClass1(Native Method)
       at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
       at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
       at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:381)
       at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:293)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
       at hudson.FilePath.<clinit>(FilePath.java:2363)
       at sun.misc.Unsafe.ensureClassInitialized(Native Method)
       at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
       at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156)
       at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
       at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
       at java.lang.reflect.Field.getLong(Field.java:611)
       at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1707)
       at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:72)
       at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:484)
       at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:472)
       at java.security.AccessController.doPrivileged(Native Method)
       at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:472)
       at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:369)
       at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:598)
       at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
       at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
       at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
       at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
       at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
       at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
       at hudson.remoting.UserRequest.deserialize(UserRequest.java:245)
       at hudson.remoting.UserRequest.perform(UserRequest.java:159)
       at hudson.remoting.UserRequest.perform(UserRequest.java:52)
       at hudson.remoting.Request$2.run(Request.java:336)
       at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
       ... 4 more
       Caused by: java.lang.ClassNotFoundException: javax.servlet.ServletContextListener
       at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:788)
       at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:896)
       at hudson.remoting.Request$2.run(Request.java:336)
       at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
       at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19)
       at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
       at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
       at ......remote call to channel(Native Method)
       at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1554)
       at hudson.remoting.Request.call(Request.java:172)
       at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:260)
       at com.sun.proxy.$Proxy5.fetch(Unknown Source)
       at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:301)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
       ... 40 more
       ERROR: Connection terminated
       java.io.EOFException
       at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638)
       at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113)
       at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
       at java.io.ObjectInputStream.<init>(ObjectInputStream.java:349)
       at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
       at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
       at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
       Caused: java.io.IOException: Unexpected termination of the channel
       at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)

       This doesn't happen every time. If the failure does happen, it seems to work if I manually relaunch the agent through the UI. In the remoting/logs on the agent itself, all it has is the ERROR: Connection terminated stuff towards the bottom

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                oleg_nenashev Oleg Nenashev
                Reporter:
                stevenfoster Steven Foster
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: