-
Bug
-
Resolution: Not A Defect
-
Major
-
None
-
Jenkins 1.561.1
ssh-slave 1.6
Test environment reproducing the issue is a Jenkins instance with 24 slave nodes all on the same machine with the same remote FS.
To not get false negatives because of sshd limits /etc/ssh/sshd cofing need a change:
< MaxSessions 200
< MaxStartups 200:30:400
The output of failing nodes are different because of the jar file being corrupted in different ways. I pasted some examples below. The issues is that the slave jar files are removed and copy by all ssh-slaves nodes. Code from src/main/java/hudson/plugins/sshslaves/SSHLauncher.java:
[...]
connection.exec("rm " + workingDirectory + "/slave.jar", new NullStream());
[...]
scp.put(IOUtils.toByteArray(is), "slave.jar", workingDirectory, "0644");
[...]
I have created a pull request with suggested fix:
https://github.com/jenkinsci/ssh-slaves-plugin/pull/17
Asmund
[05/02/14 12:23:38] [SSH] Checking java version of java
[05/02/14 12:23:39] [SSH] java -version returned 1.7.0_55.
[05/02/14 12:23:39] [SSH] Starting sftp client.
[05/02/14 12:23:39] [SSH] Copying latest slave.jar...
[05/02/14 12:23:39] [SSH] Copied 409,398 bytes.
Expanded the channel window size to 4MB
[05/02/14 12:23:39] [SSH] Starting slave process: cd "/tmp" && java -jar slave.jar
Error: Invalid or corrupt jarfile slave.jar
java.io.EOFException: unexpected stream termination
at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:200)
at hudson.remoting.Channel.<init>(Channel.java:419)
at hudson.remoting.Channel.<init>(Channel.java:398)
at hudson.remoting.Channel.<init>(Channel.java:394)
at hudson.remoting.Channel.<init>(Channel.java:383)
at hudson.remoting.Channel.<init>(Channel.java:375)
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:338)
at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:901)
at hudson.plugins.sshslaves.SSHLauncher.access$400(SSHLauncher.java:126)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:658)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:642)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
[05/02/14 12:23:39] [SSH] Connection closed.
[05/02/14 12:25:39] [SSH] Checking java version of java
[05/02/14 12:25:39] [SSH] java -version returned 1.7.0_55.
[05/02/14 12:25:39] [SSH] Starting sftp client.
[05/02/14 12:25:39] [SSH] Copying latest slave.jar...
[05/02/14 12:25:39] [SSH] Copied 409,398 bytes.
Expanded the channel window size to 4MB
[05/02/14 12:25:39] [SSH] Starting slave process: cd "/tmp" && java -jar slave.jar
Error: Unable to access jarfile slave.jar
java.io.IOException: SSH channel is closed. (Close requested by remote)
at com.trilead.ssh2.channel.ChannelManager.sendData(ChannelManager.java:382)
at com.trilead.ssh2.channel.ChannelOutputStream.write(ChannelOutputStream.java:63)
at hudson.remoting.BinarySafeStream$2.flush(BinarySafeStream.java:299)
at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1822)
at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:718)
at hudson.remoting.Capability.writePreamble(Capability.java:116)
at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:182)
at hudson.remoting.Channel.<init>(Channel.java:419)
at hudson.remoting.Channel.<init>(Channel.java:398)
at hudson.remoting.Channel.<init>(Channel.java:394)
at hudson.remoting.Channel.<init>(Channel.java:383)
at hudson.remoting.Channel.<init>(Channel.java:375)
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:338)
at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:901)
at hudson.plugins.sshslaves.SSHLauncher.access$400(SSHLauncher.java:126)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:658)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:642)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
[05/02/14 12:25:39] [SSH] Starting slave process: cd "/tmp" && java -jar slave.jar
Error: Invalid or corrupt jarfile slave.jar
java.io.IOException: SSH channel is closed. (Close requested by remote)
at com.trilead.ssh2.channel.ChannelManager.sendData(ChannelManager.java:382)
at com.trilead.ssh2.channel.ChannelOutputStream.write(ChannelOutputStream.java:63)
at hudson.remoting.BinarySafeStream$2._write(BinarySafeStream.java:272)
at hudson.remoting.BinarySafeStream$2.write(BinarySafeStream.java:255)
at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1876)
at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1785)
at java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1285)
at java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1230)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1426)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.writeFatalException(ObjectOutputStream.java:1576)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:350)
at hudson.remoting.Capability.writePreamble(Capability.java:115)
at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:182)
at hudson.remoting.Channel.<init>(Channel.java:419)
at hudson.remoting.Channel.<init>(Channel.java:398)
at hudson.remoting.Channel.<init>(Channel.java:394)
at hudson.remoting.Channel.<init>(Channel.java:383)
at hudson.remoting.Channel.<init>(Channel.java:375)
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:338)
at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:901)
at hudson.plugins.sshslaves.SSHLauncher.access$400(SSHLauncher.java:126)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:658)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:642)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)