Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-5073

hudson.util.IOException2: Failed to join the process - on a Windows slave

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Component/s: remoting
    • Labels:
      None
    • Environment:
      Windows XP master - Windows XP slave, Linux

      Description

      When executing a free-style job in a Windows agent, this error appers:

      //wc/psp/Branches/Release/PSP/content/EaTrax/MichaelFranti.wav#1 - added as P:\Release\PSP\content\EaTrax\MichaelFranti.wav
      //wc/psp/Branches/Release/PSP/content/EaTrax/MiikeSnow.wav#1 - added as P:\Release\PSPFATAL: command execution failed
      hudson.util.IOException2: Failed to join the process
      	at hudson.Proc$RemoteProc.join(Proc.java:311)
      	at hudson.Launcher$ProcStarter.join(Launcher.java:275)
      	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83)
      	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
      	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:577)
      	at hudson.model.Build$RunnerImpl.build(Build.java:165)
      	at hudson.model.Build$RunnerImpl.doRun(Build.java:133)
      	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:417)
      	at hudson.model.Run.run(Run.java:1176)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:123)
      Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException
      	at hudson.remoting.Request$1.get(Request.java:218)
      	at hudson.remoting.Request$1.get(Request.java:172)
      	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
      	at hudson.Proc$RemoteProc.join(Proc.java:303)
      	... 12 more
      Caused by: hudson.remoting.RequestAbortedException: java.io.EOFException
      	at hudson.remoting.Request.abort(Request.java:257)
      	at hudson.remoting.Channel.terminate(Channel.java:594)
      	at hudson.remoting.Channel$ReaderThread.run(Channel.java:872)
      Caused by: java.io.EOFException
      	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
      	at java.io.ObjectInputStream.readObject0(Unknown Source)
      	at java.io.ObjectInputStream.readObject(Unknown Source)
      	at hudson.remoting.Channel$ReaderThread.run(Channel.java:852)
      FATAL: Unable to delete script file C:\DOCUME~1\pcfarm08\LOCALS~1\Temp\2\hudson2101488692771117152.bat
      hudson.util.IOException2: remote file operation failed
      	at hudson.FilePath.act(FilePath.java:677)
      	at hudson.FilePath.act(FilePath.java:665)
      	at hudson.FilePath.delete(FilePath.java:922)
      	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93)
      	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
      	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:577)
      	at hudson.model.Build$RunnerImpl.build(Build.java:165)
      	at hudson.model.Build$RunnerImpl.doRun(Build.java:133)
      	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:417)
      	at hudson.model.Run.run(Run.java:1176)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:123)
      Caused by: hudson.remoting.ChannelClosedException: channel is already closed
      	at hudson.remoting.Channel.send(Channel.java:408)
      	at hudson.remoting.Request.call(Request.java:105)
      	at hudson.remoting.Channel.call(Channel.java:547)
      	at hudson.FilePath.act(FilePath.java:672)
      	... 13 more
      [locks-and-latches] Releasing all the locks
      [locks-and-latches] All the locks released
      FATAL: channel is already closed
      hudson.remoting.ChannelClosedException: channel is already closed
      	at hudson.remoting.Channel.send(Channel.java:408)
      	at hudson.remoting.Request.call(Request.java:105)
      	at hudson.remoting.Channel.call(Channel.java:547)
      	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:734)
      	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:422)
      	at hudson.model.Run.run(Run.java:1176)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:123)
      

      The job executes a batch command and it tooks 1h 17 min to fail in Hudson 1.336.

        Issue Links

          Activity

          Hide
          deccico deccico added a comment -

          Hi, we noticed that this happens just in a job that execute a job with a duration of around 2hours that has a step with no output in a long time (don't know exactly how long)

          thanks

          Show
          deccico deccico added a comment - Hi, we noticed that this happens just in a job that execute a job with a duration of around 2hours that has a step with no output in a long time (don't know exactly how long) thanks
          Hide
          erwan_q erwan_q added a comment -

          I have the same problem on other platform like ibm aix and with hudson 1.338 and with jdk 1.6
          Seems to be a general bug:
          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          at hudson.Proc$RemoteProc.join(Proc.java:311)
          at hudson.Launcher$ProcStarter.join(Launcher.java:275)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:583)
          at hudson.model.Build$RunnerImpl.build(Build.java:165)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:133)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:417)
          at hudson.model.Run.run(Run.java:1179)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:122)
          Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: hudson.remoting.ProxyOutputStream$Flush
          at hudson.remoting.Channel$1.adapt(Channel.java:580)
          at hudson.remoting.Channel$1.adapt(Channel.java:575)
          at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          at hudson.Proc$RemoteProc.join(Proc.java:303)
          ... 12 more
          Caused by: java.lang.NoClassDefFoundError: hudson.remoting.ProxyOutputStream$Flush
          at java.lang.J9VMInternals.verifyImpl(Native Method)
          at java.lang.J9VMInternals.verify(J9VMInternals.java:72)
          at java.lang.J9VMInternals.initialize(J9VMInternals.java:134)
          at hudson.remoting.RemoteOutputStream.readObject(RemoteOutputStream.java:90)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
          at java.lang.reflect.Method.invoke(Method.java:600)
          at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:986)
          at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1865)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)
          at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1963)
          at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1887)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:364)
          at hudson.remoting.UserRequest.deserialize(UserRequest.java:168)
          at hudson.remoting.UserRequest.perform(UserRequest.java:88)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:270)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:453)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:315)
          at java.util.concurrent.FutureTask.run(FutureTask.java:150)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:736)
          Caused by: java.lang.ClassNotFoundException: hudson.remoting.ProxyOutputStream$Flush
          at java.net.URLClassLoader.findClass(URLClassLoader.java:421)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:643)
          at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:346)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:609)
          ... 27 more
          Notifying upstream projects of job completion
          Finished: FAILURE

          Show
          erwan_q erwan_q added a comment - I have the same problem on other platform like ibm aix and with hudson 1.338 and with jdk 1.6 Seems to be a general bug: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:311) at hudson.Launcher$ProcStarter.join(Launcher.java:275) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:583) at hudson.model.Build$RunnerImpl.build(Build.java:165) at hudson.model.Build$RunnerImpl.doRun(Build.java:133) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:417) at hudson.model.Run.run(Run.java:1179) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: hudson.remoting.ProxyOutputStream$Flush at hudson.remoting.Channel$1.adapt(Channel.java:580) at hudson.remoting.Channel$1.adapt(Channel.java:575) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:303) ... 12 more Caused by: java.lang.NoClassDefFoundError: hudson.remoting.ProxyOutputStream$Flush at java.lang.J9VMInternals.verifyImpl(Native Method) at java.lang.J9VMInternals.verify(J9VMInternals.java:72) at java.lang.J9VMInternals.initialize(J9VMInternals.java:134) at hudson.remoting.RemoteOutputStream.readObject(RemoteOutputStream.java:90) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:600) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:986) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1865) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1963) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1887) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:364) at hudson.remoting.UserRequest.deserialize(UserRequest.java:168) at hudson.remoting.UserRequest.perform(UserRequest.java:88) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:270) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:453) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:315) at java.util.concurrent.FutureTask.run(FutureTask.java:150) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:736) Caused by: java.lang.ClassNotFoundException: hudson.remoting.ProxyOutputStream$Flush at java.net.URLClassLoader.findClass(URLClassLoader.java:421) at java.lang.ClassLoader.loadClass(ClassLoader.java:643) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:346) at java.lang.ClassLoader.loadClass(ClassLoader.java:609) ... 27 more Notifying upstream projects of job completion Finished: FAILURE
          Hide
          erwan_q erwan_q added a comment -

          Sorry it's not exactly the same issue.
          Your connection problem was linked to:
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException
          And my connection problem was linked to:
          Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: hudson.remoting.ProxyOutputStream$Flush

          Show
          erwan_q erwan_q added a comment - Sorry it's not exactly the same issue. Your connection problem was linked to: Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException And my connection problem was linked to: Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: hudson.remoting.ProxyOutputStream$Flush
          Hide
          mdonohue mdonohue added a comment -

          @erwan_q
          Your issue looks very similar to JENKINS-5141. I'd suggest commenting there.

          Show
          mdonohue mdonohue added a comment - @erwan_q Your issue looks very similar to JENKINS-5141 . I'd suggest commenting there.
          Hide
          erwan_q erwan_q added a comment -

          Thanks for your comment.
          The Hudson-5141 is more oriented on "ClassNotFoundException: hudson.model.job" i think.
          My problem is more on the hudson.remoting.ProxyOutputStream$Flush
          I created a new bug JENKINS-5149 on this specific issue.
          I'm trying to find a solution to have a stable env but no success

          Show
          erwan_q erwan_q added a comment - Thanks for your comment. The Hudson-5141 is more oriented on "ClassNotFoundException: hudson.model.job" i think. My problem is more on the hudson.remoting.ProxyOutputStream$Flush I created a new bug JENKINS-5149 on this specific issue. I'm trying to find a solution to have a stable env but no success
          Hide
          sini_m sini_m added a comment -

          We have encountered this same exact issue with Windows XP master/slave (Hudson 1.342). It occurs about 2 hours into build process, but not consistently. Usually builds of the same project do succeed without any problems. Here's a snippet of our failures (omitting some of the lines, since it is exactly the same as in the original post).

          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          at hudson.Proc$RemoteProc.join(Proc.java:312)
          at hudson.Launcher$ProcStarter.join(Launcher.java:275)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:582)
          at hudson.model.Build$RunnerImpl.build(Build.java:165)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:132)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          at hudson.model.Run.run(Run.java:1198)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:122)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException
          ...
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          at hudson.remoting.Channel.send(Channel.java:412)
          at hudson.remoting.Request.call(Request.java:105)
          at hudson.remoting.Channel.call(Channel.java:551)
          at hudson.Launcher$RemoteLauncher.kill(Launcher.java:735)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          at hudson.model.Run.run(Run.java:1198)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:122)

          @decicco: Thanks for the tip about lack of output, will see if adding some output (if possible) would help.

          Show
          sini_m sini_m added a comment - We have encountered this same exact issue with Windows XP master/slave (Hudson 1.342). It occurs about 2 hours into build process, but not consistently. Usually builds of the same project do succeed without any problems. Here's a snippet of our failures (omitting some of the lines, since it is exactly the same as in the original post). FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:312) at hudson.Launcher$ProcStarter.join(Launcher.java:275) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:582) at hudson.model.Build$RunnerImpl.build(Build.java:165) at hudson.model.Build$RunnerImpl.doRun(Build.java:132) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1198) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException ... FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:551) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:735) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1198) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) @decicco: Thanks for the tip about lack of output, will see if adding some output (if possible) would help.
          Hide
          levante levante added a comment -

          We've seen similar join related issues on our Hudson deployment (master/slaves all running 64-bit Windows 2003 Server, slaves as Windows Service).

          Usually happens on a job running for a little longer than an hour. Latest Hudson version I can report this happening is 1.356.

          Output:

          FATAL: command execution failed.
          hudson.util.IOException2: Failed to join the process
          at hudson.Proc$RemoteProc.join(Proc.java:312)
          at hudson.Launcher$ProcStarter.join(Launcher.java:278)
          at hudson.tasks.Ant.perform(Ant.java:212)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:600)
          at hudson.model.Build$RunnerImpl.build(Build.java:174)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          at hudson.model.Run.run(Run.java:1244)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:122)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
          at hudson.remoting.Request$1.get(Request.java:218)
          at hudson.remoting.Request$1.get(Request.java:172)
          at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          at hudson.Proc$RemoteProc.join(Proc.java:304)
          ... 11 more
          Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
          at hudson.remoting.Request.abort(Request.java:257)
          at hudson.remoting.Channel.terminate(Channel.java:598)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:880)
          Caused by: java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(Unknown Source)
          at java.io.FilterInputStream.read(Unknown Source)
          at java.io.BufferedInputStream.fill(Unknown Source)
          at java.io.BufferedInputStream.read(Unknown Source)
          at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)
          at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
          at java.io.ObjectInputStream.readObject0(Unknown Source)
          at java.io.ObjectInputStream.readObject(Unknown Source)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:856)
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          at hudson.remoting.Channel.send(Channel.java:412)
          at hudson.remoting.Request.call(Request.java:105)
          at hudson.remoting.Channel.call(Channel.java:551)
          at hudson.Launcher$RemoteLauncher.kill(Launcher.java:738)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          at hudson.model.Run.run(Run.java:1244)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:122)

          Is this a problem with Hudson or the Join plugin?

          Show
          levante levante added a comment - We've seen similar join related issues on our Hudson deployment (master/slaves all running 64-bit Windows 2003 Server, slaves as Windows Service). Usually happens on a job running for a little longer than an hour. Latest Hudson version I can report this happening is 1.356. Output: FATAL: command execution failed. hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:312) at hudson.Launcher$ProcStarter.join(Launcher.java:278) at hudson.tasks.Ant.perform(Ant.java:212) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:600) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1244) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:304) ... 11 more Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:598) at hudson.remoting.Channel$ReaderThread.run(Channel.java:880) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(Unknown Source) at java.io.FilterInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Channel$ReaderThread.run(Channel.java:856) FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:551) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:738) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1244) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) Is this a problem with Hudson or the Join plugin?
          Hide
          sbechter sbechter added a comment -

          Same problem on ubuntu linux.

           
          hudson.util.IOException2: Failed to join the process
                  at hudson.Proc$RemoteProc.join(Proc.java:312)
                  at hudson.Launcher$ProcStarter.join(Launcher.java:278)
                  at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83)
                  at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
                  at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
                  at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:584)
                  at hudson.model.Build$RunnerImpl.build(Build.java:174)
                  at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
                  at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
                  at hudson.model.Run.run(Run.java:1244)
                  at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
                  at hudson.model.ResourceController.execute(ResourceController.java:88)
                  at hudson.model.Executor.run(Executor.java:122)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
                  at hudson.remoting.Request$1.get(Request.java:218)
                  at hudson.remoting.Request$1.get(Request.java:172)
                  at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
                  at hudson.Proc$RemoteProc.join(Proc.java:304)
                  ... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
                  at hudson.remoting.Request.abort(Request.java:257)
                  at hudson.remoting.Channel.terminate(Channel.java:598)
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:880)
          Caused by: java.io.IOException: Unexpected termination of the channel
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:862)
          Caused by: java.io.EOFException
                  at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
                  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
                  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:856)
          FATAL: Unable to delete script file /tmp/hudson4172490882779075596.sh
          hudson.util.IOException2: remote file operation failed: /tmp/hudson4172490882779075596.sh at hudson.remoting.Channel@4b4e563e:hudson_slave
                  at hudson.FilePath.act(FilePath.java:743)
                  at hudson.FilePath.act(FilePath.java:729)
                  at hudson.FilePath.delete(FilePath.java:984)
                  at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93)
                  at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
                  at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
                  at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:584)
                  at hudson.model.Build$RunnerImpl.build(Build.java:174)
                  at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
                  at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
                  at hudson.model.Run.run(Run.java:1244)
                  at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
                  at hudson.model.ResourceController.execute(ResourceController.java:88)
                  at hudson.model.Executor.run(Executor.java:122)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
                  at hudson.remoting.Channel.send(Channel.java:412)
                  at hudson.remoting.Request.call(Request.java:105)
                  at hudson.remoting.Channel.call(Channel.java:551)
                  at hudson.FilePath.act(FilePath.java:736)
                  ... 13 more
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
                  at hudson.remoting.Channel.send(Channel.java:412)
                  at hudson.remoting.Request.call(Request.java:105)
                  at hudson.remoting.Channel.call(Channel.java:551)
                  at hudson.Launcher$RemoteLauncher.kill(Launcher.java:738)
                  at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
                  at hudson.model.Run.run(Run.java:1244)
                  at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
                  at hudson.model.ResourceController.execute(ResourceController.java:88)
                  at hudson.model.Executor.run(Executor.java:122)
          
          Show
          sbechter sbechter added a comment - Same problem on ubuntu linux. hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:312) at hudson.Launcher$ProcStarter.join(Launcher.java:278) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:584) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1244) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:304) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:598) at hudson.remoting.Channel$ReaderThread.run(Channel.java:880) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:862) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:856) FATAL: Unable to delete script file /tmp/hudson4172490882779075596.sh hudson.util.IOException2: remote file operation failed: /tmp/hudson4172490882779075596.sh at hudson.remoting.Channel@4b4e563e:hudson_slave at hudson.FilePath.act(FilePath.java:743) at hudson.FilePath.act(FilePath.java:729) at hudson.FilePath.delete(FilePath.java:984) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:584) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1244) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:551) at hudson.FilePath.act(FilePath.java:736) ... 13 more FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:551) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:738) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1244) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122)
          Hide
          davida2009 davida2009 added a comment -

          I occasionally get a similar error when Hudson checks out a large svn project. It happens at night when system backups are also happening on the network, so the check out may take a very long time.

          I am running Hudson v.1.360. Master runs on Windows XP, slaves are Linux. Failure seen on jobs running on master or slave.

          ERROR: Failed to check out https://<snip>
          org.tmatesoft.svn.core.SVNException: svn: SSL peer shut down incorrectly
          svn: REPORT request failed on '/subversion/zodiac/!svn/vcc/default'
          at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:103)
          at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:87)
          at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:616)
          at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:273)
          at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:261)
          at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:266)
          at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1263)
          at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:820)
          at org.tmatesoft.svn.core.wc.SVNUpdateClient.update(SVNUpdateClient.java:558)
          at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:934)
          at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:742)
          at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:660)
          at hudson.FilePath.act(FilePath.java:753)
          at hudson.FilePath.act(FilePath.java:735)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:653)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:601)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1044)
          at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411)
          at hudson.model.Run.run(Run.java:1241)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:124)
          Caused by: javax.net.ssl.SSLException: SSL peer shut down incorrectly
          at com.sun.net.ssl.internal.ssl.InputRecord.readV3Record(Unknown Source)
          at com.sun.net.ssl.internal.ssl.InputRecord.read(Unknown Source)
          at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
          at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
          at com.sun.net.ssl.internal.ssl.AppInputStream.read(Unknown Source)
          at java.io.BufferedInputStream.read1(Unknown Source)
          at java.io.BufferedInputStream.read(Unknown Source)
          at org.tmatesoft.svn.core.internal.util.ChunkedInputStream.read(ChunkedInputStream.java:70)
          at sun.nio.cs.StreamDecoder.readBytes(Unknown Source)
          at sun.nio.cs.StreamDecoder.implRead(Unknown Source)
          at sun.nio.cs.StreamDecoder.read(Unknown Source)
          at java.io.InputStreamReader.read(Unknown Source)
          at org.tmatesoft.svn.core.internal.io.dav.http.XMLReader.read(XMLReader.java:39)
          at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipChar(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
          at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
          at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
          at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
          at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
          at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
          at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:726)
          at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:691)
          at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:216)
          at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:345)
          ... 20 more
          Email was triggered for: Failure
          Sending email for trigger: Failure

          Show
          davida2009 davida2009 added a comment - I occasionally get a similar error when Hudson checks out a large svn project. It happens at night when system backups are also happening on the network, so the check out may take a very long time. I am running Hudson v.1.360. Master runs on Windows XP, slaves are Linux. Failure seen on jobs running on master or slave. ERROR: Failed to check out https://<snip> org.tmatesoft.svn.core.SVNException: svn: SSL peer shut down incorrectly svn: REPORT request failed on '/subversion/zodiac/!svn/vcc/default' at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:103) at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:87) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:616) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:273) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:261) at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:266) at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1263) at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:820) at org.tmatesoft.svn.core.wc.SVNUpdateClient.update(SVNUpdateClient.java:558) at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:934) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:742) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:660) at hudson.FilePath.act(FilePath.java:753) at hudson.FilePath.act(FilePath.java:735) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:653) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:601) at hudson.model.AbstractProject.checkout(AbstractProject.java:1044) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411) at hudson.model.Run.run(Run.java:1241) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:124) Caused by: javax.net.ssl.SSLException: SSL peer shut down incorrectly at com.sun.net.ssl.internal.ssl.InputRecord.readV3Record(Unknown Source) at com.sun.net.ssl.internal.ssl.InputRecord.read(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(Unknown Source) at com.sun.net.ssl.internal.ssl.AppInputStream.read(Unknown Source) at java.io.BufferedInputStream.read1(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at org.tmatesoft.svn.core.internal.util.ChunkedInputStream.read(ChunkedInputStream.java:70) at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) at sun.nio.cs.StreamDecoder.implRead(Unknown Source) at sun.nio.cs.StreamDecoder.read(Unknown Source) at java.io.InputStreamReader.read(Unknown Source) at org.tmatesoft.svn.core.internal.io.dav.http.XMLReader.read(XMLReader.java:39) at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipChar(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:726) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:691) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:216) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:345) ... 20 more Email was triggered for: Failure Sending email for trigger: Failure
          Hide
          nirmal_patel nirmal_patel added a comment - - edited

          I am seeing the same on my Windows XP master-slave setup. I am running latest Hudson ver. 1.363
          I am using the close-workspace-scm plugin to copy my workspace from master to slave(150).

          Started by user anonymous
          Building remotely on 150
          FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Request.call(Request.java:137)
          at hudson.remoting.Channel.call(Channel.java:555)
          at hudson.FilePath.act(FilePath.java:742)
          at hudson.FilePath.act(FilePath.java:735)
          at hudson.FilePath.unzip(FilePath.java:415)
          at hudson.FileSystemProvisioner$Default$WorkspaceSnapshotImpl.restoreTo(FileSystemProvisioner.java:227)
          at hudson.plugins.cloneworkspace.CloneWorkspaceSCM$Snapshot.restoreTo(CloneWorkspaceSCM.java:344)
          at hudson.plugins.cloneworkspace.CloneWorkspaceSCM.checkout(CloneWorkspaceSCM.java:126)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1044)
          at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411)
          at hudson.model.Run.run(Run.java:1253)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:127)
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Request.abort(Request.java:257)
          at hudson.remoting.Channel.terminate(Channel.java:602)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:893)
          Caused by: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:875)
          Caused by: java.io.EOFException
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
          at java.io.ObjectInputStream.readObject0(Unknown Source)
          at java.io.ObjectInputStream.readObject(Unknown Source)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:869)

          Show
          nirmal_patel nirmal_patel added a comment - - edited I am seeing the same on my Windows XP master-slave setup. I am running latest Hudson ver. 1.363 I am using the close-workspace-scm plugin to copy my workspace from master to slave(150). Started by user anonymous Building remotely on 150 FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.call(Request.java:137) at hudson.remoting.Channel.call(Channel.java:555) at hudson.FilePath.act(FilePath.java:742) at hudson.FilePath.act(FilePath.java:735) at hudson.FilePath.unzip(FilePath.java:415) at hudson.FileSystemProvisioner$Default$WorkspaceSnapshotImpl.restoreTo(FileSystemProvisioner.java:227) at hudson.plugins.cloneworkspace.CloneWorkspaceSCM$Snapshot.restoreTo(CloneWorkspaceSCM.java:344) at hudson.plugins.cloneworkspace.CloneWorkspaceSCM.checkout(CloneWorkspaceSCM.java:126) at hudson.model.AbstractProject.checkout(AbstractProject.java:1044) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411) at hudson.model.Run.run(Run.java:1253) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:127) Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:602) at hudson.remoting.Channel$ReaderThread.run(Channel.java:893) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:875) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Channel$ReaderThread.run(Channel.java:869)
          Hide
          williamleara williamleara added a comment -

          I think I'm seeing the same thing, at least the same thing as levante reported.

          Hudson 1.362 in Tomcat 5.5 on Server 2003 32-bit, JRE 1.6. Slave is Server 2008 R2, 64-bit, JRE 1.6, connected via hudsonslave service. When a lot of jobs come in at once the hudsonslave service crashes.

          hudson.util.IOException2: Failed to join the process
          at hudson.Proc$RemoteProc.join(Proc.java:312)
          at hudson.Launcher$ProcStarter.join(Launcher.java:280)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          at hudson.model.Build$RunnerImpl.build(Build.java:174)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          at hudson.model.Run.run(Run.java:1253)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:124)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
          at hudson.remoting.Request$1.get(Request.java:218)
          at hudson.remoting.Request$1.get(Request.java:172)
          at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          at hudson.Proc$RemoteProc.join(Proc.java:304)
          ... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
          at hudson.remoting.Request.abort(Request.java:257)
          at hudson.remoting.Channel.terminate(Channel.java:602)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:893)
          Caused by: java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(Unknown Source)
          at java.io.BufferedInputStream.fill(Unknown Source)
          at java.io.BufferedInputStream.read(Unknown Source)
          at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)
          at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
          at java.io.ObjectInputStream.readObject0(Unknown Source)
          at java.io.ObjectInputStream.readObject(Unknown Source)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:869)
          FATAL: Unable to delete script file C:\Windows\TEMP\hudson750406825412804255.bat
          hudson.util.IOException2: remote file operation failed: C:\Windows\TEMP\hudson750406825412804255.bat at hudson.remoting.Channel@1db6bbe:weyoun1
          at hudson.FilePath.act(FilePath.java:749)
          at hudson.FilePath.act(FilePath.java:735)
          at hudson.FilePath.delete(FilePath.java:990)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          at hudson.model.Build$RunnerImpl.build(Build.java:174)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          at hudson.model.Run.run(Run.java:1253)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:124)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          at hudson.remoting.Channel.send(Channel.java:412)
          at hudson.remoting.Request.call(Request.java:105)
          at hudson.remoting.Channel.call(Channel.java:555)
          at hudson.FilePath.act(FilePath.java:742)
          ... 13 more
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          at hudson.remoting.Channel.send(Channel.java:412)
          at hudson.remoting.Request.call(Request.java:105)
          at hudson.remoting.Channel.call(Channel.java:555)
          at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          at hudson.model.Run.run(Run.java:1253)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:124)

          Show
          williamleara williamleara added a comment - I think I'm seeing the same thing, at least the same thing as levante reported. Hudson 1.362 in Tomcat 5.5 on Server 2003 32-bit, JRE 1.6. Slave is Server 2008 R2, 64-bit, JRE 1.6, connected via hudsonslave service. When a lot of jobs come in at once the hudsonslave service crashes. hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:312) at hudson.Launcher$ProcStarter.join(Launcher.java:280) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1253) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:124) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:304) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:602) at hudson.remoting.Channel$ReaderThread.run(Channel.java:893) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Channel$ReaderThread.run(Channel.java:869) FATAL: Unable to delete script file C:\Windows\TEMP\hudson750406825412804255.bat hudson.util.IOException2: remote file operation failed: C:\Windows\TEMP\hudson750406825412804255.bat at hudson.remoting.Channel@1db6bbe:weyoun1 at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.FilePath.delete(FilePath.java:990) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1253) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:124) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:555) at hudson.FilePath.act(FilePath.java:742) ... 13 more FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:555) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1253) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:124)
          Hide
          gkesavan gkesavan added a comment -

          most of the Apache Hadoop projects started hitting the same issues.

          FATAL: Unable to delete script file /tmp/hudson6048046841607430352.sh
          hudson.util.IOException2: remote file operation failed: /tmp/hudson6048046841607430352.sh at hudson.remoting.Channel@af5ea0: (Ubuntu)
          at hudson.FilePath.act(FilePath.java:743)
          at hudson.FilePath.act(FilePath.java:729)
          at hudson.FilePath.delete(FilePath.java:984)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

          Show
          gkesavan gkesavan added a comment - most of the Apache Hadoop projects started hitting the same issues. FATAL: Unable to delete script file /tmp/hudson6048046841607430352.sh hudson.util.IOException2: remote file operation failed: /tmp/hudson6048046841607430352.sh at hudson.remoting.Channel@af5ea0: (Ubuntu) at hudson.FilePath.act(FilePath.java:743) at hudson.FilePath.act(FilePath.java:729) at hudson.FilePath.delete(FilePath.java:984) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          When this error happens, what does the slave log report?

          Show
          kohsuke Kohsuke Kawaguchi added a comment - When this error happens, what does the slave log report?
          Hide
          gkesavan gkesavan added a comment -

          I click on the log like on the node and here is the info that I got.

          [07/13/10 16:55:37] Launching slave agent
          $ ssh <hostname>/homes/hudson/tools/java/latest/bin/java -Xmx512m -jar /grid/0/hudson/hudson-slave/slave.jar
          <===[HUDSON REMOTING CAPACITY]===>���channel started
          Slave.jar version: 1.364
          This is a Unix slave
          Copied maven-agent.jar
          Copied maven-interceptor.jar
          Copied maven2.1-interceptor.jar
          Slave successfully connected and online
          Jul 13, 2010 4:55:43 PM com.youdevise.hudson.slavestatus.SlaveListener call
          INFO: Slave-status listener starting
          Jul 13, 2010 4:55:44 PM com.youdevise.hudson.slavestatus.SocketHTTPListener waitForConnection
          INFO: Slave-status listener ready on port 3141

          Show
          gkesavan gkesavan added a comment - I click on the log like on the node and here is the info that I got. [07/13/10 16:55:37] Launching slave agent $ ssh <hostname>/homes/hudson/tools/java/latest/bin/java -Xmx512m -jar /grid/0/hudson/hudson-slave/slave.jar <=== [HUDSON REMOTING CAPACITY] ===>���channel started Slave.jar version: 1.364 This is a Unix slave Copied maven-agent.jar Copied maven-interceptor.jar Copied maven2.1-interceptor.jar Slave successfully connected and online Jul 13, 2010 4:55:43 PM com.youdevise.hudson.slavestatus.SlaveListener call INFO: Slave-status listener starting Jul 13, 2010 4:55:44 PM com.youdevise.hudson.slavestatus.SocketHTTPListener waitForConnection INFO: Slave-status listener ready on port 3141
          Hide
          linustolke Linus Tolke added a comment - - edited

          I notice the same (or a similar) problem on my Debian/Linux host and Hudson 1.357. I have the remote job running on the same host started with the Launch slave via execution of command on the master:

          nice -n 5 /usr/local/lib/java/jdk1.6.0/bin/java -Xdebug -Xrunjdwp:transport=dt_socket,address=8003,server=y,suspend=n -Xms16M -jar /home/argonightly/tools/WEB-INF/slave.jar
          

          (The same problems occur also without the -Xdebug -Xrunjdwp... arguments). The purpose of starting in this way is to get some long-running jobs to run with a lower nice to allow other jobs to finish quicker.

          For me this has only happened when there are two jobs running on the slave. Both jobs fail with the same error message at the same time. Then the slave(s) is restarted and new jobs are allocated on the slave. I have previously noticed that the jobs on the old slave is still running so I assume that it is just the connection between the master and the slave that is the problem.

          Today I have noticed this twice and managed to capture the output from the master hudson. The first time with two jobs running on one slave and probably none on the other, the second time with just one of the slaves without any jobs running.

          Here is the first time with the output from the master. Notice that both slaves (updateshow and nightly) are crashing, taken down, or failing at the same time. You can also notice that I run the master hudson with the -verbosegc argument.

          [GC 127674K->117585K(160932K), 9.4880730 secs]
          Jul 22, 2010 6:35:43 AM hudson.model.Run run
          INFO: update-mirror #30764 main build action completed: SUCCESS
          [GC 127697K->117678K(160932K), 6.2626260 secs]
          Jul 22, 2010 6:44:51 AM hudson.remoting.Channel$ReaderThread run
          SEVERE: I/O error in channel updateshow
          java.io.IOException: Unexpected termination of the channel
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:862)
          Caused by: java.io.EOFException
                  at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputSt
          ream.java:2552)
                  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
                  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:856)
          Jul 22, 2010 6:44:57 AM hudson.remoting.Channel$ReaderThread run
          SEVERE: I/O error in channel nightly
          java.io.IOException: Unexpected termination of the channel
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:862)
          Caused by: java.io.EOFException
                  at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputSt
          ream.java:2552)
                  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
                  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:856)
          Jul 22, 2010 6:44:57 AM hudson.remoting.Request$2 run
          SEVERE: Failed to send back a reply
          java.io.IOException: Broken pipe
                  at java.io.FileOutputStream.writeBytes(Native Method)
                  at java.io.FileOutputStream.write(FileOutputStream.java:260)
                  at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65
          )
                  at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
                  at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputSt
          ream.java:1784)
                  at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:691)
                  at hudson.remoting.Channel.send(Channel.java:418)
                  at hudson.remoting.Request$2.run(Request.java:282)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:44
          1)
                  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
                  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
          utor.java:885)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
          .java:907)
                  at java.lang.Thread.run(Thread.java:619)
          Jul 22, 2010 6:45:03 AM hudson.slaves.SlaveComputer tryReconnect
          INFO: Attempting to reconnect nightly
          Jul 22, 2010 6:45:04 AM hudson.slaves.SlaveComputer tryReconnect
          INFO: Attempting to reconnect updateshow
          [GC 127787K->118134K(160932K), 6.6658860 secs]
          

          This is the backtrace from the log of one of the jobs on the nightly slave:

          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          	at hudson.Proc$RemoteProc.join(Proc.java:312)
          	at hudson.Launcher$ProcStarter.join(Launcher.java:278)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:600)
          	at hudson.model.Build$RunnerImpl.build(Build.java:174)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1241)
          	at hudson.matrix.MatrixRun.run(MatrixRun.java:130)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:122)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request$1.get(Request.java:218)
          	at hudson.remoting.Request$1.get(Request.java:172)
          	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          	at hudson.Proc$RemoteProc.join(Proc.java:304)
          	... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request.abort(Request.java:257)
          	at hudson.remoting.Channel.terminate(Channel.java:598)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:880)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:862)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:856)
          FATAL: Unable to delete script file /tmp/hudson18796.sh
          hudson.util.IOException2: remote file operation failed: /tmp/hudson18796.sh at hudson.remoting.Channel@2e646f:nightly
          	at hudson.FilePath.act(FilePath.java:752)
          	at hudson.FilePath.act(FilePath.java:738)
          	at hudson.FilePath.delete(FilePath.java:993)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:600)
          	at hudson.model.Build$RunnerImpl.build(Build.java:174)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1241)
          	at hudson.matrix.MatrixRun.run(MatrixRun.java:130)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:122)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:412)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:551)
          	at hudson.FilePath.act(FilePath.java:745)
          	... 13 more
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:412)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:551)
          	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:738)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          	at hudson.model.Run.run(Run.java:1241)
          	at hudson.matrix.MatrixRun.run(MatrixRun.java:130)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:122)
          

          The second time today only the updateshow job got this problem and I cannot find any failed jobs. The jobs on updateshow are a lot shorter (around 5m). Here is the output from the master process:

          [GC 151557K->148710K(160932K), 4.0233760 secs]
          Jul 22, 2010 12:23:09 PM hudson.model.Run run
          INFO: update-mirror #30799 main build action completed: SUCCESS
          [GC 158822K->150464K(161356K), 1.0420050 secs]
          [Full GC[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14
          720]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14411]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14660]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14581]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14459]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14746]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14554]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14357]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13929]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14437]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14747]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14707]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14743]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14384]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14754]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14496]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14383]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14508]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14664]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14346]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14468]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14397]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14712]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14549]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14419]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14671]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14620]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14365]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14328]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14429]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14415]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14408]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13652]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14593]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14456]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14425]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14609]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14636]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14574]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14626]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14539]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14441]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14568]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14639]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14694]
          [Unloading class sun.reflect.GeneratedConstructorAccessor334]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14667]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14577]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14550]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14650]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14351]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14633]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14744]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14364]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14725]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14356]
          [Unloading class sun.reflect.GeneratedMethodAccessor1350]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13782]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13807]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14463]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14360]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14576]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14560]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14515]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14462]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14349]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14489]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14647]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14579]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14538]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14524]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14482]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14445]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14398]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14402]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14358]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14651]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14627]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14421]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14400]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14391]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14432]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14635]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13571]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14481]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14572]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14734]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14336]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14440]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14467]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14732]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14713]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14629]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14377]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14715]
          [Unloading class sun.reflect.GeneratedConstructorAccessor333]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14632]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14724]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14418]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14617]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14361]
          [Unloading class sun.reflect.GeneratedConstructorAccessor322]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14460]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14392]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14599]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14691]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14368]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14613]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14705]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14587]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14511]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13936]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14396]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14696]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14634]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14548]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14700]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14388]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14504]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14491]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14678]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14645]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13930]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14733]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14592]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14444]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14433]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14665]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14551]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14479]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14730]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14615]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14589]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14355]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14472]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14708]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14662]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14683]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14507]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14521]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14470]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14393]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14422]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14420]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14674]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14637]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14476]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14716]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14477]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14352]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13805]
          [Unloading class sun.reflect.GeneratedMethodAccessor1352]
          [Unloading class sun.reflect.GeneratedConstructorAccessor327]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14516]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14403]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14719]
          [Unloading class sun.reflect.GeneratedConstructorAccessor329]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14416]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14466]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14692]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14711]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14565]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14685]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14698]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14677]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14575]
          [Unloading class sun.reflect.GeneratedConstructorAccessor332]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14333]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14571]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14412]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14717]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14679]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14654]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14638]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14652]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14604]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14457]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14534]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13586]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14329]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14409]
          [Unloading class sun.reflect.GeneratedMethodAccessor1355]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14449]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14544]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14424]
          [Unloading class sun.reflect.GeneratedConstructorAccessor324]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14385]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14703]
          [Unloading class sun.reflect.GeneratedConstructorAccessor331]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14343]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14621]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14531]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14379]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14542]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14526]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14518]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14506]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14666]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14564]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14656]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14390]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14598]
          [Unloading class sun.reflect.GeneratedConstructorAccessor325]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14567]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14453]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14488]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14625]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14528]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14502]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14558]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14741]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14668]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14586]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14438]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14670]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14541]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14375]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14641]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14646]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14596]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14728]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14367]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14439]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14597]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14736]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14451]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13538]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14490]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14382]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14584]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14347]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14628]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14704]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14749]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13937]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14378]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14423]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14658]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14386]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14399]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14697]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14350]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14553]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14622]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14461]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14417]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14503]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14687]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14381]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14731]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14591]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14748]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14475]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14448]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14695]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14362]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14370]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14458]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14363]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14582]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14753]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14334]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14478]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14610]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14492]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14480]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14513]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14473]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14607]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13935]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14330]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14585]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13580]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14659]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14573]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14689]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14557]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14332]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14532]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14739]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14588]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13583]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14680]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14605]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14616]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14623]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14369]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14341]
          [Unloading class sun.reflect.GeneratedMethodAccessor1351]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14738]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14514]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14436]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14465]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14642]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14740]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14723]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14742]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14520]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14500]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14517]
          [Unloading class sun.reflect.GeneratedConstructorAccessor321]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14452]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14331]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14547]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14373]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14701]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14663]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14600]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14606]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14603]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14522]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14340]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14485]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13581]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14527]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14619]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14474]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14413]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14611]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14630]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14454]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14446]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14536]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14512]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14750]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedMethodAccessor1294]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14543]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14669]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13582]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14435]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14583]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14407]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14608]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14430]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14427]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14612]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14426]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14394]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14344]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14745]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14519]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14471]
          [Unloading class sun.reflect.GeneratedConstructorAccessor326]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14672]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14682]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14673]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14359]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14580]
          [Unloading class sun.reflect.GeneratedConstructorAccessor328]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14643]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14497]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14410]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14722]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14533]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14540]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14354]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14469]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14631]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14537]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14595]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14688]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14661]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14601]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14702]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14487]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14690]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14376]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14374]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13579]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14594]
          [Unloading class Script1]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13938]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14483]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14484]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14525]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14751]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14556]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14464]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14428]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14505]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14501]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14486]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14348]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14405]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14561]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14726]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14676]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14675]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14530]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14335]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14590]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14699]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14578]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14566]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14395]
          [Unloading class sun.reflect.GeneratedMethodAccessor1354]
          [Unloading class sun.reflect.GeneratedMethodAccessor1290]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14510]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14404]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14727]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14406]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14618]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14657]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14494]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13534]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14681]
          [Unloading class sun.reflect.GeneratedMethodAccessor1298]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14755]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14366]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14570]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14545]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14353]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14546]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14431]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14710]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14718]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14380]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14752]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14648]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14552]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14499]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14706]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14714]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14342]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14640]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14602]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14559]
          [Unloading class sun.reflect.GeneratedConstructorAccessor330]
          [Unloading class sun.reflect.GeneratedConstructorAccessor302]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14569]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14414]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14447]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14372]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14614]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14523]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13769]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14493]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14684]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14371]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14649]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14562]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14450]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14389]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14729]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14535]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14655]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14624]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14509]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14737]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14735]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13806]
          [Unloading class Script1$_run_closure1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14721]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14442]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14653]
          [Unloading class sun.reflect.GeneratedConstructorAccessor323]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14345]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14555]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14455]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14434]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14387]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14529]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14498]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14495]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14563]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14443]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14693]
          [Unloading class Script1]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14401]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14644]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13931]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14686]
          [Unloading class sun.reflect.GeneratedMethodAccessor1356]
          [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14709]
           150464K->51729K(161356K), 225.1859240 secs]
          Jul 22, 2010 12:26:56 PM hudson.remoting.Channel$ReaderThread run
          SEVERE: I/O error in channel updateshow
          java.io.IOException: Unexpected termination of the channel
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:862)
          Caused by: java.io.EOFException
                  at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputSt
          ream.java:2552)
                  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
                  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
                  at hudson.remoting.Channel$ReaderThread.run(Channel.java:856)
          [GC 61841K->52274K(160932K), 0.0367730 secs]
          [GC 62386K->52083K(160932K), 0.0317730 secs]
          Jul 22, 2010 12:28:38 PM hudson.slaves.SlaveComputer tryReconnect
          INFO: Attempting to reconnect updateshow
          Jul 22, 2010 12:28:47 PM hudson.slaves.CommandLauncher launch
          INFO: slave agent launched for updateshow
          [GC 62195K->52411K(160932K), 0.0747990 secs]
          

          In this second case I guess that the following has happened: The FullGC on the master takes 225 seconds (almost four minutes). During that time the master cannot answer the ping. The slave gives up on the master and aborts. Eventually the master detects this and restarts the slave. This happens after the job started at 12:20 is completed and before the job starting at 12:30 so no jobs are affected. To this, the simple fix would be to increase the TIME_OUT of the Ping (end of the hudson.remoting.PingThread class).

          In the first case I don't understand what has happened.

          This is a complicated problem to investigate because it just happens occasionally and the log is overwritten.

          To allow us to investigate this further, let me suggest that the master-slave function is improved with a function to move or save the old log every time a slave starts so the log of the slave that crashed, failed, or terminated is preserved.

          Show
          linustolke Linus Tolke added a comment - - edited I notice the same (or a similar) problem on my Debian/Linux host and Hudson 1.357. I have the remote job running on the same host started with the Launch slave via execution of command on the master: nice -n 5 /usr/local/lib/java/jdk1.6.0/bin/java -Xdebug -Xrunjdwp:transport=dt_socket,address=8003,server=y,suspend=n -Xms16M -jar /home/argonightly/tools/WEB-INF/slave.jar (The same problems occur also without the -Xdebug -Xrunjdwp... arguments). The purpose of starting in this way is to get some long-running jobs to run with a lower nice to allow other jobs to finish quicker. For me this has only happened when there are two jobs running on the slave. Both jobs fail with the same error message at the same time. Then the slave(s) is restarted and new jobs are allocated on the slave. I have previously noticed that the jobs on the old slave is still running so I assume that it is just the connection between the master and the slave that is the problem. Today I have noticed this twice and managed to capture the output from the master hudson. The first time with two jobs running on one slave and probably none on the other, the second time with just one of the slaves without any jobs running. Here is the first time with the output from the master. Notice that both slaves (updateshow and nightly) are crashing, taken down, or failing at the same time. You can also notice that I run the master hudson with the -verbosegc argument. [GC 127674K->117585K(160932K), 9.4880730 secs] Jul 22, 2010 6:35:43 AM hudson.model.Run run INFO: update-mirror #30764 main build action completed: SUCCESS [GC 127697K->117678K(160932K), 6.2626260 secs] Jul 22, 2010 6:44:51 AM hudson.remoting.Channel$ReaderThread run SEVERE: I/O error in channel updateshow java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:862) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputSt ream.java:2552) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:856) Jul 22, 2010 6:44:57 AM hudson.remoting.Channel$ReaderThread run SEVERE: I/O error in channel nightly java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:862) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputSt ream.java:2552) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:856) Jul 22, 2010 6:44:57 AM hudson.remoting.Request$2 run SEVERE: Failed to send back a reply java.io.IOException: Broken pipe at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:260) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65 ) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123) at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputSt ream.java:1784) at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:691) at hudson.remoting.Channel.send(Channel.java:418) at hudson.remoting.Request$2.run(Request.java:282) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:44 1) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec utor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor .java:907) at java.lang.Thread.run(Thread.java:619) Jul 22, 2010 6:45:03 AM hudson.slaves.SlaveComputer tryReconnect INFO: Attempting to reconnect nightly Jul 22, 2010 6:45:04 AM hudson.slaves.SlaveComputer tryReconnect INFO: Attempting to reconnect updateshow [GC 127787K->118134K(160932K), 6.6658860 secs] This is the backtrace from the log of one of the jobs on the nightly slave: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:312) at hudson.Launcher$ProcStarter.join(Launcher.java:278) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:600) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1241) at hudson.matrix.MatrixRun.run(MatrixRun.java:130) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:304) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:598) at hudson.remoting.Channel$ReaderThread.run(Channel.java:880) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:862) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:856) FATAL: Unable to delete script file /tmp/hudson18796.sh hudson.util.IOException2: remote file operation failed: /tmp/hudson18796.sh at hudson.remoting.Channel@2e646f:nightly at hudson.FilePath.act(FilePath.java:752) at hudson.FilePath.act(FilePath.java:738) at hudson.FilePath.delete(FilePath.java:993) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:600) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1241) at hudson.matrix.MatrixRun.run(MatrixRun.java:130) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:551) at hudson.FilePath.act(FilePath.java:745) ... 13 more FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:551) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:738) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1241) at hudson.matrix.MatrixRun.run(MatrixRun.java:130) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122) The second time today only the updateshow job got this problem and I cannot find any failed jobs. The jobs on updateshow are a lot shorter (around 5m). Here is the output from the master process: [GC 151557K->148710K(160932K), 4.0233760 secs] Jul 22, 2010 12:23:09 PM hudson.model.Run run INFO: update-mirror #30799 main build action completed: SUCCESS [GC 158822K->150464K(161356K), 1.0420050 secs] [Full GC[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14 720] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14411] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14660] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14581] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14459] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14746] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14554] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14357] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13929] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14437] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14747] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14707] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14743] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14384] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14754] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14496] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14383] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14508] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14664] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14346] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14468] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14397] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14712] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14549] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14419] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14671] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14620] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14365] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14328] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14429] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14415] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14408] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13652] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14593] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14456] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14425] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14609] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14636] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14574] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14626] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14539] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14441] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14568] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14639] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14694] [Unloading class sun.reflect.GeneratedConstructorAccessor334] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14667] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14577] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14550] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14650] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14351] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14633] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14744] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14364] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14725] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14356] [Unloading class sun.reflect.GeneratedMethodAccessor1350] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13782] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13807] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14463] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14360] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14576] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14560] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14515] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14462] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14349] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14489] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14647] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14579] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14538] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14524] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14482] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14445] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14398] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14402] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14358] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14651] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14627] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14421] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14400] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14391] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14432] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14635] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13571] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14481] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14572] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14734] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14336] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14440] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14467] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14732] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14713] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14629] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14377] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14715] [Unloading class sun.reflect.GeneratedConstructorAccessor333] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14632] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14724] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14418] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14617] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14361] [Unloading class sun.reflect.GeneratedConstructorAccessor322] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14460] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14392] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14599] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14691] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14368] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14613] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14705] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14587] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14511] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13936] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14396] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14696] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14634] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14548] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14700] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14388] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14504] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14491] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14678] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14645] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13930] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14733] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14592] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14444] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14433] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14665] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14551] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14479] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14730] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14615] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14589] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14355] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14472] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14708] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14662] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14683] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14507] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14521] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14470] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14393] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14422] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14420] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14674] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14637] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14476] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14716] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14477] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14352] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13805] [Unloading class sun.reflect.GeneratedMethodAccessor1352] [Unloading class sun.reflect.GeneratedConstructorAccessor327] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14516] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14403] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14719] [Unloading class sun.reflect.GeneratedConstructorAccessor329] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14416] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14466] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14692] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14711] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14565] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14685] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14698] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14677] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14575] [Unloading class sun.reflect.GeneratedConstructorAccessor332] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14333] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14571] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14412] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14717] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14679] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14654] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14638] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14652] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14604] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14457] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14534] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13586] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14329] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14409] [Unloading class sun.reflect.GeneratedMethodAccessor1355] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14449] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14544] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14424] [Unloading class sun.reflect.GeneratedConstructorAccessor324] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14385] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14703] [Unloading class sun.reflect.GeneratedConstructorAccessor331] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14343] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14621] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14531] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14379] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14542] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14526] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14518] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14506] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14666] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14564] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14656] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14390] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14598] [Unloading class sun.reflect.GeneratedConstructorAccessor325] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14567] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14453] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14488] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14625] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14528] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14502] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14558] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14741] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14668] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14586] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14438] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14670] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14541] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14375] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14641] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14646] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14596] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14728] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14367] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14439] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14597] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14736] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14451] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13538] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14490] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14382] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14584] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14347] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14628] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14704] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14749] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13937] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14378] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14423] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14658] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14386] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14399] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14697] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14350] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14553] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14622] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14461] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14417] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14503] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14687] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14381] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14731] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14591] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14748] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14475] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14448] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14695] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14362] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14370] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14458] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14363] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14582] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14753] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14334] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14478] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14610] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14492] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14480] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14513] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14473] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14607] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13935] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14330] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14585] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13580] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14659] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14573] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14689] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14557] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14332] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14532] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14739] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14588] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13583] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14680] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14605] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14616] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14623] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14369] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14341] [Unloading class sun.reflect.GeneratedMethodAccessor1351] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14738] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14514] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14436] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14465] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14642] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14740] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14723] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14742] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14520] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14500] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14517] [Unloading class sun.reflect.GeneratedConstructorAccessor321] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14452] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14331] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14547] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14373] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14701] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14663] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14600] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14606] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14603] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14522] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14340] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14485] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13581] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14527] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14619] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14474] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14413] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14611] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14630] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14454] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14446] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14536] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14512] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14750] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedMethodAccessor1294] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14543] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14669] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13582] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14435] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14583] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14407] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14608] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14430] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14427] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14612] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14426] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14394] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14344] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14745] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14519] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14471] [Unloading class sun.reflect.GeneratedConstructorAccessor326] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14672] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14682] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14673] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14359] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14580] [Unloading class sun.reflect.GeneratedConstructorAccessor328] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14643] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14497] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14410] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14722] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14533] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14540] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14354] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14469] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14631] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14537] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14595] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14688] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14661] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14601] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14702] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14487] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14690] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14376] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14374] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13579] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14594] [Unloading class Script1] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13938] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14483] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14484] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14525] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14751] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14556] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14464] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14428] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14505] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14501] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14486] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14348] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14405] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14561] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14726] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14676] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14675] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14530] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14335] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14590] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14699] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14578] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14566] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14395] [Unloading class sun.reflect.GeneratedMethodAccessor1354] [Unloading class sun.reflect.GeneratedMethodAccessor1290] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14510] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14404] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14727] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14406] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14618] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14657] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14494] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13534] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14681] [Unloading class sun.reflect.GeneratedMethodAccessor1298] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14755] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14366] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14570] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14545] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14353] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14546] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14431] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14710] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14718] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14380] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14752] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14648] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14552] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14499] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14706] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14714] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14342] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14640] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14602] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14559] [Unloading class sun.reflect.GeneratedConstructorAccessor330] [Unloading class sun.reflect.GeneratedConstructorAccessor302] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14569] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14414] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14447] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14372] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14614] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14523] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13769] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14493] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14684] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14371] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14649] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14562] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14450] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14389] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14729] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14535] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14655] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14624] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14509] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14737] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14735] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13806] [Unloading class Script1$_run_closure1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14721] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14442] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14653] [Unloading class sun.reflect.GeneratedConstructorAccessor323] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14345] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14555] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14455] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14434] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14387] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14529] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14498] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14495] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14563] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14443] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14693] [Unloading class Script1] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14401] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14644] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor13931] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14686] [Unloading class sun.reflect.GeneratedMethodAccessor1356] [Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14709] 150464K->51729K(161356K), 225.1859240 secs] Jul 22, 2010 12:26:56 PM hudson.remoting.Channel$ReaderThread run SEVERE: I/O error in channel updateshow java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:862) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputSt ream.java:2552) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:856) [GC 61841K->52274K(160932K), 0.0367730 secs] [GC 62386K->52083K(160932K), 0.0317730 secs] Jul 22, 2010 12:28:38 PM hudson.slaves.SlaveComputer tryReconnect INFO: Attempting to reconnect updateshow Jul 22, 2010 12:28:47 PM hudson.slaves.CommandLauncher launch INFO: slave agent launched for updateshow [GC 62195K->52411K(160932K), 0.0747990 secs] In this second case I guess that the following has happened: The FullGC on the master takes 225 seconds (almost four minutes). During that time the master cannot answer the ping. The slave gives up on the master and aborts. Eventually the master detects this and restarts the slave. This happens after the job started at 12:20 is completed and before the job starting at 12:30 so no jobs are affected. To this, the simple fix would be to increase the TIME_OUT of the Ping (end of the hudson.remoting.PingThread class). In the first case I don't understand what has happened. This is a complicated problem to investigate because it just happens occasionally and the log is overwritten. To allow us to investigate this further, let me suggest that the master-slave function is improved with a function to move or save the old log every time a slave starts so the log of the slave that crashed, failed, or terminated is preserved.
          Hide
          linustolke Linus Tolke added a comment -

          It happened again (at 04:59 AM this morning). I guess it was the long Garbage Collect (221 seconds) that made the client close because of the ping not being answered.

          This time I had moved the slave-nightly.log file to another name. Unluckily, nothing was written to the file. I had hoped to see the "No Ping response received" log error. I guess the slave-nightly.log file is written by the master on the master and since the connection is already closed by the client at this point, the master cannot write anything.

          I still think the most frustrating part of this is that the logs of the slave cannot be accessed. I will see if I understand how to start the slaves logging to a file.

          Show
          linustolke Linus Tolke added a comment - It happened again (at 04:59 AM this morning). I guess it was the long Garbage Collect (221 seconds) that made the client close because of the ping not being answered. This time I had moved the slave-nightly.log file to another name. Unluckily, nothing was written to the file. I had hoped to see the "No Ping response received" log error. I guess the slave-nightly.log file is written by the master on the master and since the connection is already closed by the client at this point, the master cannot write anything. I still think the most frustrating part of this is that the logs of the slave cannot be accessed. I will see if I understand how to start the slaves logging to a file.
          Hide
          glundh glundh added a comment -

          linustolke: "To this, the simple fix would be to increase the TIME_OUT of the Ping (end of the hudson.remoting.PingThread class)."

          This particular issue was fixed in 1.367.

          Please see this discussion for more info:
          http://hudson.361315.n4.nabble.com/Regarding-the-IOException2-ChannelClosedException-many-of-us-has-been-seeing-lately-td2280769.html

          Show
          glundh glundh added a comment - linustolke: "To this, the simple fix would be to increase the TIME_OUT of the Ping (end of the hudson.remoting.PingThread class)." This particular issue was fixed in 1.367. Please see this discussion for more info: http://hudson.361315.n4.nabble.com/Regarding-the-IOException2-ChannelClosedException-many-of-us-has-been-seeing-lately-td2280769.html
          Hide
          mistafunk mistafunk added a comment -

          are you sure that it was fixed in 1.367? i run this version and i still see this problem. additionally, this fix was not yet mentioned in the release notes...

          i'll wait for the next release...

          Show
          mistafunk mistafunk added a comment - are you sure that it was fixed in 1.367? i run this version and i still see this problem. additionally, this fix was not yet mentioned in the release notes... i'll wait for the next release...
          Hide
          glundh glundh added a comment -

          Yes.

          Change 32664 was made 09 July. However the changelog was not updated (I have noticed that many fixes actually goes in to the trunk without updating the changelog).

          http://fisheye.jenkins-ci.org/changelog/Hudson/trunk/hudson?cs=32664

          Show
          glundh glundh added a comment - Yes. Change 32664 was made 09 July. However the changelog was not updated (I have noticed that many fixes actually goes in to the trunk without updating the changelog). http://fisheye.jenkins-ci.org/changelog/Hudson/trunk/hudson?cs=32664
          Hide
          glundh glundh added a comment -
          Show
          glundh glundh added a comment - Here you can see that the correct change is a part of the 1.367 tag: http://fisheye.jenkins-ci.org/browse/Hudson/tags/hudson-1_367/remoting/src/main/java/hudson/remoting/PingThread.java?r=HEAD
          Hide
          alex_barna Tzuchien added a comment -

          I think that this "fix" may only be a workaround instead of a real fix.

          I experienced this problem in 1.363. Yesterday I upgrade to 1.368 and executed the same job in question - a matrix job with about 40 configurations, each of which took from 3 to 4 hours. Most of them used to fail in 1.363, but after switching to 1.368, only one of them failed. So I think the problem still exists.

          Show
          alex_barna Tzuchien added a comment - I think that this "fix" may only be a workaround instead of a real fix. I experienced this problem in 1.363. Yesterday I upgrade to 1.368 and executed the same job in question - a matrix job with about 40 configurations, each of which took from 3 to 4 hours. Most of them used to fail in 1.363, but after switching to 1.368, only one of them failed. So I think the problem still exists.
          Hide
          mistafunk mistafunk added a comment - - edited

          yes, i think there is still something broken in this regard. i am running 1.368 on a linux64 master with two 32bit virtualbox slave VMs (win7/linux) and another linux64 node with three 64bit virtualbox slave VMs (win7/linux/osx).

          below is the typical stacktrace i see regularly (2 out of 3 jobs) on the windows slaves (32bit and 64bit win7, not on linux or osx). it does not seem to be related to the host machines. the exceptions almost always occurs on longish linker, nsis or file transfer (archiving artifacts) operations.

          update: the slave log does not contain anything special when it happens.

          i appreciate any help to debug this.
          kind regards,
          simon

          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          at hudson.Proc$RemoteProc.join(Proc.java:355)
          at hudson.Launcher$ProcStarter.join(Launcher.java:280)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          at hudson.model.Build$RunnerImpl.build(Build.java:174)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          at hudson.model.Run.run(Run.java:1257)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:129)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Request$1.get(Request.java:218)
          at hudson.remoting.Request$1.get(Request.java:172)
          at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          at hudson.Proc$RemoteProc.join(Proc.java:347)
          ... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Request.abort(Request.java:257)
          at hudson.remoting.Channel.terminate(Channel.java:602)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:893)
          Caused by: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:875)
          Caused by: java.io.EOFException
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:869)
          FATAL: Unable to delete script file C:\Windows\TEMP\hudson1632955036060142154.bat
          hudson.util.IOException2: remote file operation failed: C:\Windows\TEMP\hudson1632955036060142154.bat at hudson.remoting.Channel@72945e31:sidian_win7_32
          at hudson.FilePath.act(FilePath.java:749)
          at hudson.FilePath.act(FilePath.java:735)
          at hudson.FilePath.delete(FilePath.java:990)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          at hudson.model.Build$RunnerImpl.build(Build.java:174)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          at hudson.model.Run.run(Run.java:1257)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:129)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          at hudson.remoting.Channel.send(Channel.java:412)
          at hudson.remoting.Request.call(Request.java:105)
          at hudson.remoting.Channel.call(Channel.java:555)
          at hudson.FilePath.act(FilePath.java:742)
          ... 13 more
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          at hudson.remoting.Channel.send(Channel.java:412)
          at hudson.remoting.Request.call(Request.java:105)
          at hudson.remoting.Channel.call(Channel.java:555)
          at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          at hudson.model.Run.run(Run.java:1257)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:129)

          Show
          mistafunk mistafunk added a comment - - edited yes, i think there is still something broken in this regard. i am running 1.368 on a linux64 master with two 32bit virtualbox slave VMs (win7/linux) and another linux64 node with three 64bit virtualbox slave VMs (win7/linux/osx). below is the typical stacktrace i see regularly (2 out of 3 jobs) on the windows slaves (32bit and 64bit win7, not on linux or osx). it does not seem to be related to the host machines. the exceptions almost always occurs on longish linker, nsis or file transfer (archiving artifacts) operations. update: the slave log does not contain anything special when it happens. i appreciate any help to debug this. kind regards, simon FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:355) at hudson.Launcher$ProcStarter.join(Launcher.java:280) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1257) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:129) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:347) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:602) at hudson.remoting.Channel$ReaderThread.run(Channel.java:893) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:875) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:869) FATAL: Unable to delete script file C:\Windows\TEMP\hudson1632955036060142154.bat hudson.util.IOException2: remote file operation failed: C:\Windows\TEMP\hudson1632955036060142154.bat at hudson.remoting.Channel@72945e31:sidian_win7_32 at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.FilePath.delete(FilePath.java:990) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1257) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:129) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:555) at hudson.FilePath.act(FilePath.java:742) ... 13 more FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:555) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1257) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:129)
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/Channel.java
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/ChannelClosedException.java
          http://jenkins-ci.org/commit/33537
          Log:
          [JENKINS-5073 JENKINS-3412] improved the error diagnostics on ChannelClosedException by having it report who/how the connection was closed.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/remoting/src/main/java/hudson/remoting/Channel.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/ChannelClosedException.java http://jenkins-ci.org/commit/33537 Log: [JENKINS-5073 JENKINS-3412] improved the error diagnostics on ChannelClosedException by having it report who/how the connection was closed.
          Hide
          mistafunk mistafunk added a comment -

          i disabled indexing (windows search service) on my win7 vmware slaves - still the same behavior

          Show
          mistafunk mistafunk added a comment - i disabled indexing (windows search service) on my win7 vmware slaves - still the same behavior
          Hide
          kflorian kflorian added a comment -

          We see this on a Linux slave, running on the same machine as the master.
          Seems to happen more frequently under high load.
          Stacktrace is slightly differen:

          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          	at hudson.Proc$RemoteProc.join(Proc.java:312)
          	at hudson.Launcher$ProcStarter.join(Launcher.java:278)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          	at hudson.model.Build$RunnerImpl.build(Build.java:174)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1241)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:124)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request$1.get(Request.java:218)
          	at hudson.remoting.Request$1.get(Request.java:172)
          	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          	at hudson.Proc$RemoteProc.join(Proc.java:304)
          	... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request.abort(Request.java:257)
          	at hudson.remoting.Channel.terminate(Channel.java:598)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:880)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:862)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.readFully(ObjectInputStream.java:2700)
          	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1648)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
          	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
          	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
          	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:856)
          FATAL: Unable to delete script file /tmp/hudson700909144823232476.sh
          hudson.util.IOException2: remote file operation failed: /tmp/hudson700909144823232476.sh at hudson.remoting.Channel@7f5a75a2:hudson_slave
          	at hudson.FilePath.act(FilePath.java:749)
          	at hudson.FilePath.act(FilePath.java:735)
          	at hudson.FilePath.delete(FilePath.java:990)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          	at hudson.model.Build$RunnerImpl.build(Build.java:174)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1241)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:124)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:412)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:551)
          	at hudson.FilePath.act(FilePath.java:742)
          	... 13 more
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:412)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:551)
          	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:738)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          	at hudson.model.Run.run(Run.java:1241)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:124)
          
          Show
          kflorian kflorian added a comment - We see this on a Linux slave, running on the same machine as the master. Seems to happen more frequently under high load. Stacktrace is slightly differen: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:312) at hudson.Launcher$ProcStarter.join(Launcher.java:278) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:83) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1241) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:124) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:304) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:598) at hudson.remoting.Channel$ReaderThread.run(Channel.java:880) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:862) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.readFully(ObjectInputStream.java:2700) at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1648) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:856) FATAL: Unable to delete script file /tmp/hudson700909144823232476.sh hudson.util.IOException2: remote file operation failed: /tmp/hudson700909144823232476.sh at hudson.remoting.Channel@7f5a75a2:hudson_slave at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.FilePath.delete(FilePath.java:990) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:93) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1241) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:124) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:551) at hudson.FilePath.act(FilePath.java:742) ... 13 more FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:412) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:551) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:738) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1241) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:124)
          Hide
          mistafunk mistafunk added a comment -

          a random guess: could this issue be related to high slave response times? on my 5 slaves i observe 5-40ms for the linux32/linux64/osx slaves and 200-600ms for the two win32/win64 slaves. the former three slaves run rock solid, on the two windows slaves i regularly (75%) get "failed to join the process".

          note that our linux32/win32 and linux64/win64 slave pairs run on the same virtualbox hosts (linux64) each. i guess, this rules out the hosts as problem sources. so is this a problem of hudson slave communication in combination with virtualbox!? i'm going to try different vbox network setups, maybe the vbox nat is the offender...

          i am veeery motivated to finally resolve this one

          Show
          mistafunk mistafunk added a comment - a random guess: could this issue be related to high slave response times? on my 5 slaves i observe 5-40ms for the linux32/linux64/osx slaves and 200-600ms for the two win32/win64 slaves. the former three slaves run rock solid, on the two windows slaves i regularly (75%) get "failed to join the process". note that our linux32/win32 and linux64/win64 slave pairs run on the same virtualbox hosts (linux64) each. i guess, this rules out the hosts as problem sources. so is this a problem of hudson slave communication in combination with virtualbox!? i'm going to try different vbox network setups, maybe the vbox nat is the offender... i am veeery motivated to finally resolve this one
          Hide
          mike_l Mike_L added a comment - - edited

          Also having the same problem. Running Hudson 1.378 on CentOS 5.5 connecting to a CentOS 5.2 slave via ssh.

          Hi, we noticed that this happens just in a job that execute a job with a duration of around 2hours that has a step with no output in a long time (don't know exactly how long)

          -deccico
          This is when it fails for me, too. The error occurs while running a script that takes quite a while and does not produce output. However, if I make the script print to the console frequently as it's running, this error doesn't pop up.

          Is it because the ssh connection seems idle and isn't using a "keep-alive"?

          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          	at hudson.Proc$RemoteProc.join(Proc.java:355)
          	at hudson.Launcher$ProcStarter.join(Launcher.java:280)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          	at hudson.model.Build$RunnerImpl.build(Build.java:174)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1280)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:137)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request$1.get(Request.java:218)
          	at hudson.remoting.Request$1.get(Request.java:172)
          	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          	at hudson.Proc$RemoteProc.join(Proc.java:347)
          	... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request.abort(Request.java:257)
          	at hudson.remoting.Channel.terminate(Channel.java:658)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:950)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:931)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:925)
          FATAL: Unable to delete script file /tmp/hudson8197175197320902892.sh
          hudson.util.IOException2: remote file operation failed: /tmp/hudson8197175197320902892.sh at hudson.remoting.Channel@7069b861:slave
          	at hudson.FilePath.act(FilePath.java:749)
          	at hudson.FilePath.act(FilePath.java:735)
          	at hudson.FilePath.delete(FilePath.java:990)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          	at hudson.model.Build$RunnerImpl.build(Build.java:174)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1280)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:137)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:451)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:607)
          	at hudson.FilePath.act(FilePath.java:742)
          	... 13 more
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:931)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:925)
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:451)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:607)
          	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          	at hudson.model.Run.run(Run.java:1280)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:137)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:931)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:925)
          
          Show
          mike_l Mike_L added a comment - - edited Also having the same problem. Running Hudson 1.378 on CentOS 5.5 connecting to a CentOS 5.2 slave via ssh. Hi, we noticed that this happens just in a job that execute a job with a duration of around 2hours that has a step with no output in a long time (don't know exactly how long) - deccico This is when it fails for me, too. The error occurs while running a script that takes quite a while and does not produce output. However, if I make the script print to the console frequently as it's running, this error doesn't pop up. Is it because the ssh connection seems idle and isn't using a "keep-alive"? FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:355) at hudson.Launcher$ProcStarter.join(Launcher.java:280) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1280) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:137) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:347) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:658) at hudson.remoting.Channel$ReaderThread.run(Channel.java:950) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:931) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:925) FATAL: Unable to delete script file /tmp/hudson8197175197320902892.sh hudson.util.IOException2: remote file operation failed: /tmp/hudson8197175197320902892.sh at hudson.remoting.Channel@7069b861:slave at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.FilePath.delete(FilePath.java:990) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1280) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:137) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:451) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:607) at hudson.FilePath.act(FilePath.java:742) ... 13 more Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:931) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:925) FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:451) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:607) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1280) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:137) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:931) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:925)
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          More recent version of Hudson, like the one Mike_L used, shows the root cause of the "hudson.remoting.ChannelClosedException: channel is already closed". So those who are experiencing this problem, please report your stack trace so that we can collect more data.

          So far the error seems to indicate that the network connection between the master and the slave is terminated abruptly.

          I guess I need to first improve Hudson so that it captures the failure from the slave process before we can understand the cause of the problem.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - More recent version of Hudson, like the one Mike_L used, shows the root cause of the "hudson.remoting.ChannelClosedException: channel is already closed". So those who are experiencing this problem, please report your stack trace so that we can collect more data. So far the error seems to indicate that the network connection between the master and the slave is terminated abruptly. I guess I need to first improve Hudson so that it captures the failure from the slave process before we can understand the cause of the problem.
          Hide
          jed624 jed624 added a comment - - edited

          For the more info request. The issue started plaguing us about 2 weeks ago. Running 1.375 on 32-bit Win 2003 Server, with an additional slave on the same server and one on a 64-bit Windows 2008 server. Tomcat 6.0.20, java 1.6.0_06. Java 1.6.0_20 on the 64-bit 2008 server. Slave.jars are in sync with server version.

          I have the Master + 3 slaves running as services on Windows. This particular issue strikes every 2-4 days, usually during the middle of actual build work where logs are actively being written. Recovery requires restaring the master and all slaves. The builds we do are large C and C++ builds, and they take a long time between 3-5 hours.

          The unstableness is dwindling my dev team's confidence in Hudson's usefulness. Hopefully this issue can be addressed soon.

          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          	at hudson.Proc$RemoteProc.join(Proc.java:355)
          	at hudson.Launcher$ProcStarter.join(Launcher.java:280)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          	at hudson.model.Build$RunnerImpl.build(Build.java:174)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1273)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:129)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request$1.get(Request.java:218)
          	at hudson.remoting.Request$1.get(Request.java:172)
          	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          	at hudson.Proc$RemoteProc.join(Proc.java:347)
          	... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request.abort(Request.java:257)
          	at hudson.remoting.Channel.terminate(Channel.java:608)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:899)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:881)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
          	at java.io.ObjectInputStream.readObject0(Unknown Source)
          	at java.io.ObjectInputStream.readObject(Unknown Source)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:875)
          FATAL: Unable to delete script file C:\Users\lgbuild\AppData\Local\Temp\hudson8508007568280050491.bat
          hudson.util.IOException2: remote file operation failed: C:\Users\lgbuild\AppData\Local\Temp\hudson8508007568280050491.bat at hudson.remoting.Channel@367b19:Schwartz
          	at hudson.FilePath.act(FilePath.java:749)
          	at hudson.FilePath.act(FilePath.java:735)
          	at hudson.FilePath.delete(FilePath.java:990)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          	at hudson.model.Build$RunnerImpl.build(Build.java:174)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1273)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:129)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:414)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:557)
          	at hudson.FilePath.act(FilePath.java:742)
          	... 13 more
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:881)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
          	at java.io.ObjectInputStream.readObject0(Unknown Source)
          	at java.io.ObjectInputStream.readObject(Unknown Source)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:875)
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:414)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:557)
          	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          	at hudson.model.Run.run(Run.java:1273)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:129)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:881)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
          	at java.io.ObjectInputStream.readObject0(Unknown Source)
          	at java.io.ObjectInputStream.readObject(Unknown Source)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:875)
          
          Show
          jed624 jed624 added a comment - - edited For the more info request. The issue started plaguing us about 2 weeks ago. Running 1.375 on 32-bit Win 2003 Server, with an additional slave on the same server and one on a 64-bit Windows 2008 server. Tomcat 6.0.20, java 1.6.0_06. Java 1.6.0_20 on the 64-bit 2008 server. Slave.jars are in sync with server version. I have the Master + 3 slaves running as services on Windows. This particular issue strikes every 2-4 days, usually during the middle of actual build work where logs are actively being written. Recovery requires restaring the master and all slaves. The builds we do are large C and C++ builds, and they take a long time between 3-5 hours. The unstableness is dwindling my dev team's confidence in Hudson's usefulness. Hopefully this issue can be addressed soon. FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:355) at hudson.Launcher$ProcStarter.join(Launcher.java:280) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1273) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:129) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:347) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:608) at hudson.remoting.Channel$ReaderThread.run(Channel.java:899) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:881) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Channel$ReaderThread.run(Channel.java:875) FATAL: Unable to delete script file C:\Users\lgbuild\AppData\Local\Temp\hudson8508007568280050491.bat hudson.util.IOException2: remote file operation failed: C:\Users\lgbuild\AppData\Local\Temp\hudson8508007568280050491.bat at hudson.remoting.Channel@367b19:Schwartz at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.FilePath.delete(FilePath.java:990) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1273) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:129) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:414) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:557) at hudson.FilePath.act(FilePath.java:742) ... 13 more Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:881) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Channel$ReaderThread.run(Channel.java:875) FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:414) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:557) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1273) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:129) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:881) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Channel$ReaderThread.run(Channel.java:875)
          Hide
          tapiomtr tapiomtr added a comment -

          When executing a free-style job in a Windows agent, this error appers:

          2> WINVER not defined. Defaulting to 0x0502 (Windows Server 2003)
          11>CDllCall.cpp
          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          at hudson.Proc$RemoteProc.join(Proc.java:355)
          at hudson.Launcher$ProcStarter.join(Launcher.java:280)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          at hudson.model.Build$RunnerImpl.build(Build.java:174)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          at hudson.model.Run.run(Run.java:1280)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:139)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
          at hudson.remoting.Request$1.get(Request.java:218)
          at hudson.remoting.Request$1.get(Request.java:172)
          at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          at hudson.Proc$RemoteProc.join(Proc.java:347)
          ... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
          at hudson.remoting.Request.abort(Request.java:257)
          at hudson.remoting.Channel.terminate(Channel.java:681)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:972)
          Caused by: java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(SocketInputStream.java:185)
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
          at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2265)
          at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2558)
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2568)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:948)
          FATAL: Unable to delete script file C:\WINDOWS\TEMP\hudson7211599763862507406.bat
          hudson.util.IOException2: remote file operation failed: C:\WINDOWS\TEMP\hudson7211599763862507406.bat at hudson.remoting.Channel@60ac71:FXBuild
          at hudson.FilePath.act(FilePath.java:749)
          at hudson.FilePath.act(FilePath.java:735)
          at hudson.FilePath.delete(FilePath.java:990)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
          at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
          at hudson.model.Build$RunnerImpl.build(Build.java:174)
          at hudson.model.Build$RunnerImpl.doRun(Build.java:138)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          at hudson.model.Run.run(Run.java:1280)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:139)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          at hudson.remoting.Channel.send(Channel.java:467)
          at hudson.remoting.Request.call(Request.java:105)
          at hudson.remoting.Channel.call(Channel.java:630)
          at hudson.FilePath.act(FilePath.java:742)
          ... 13 more
          Caused by: java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(SocketInputStream.java:185)
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
          at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2265)
          at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2558)
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2568)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:948)
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          at hudson.remoting.Channel.send(Channel.java:467)
          at hudson.remoting.Request.call(Request.java:105)
          at hudson.remoting.Channel.call(Channel.java:630)
          at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744)
          at hudson.plugins.cygpath.CygpathLauncherDecorator$1.kill(CygpathLauncherDecorator.java:76)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          at hudson.model.Run.run(Run.java:1280)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:139)
          Caused by: java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(SocketInputStream.java:185)
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
          at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2265)
          at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2558)
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2568)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:948)

          Show
          tapiomtr tapiomtr added a comment - When executing a free-style job in a Windows agent, this error appers: 2> WINVER not defined. Defaulting to 0x0502 (Windows Server 2003) 11>CDllCall.cpp FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:355) at hudson.Launcher$ProcStarter.join(Launcher.java:280) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1280) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:347) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:681) at hudson.remoting.Channel$ReaderThread.run(Channel.java:972) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:185) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2265) at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2558) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2568) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368) at hudson.remoting.Channel$ReaderThread.run(Channel.java:948) FATAL: Unable to delete script file C:\WINDOWS\TEMP\hudson7211599763862507406.bat hudson.util.IOException2: remote file operation failed: C:\WINDOWS\TEMP\hudson7211599763862507406.bat at hudson.remoting.Channel@60ac71:FXBuild at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.FilePath.delete(FilePath.java:990) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601) at hudson.model.Build$RunnerImpl.build(Build.java:174) at hudson.model.Build$RunnerImpl.doRun(Build.java:138) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1280) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:467) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:630) at hudson.FilePath.act(FilePath.java:742) ... 13 more Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:185) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2265) at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2558) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2568) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368) at hudson.remoting.Channel$ReaderThread.run(Channel.java:948) FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:467) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:630) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744) at hudson.plugins.cygpath.CygpathLauncherDecorator$1.kill(CygpathLauncherDecorator.java:76) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1280) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:185) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2265) at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2558) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2568) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368) at hudson.remoting.Channel$ReaderThread.run(Channel.java:948)
          Hide
          swpalmer swpalmer added a comment -

          This may be the same as http://issues.jenkins-ci.org/browse/JENKINS-7690
          Which has a stack trace mentioning:
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed

          Show
          swpalmer swpalmer added a comment - This may be the same as http://issues.jenkins-ci.org/browse/JENKINS-7690 Which has a stack trace mentioning: FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed
          Hide
          mazur_roman mazur_roman added a comment -

          Hello! We also frequently experience such problem . We tried hudson 1.386 and 1.387, jdk-6.0.19 and jdk-6.0.22, Master host: Linux Gentoo, Slave host: Linux Ubuntu.
          Here is log:

           
          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          	at hudson.Proc$RemoteProc.join(Proc.java:355)
          	at hudson.Launcher$ProcStarter.join(Launcher.java:280)
          	at com.g2one.hudson.grails.GrailsBuilder.perform(GrailsBuilder.java:184)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:603)
          	at hudson.model.Build$RunnerImpl.build(Build.java:172)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:137)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:417)
          	at hudson.model.Run.run(Run.java:1325)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:139)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55
          	at hudson.remoting.Request$1.get(Request.java:218)
          	at hudson.remoting.Request$1.get(Request.java:172)
          	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          	at hudson.Proc$RemoteProc.join(Proc.java:347)
          	... 11 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55
          	at hudson.remoting.Request.abort(Request.java:257)
          	at hudson.remoting.Channel.terminate(Channel.java:680)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:971)
          Caused by: java.io.StreamCorruptedException: invalid type code: 55
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1355)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:947)
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:466)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:629)
          	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:422)
          	at hudson.model.Run.run(Run.java:1325)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:139)
          Caused by: java.io.StreamCorruptedException: invalid type code: 55
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1355)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:947)
          
          Show
          mazur_roman mazur_roman added a comment - Hello! We also frequently experience such problem . We tried hudson 1.386 and 1.387, jdk-6.0.19 and jdk-6.0.22, Master host: Linux Gentoo, Slave host: Linux Ubuntu. Here is log: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:355) at hudson.Launcher$ProcStarter.join(Launcher.java:280) at com.g2one.hudson.grails.GrailsBuilder.perform(GrailsBuilder.java:184) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:603) at hudson.model.Build$RunnerImpl.build(Build.java:172) at hudson.model.Build$RunnerImpl.doRun(Build.java:137) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:417) at hudson.model.Run.run(Run.java:1325) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55 at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:347) ... 11 more Caused by: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55 at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:680) at hudson.remoting.Channel$ReaderThread.run(Channel.java:971) Caused by: java.io.StreamCorruptedException: invalid type code: 55 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1355) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at hudson.remoting.Channel$ReaderThread.run(Channel.java:947) FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:466) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:629) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:422) at hudson.model.Run.run(Run.java:1325) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: java.io.StreamCorruptedException: invalid type code: 55 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1355) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at hudson.remoting.Channel$ReaderThread.run(Channel.java:947)
          Hide
          ccaraivan Costin Caraivan added a comment - - edited

          Same problem.
          Configuration:
          Master: Red Hat Enterprise Linux Server release 5.5 (Tikanga), x86-64
          Slave: Debian Lenny, x86-64
          Freestyle job running on slave fails randomly when trying to launch a Jython script from a shell. Error message is:

          FATAL: command execution failed
          hudson.util.IOException2: Failed to join the process
          	at hudson.Proc$RemoteProc.join(Proc.java:355)
          	at hudson.Launcher$ProcStarter.join(Launcher.java:280)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper.executeBuildSteps(M2ExtraStepsWrapper.java:166)
          	at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper.access$200(M2ExtraStepsWrapper.java:43)
          	at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper$1.tearDown(M2ExtraStepsWrapper.java:137)
          	at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:494)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1280)
          	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:293)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:139)
          Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request$1.get(Request.java:218)
          	at hudson.remoting.Request$1.get(Request.java:172)
          	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
          	at hudson.Proc$RemoteProc.join(Proc.java:347)
          	... 12 more
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request.abort(Request.java:257)
          	at hudson.remoting.Channel.terminate(Channel.java:681)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:972)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:954)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:948)
          FATAL: Unable to delete script file /tmp/hudson1272208775265435208.sh
          hudson.util.IOException2: remote file operation failed: /tmp/hudson1272208775265435208.sh at hudson.remoting.Channel@35267641:vmlnxbld1buch-axhudson-x64
          	at hudson.FilePath.act(FilePath.java:749)
          	at hudson.FilePath.act(FilePath.java:735)
          	at hudson.FilePath.delete(FilePath.java:990)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper.executeBuildSteps(M2ExtraStepsWrapper.java:166)
          	at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper.access$200(M2ExtraStepsWrapper.java:43)
          	at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper$1.tearDown(M2ExtraStepsWrapper.java:137)
          	at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:494)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416)
          	at hudson.model.Run.run(Run.java:1280)
          	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:293)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:139)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:467)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:630)
          	at hudson.FilePath.act(FilePath.java:742)
          	... 13 more
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:954)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:948)
          FATAL: channel is already closed
          hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:467)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:630)
          	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421)
          	at hudson.model.Run.run(Run.java:1280)
          	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:293)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:139)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:954)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:948)
          

          Does not happen all the time, seems random so far, but is highly annoying, and blocks releases sometimes.

          Show
          ccaraivan Costin Caraivan added a comment - - edited Same problem. Configuration: Master: Red Hat Enterprise Linux Server release 5.5 (Tikanga), x86-64 Slave: Debian Lenny, x86-64 Freestyle job running on slave fails randomly when trying to launch a Jython script from a shell. Error message is: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:355) at hudson.Launcher$ProcStarter.join(Launcher.java:280) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper.executeBuildSteps(M2ExtraStepsWrapper.java:166) at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper.access$200(M2ExtraStepsWrapper.java:43) at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper$1.tearDown(M2ExtraStepsWrapper.java:137) at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:494) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1280) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:293) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request$1.get(Request.java:218) at hudson.remoting.Request$1.get(Request.java:172) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:347) ... 12 more Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:681) at hudson.remoting.Channel$ReaderThread.run(Channel.java:972) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:954) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:948) FATAL: Unable to delete script file /tmp/hudson1272208775265435208.sh hudson.util.IOException2: remote file operation failed: /tmp/hudson1272208775265435208.sh at hudson.remoting.Channel@35267641:vmlnxbld1buch-axhudson-x64 at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.FilePath.delete(FilePath.java:990) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper.executeBuildSteps(M2ExtraStepsWrapper.java:166) at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper.access$200(M2ExtraStepsWrapper.java:43) at hudson.plugins.m2extrasteps.M2ExtraStepsWrapper$1.tearDown(M2ExtraStepsWrapper.java:137) at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:494) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:416) at hudson.model.Run.run(Run.java:1280) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:293) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:467) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:630) at hudson.FilePath.act(FilePath.java:742) ... 13 more Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:954) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:948) FATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:467) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:630) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:744) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:421) at hudson.model.Run.run(Run.java:1280) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:293) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:954) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:948) Does not happen all the time, seems random so far, but is highly annoying, and blocks releases sometimes.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/slaves/CommandLauncher.java
          core/src/main/java/hudson/slaves/ComputerLauncher.java
          core/src/main/java/hudson/slaves/SlaveComputer.java
          core/src/main/java/hudson/util/io/ReopenableFileOutputStream.java
          https://github.com/hudson/hudson/commit/0ddb0c5cca757a4a34e0c162bbca6504f5cec25f
          Log:
          JENKINS-5073 Hudson was failing to record the connection termination problem in slave logs.

          This change might reveal additional information about why the connection is disrupted.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/slaves/CommandLauncher.java core/src/main/java/hudson/slaves/ComputerLauncher.java core/src/main/java/hudson/slaves/SlaveComputer.java core/src/main/java/hudson/util/io/ReopenableFileOutputStream.java https://github.com/hudson/hudson/commit/0ddb0c5cca757a4a34e0c162bbca6504f5cec25f Log: JENKINS-5073 Hudson was failing to record the connection termination problem in slave logs. This change might reveal additional information about why the connection is disrupted.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: Kohsuke Kawaguchi
          Path:
          core/src/main/java/hudson/model/AbstractBuild.java
          core/src/main/java/hudson/model/TaskListener.java
          https://github.com/hudson/hudson/commit/c6ffab117de489e7a728d47dab5c9f5a3b92e19c
          Log:
          JENKINS-5073 if the build fails because the node went offline, point the user to the slave log where the details are.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/model/AbstractBuild.java core/src/main/java/hudson/model/TaskListener.java https://github.com/hudson/hudson/commit/c6ffab117de489e7a728d47dab5c9f5a3b92e19c Log: JENKINS-5073 if the build fails because the node went offline, point the user to the slave log where the details are.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: Kohsuke Kawaguchi
          Path:
          core/src/main/java/hudson/model/AbstractBuild.java
          https://github.com/hudson/hudson/commit/5f29d615a8d8536c59111937df2e36941fb12d18
          Log:
          JENKINS-5073 point the user to the slave log if the build failed because the slave went offline during the build.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/model/AbstractBuild.java https://github.com/hudson/hudson/commit/5f29d615a8d8536c59111937df2e36941fb12d18 Log: JENKINS-5073 point the user to the slave log if the build failed because the slave went offline during the build.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          https://github.com/hudson/hudson/commit/7fbb893fd6831fa8c19b3be98df5655e1281e0da
          Log:
          noting these changes that are related to JENKINS-5073

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: Kohsuke Kawaguchi Path: changelog.html https://github.com/hudson/hudson/commit/7fbb893fd6831fa8c19b3be98df5655e1281e0da Log: noting these changes that are related to JENKINS-5073
          Hide
          dogfood dogfood added a comment -

          Integrated in hudson_main_trunk #450
          JENKINS-5073 Hudson was failing to record the connection termination problem in slave logs.
          JENKINS-5073 if the build fails because the node went offline, point the user to the slave log where the details are.
          JENKINS-5073 point the user to the slave log if the build failed because the slave went offline during the build.
          noting these changes that are related to JENKINS-5073

          Kohsuke Kawaguchi :
          Files :

          • core/src/main/java/hudson/slaves/CommandLauncher.java
          • core/src/main/java/hudson/slaves/ComputerLauncher.java
          • core/src/main/java/hudson/slaves/SlaveComputer.java
          • core/src/main/java/hudson/util/io/ReopenableFileOutputStream.java
          • changelog.html

          Kohsuke Kawaguchi :
          Files :

          • core/src/main/java/hudson/model/TaskListener.java
          • core/src/main/java/hudson/model/AbstractBuild.java

          Kohsuke Kawaguchi :
          Files :

          • core/src/main/java/hudson/model/AbstractBuild.java

          Kohsuke Kawaguchi :
          Files :

          • changelog.html
          Show
          dogfood dogfood added a comment - Integrated in hudson_main_trunk #450 JENKINS-5073 Hudson was failing to record the connection termination problem in slave logs. JENKINS-5073 if the build fails because the node went offline, point the user to the slave log where the details are. JENKINS-5073 point the user to the slave log if the build failed because the slave went offline during the build. noting these changes that are related to JENKINS-5073 Kohsuke Kawaguchi : Files : core/src/main/java/hudson/slaves/CommandLauncher.java core/src/main/java/hudson/slaves/ComputerLauncher.java core/src/main/java/hudson/slaves/SlaveComputer.java core/src/main/java/hudson/util/io/ReopenableFileOutputStream.java changelog.html Kohsuke Kawaguchi : Files : core/src/main/java/hudson/model/TaskListener.java core/src/main/java/hudson/model/AbstractBuild.java Kohsuke Kawaguchi : Files : core/src/main/java/hudson/model/AbstractBuild.java Kohsuke Kawaguchi : Files : changelog.html
          Hide
          conf_ua cOnf_ua added a comment -

          We upgraded to Hudson 1.395 (Master host: Linux Gentoo, Slave host: Linux Ubuntu.), so now our logs are:

          
          FATAL: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55
          hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55
          at hudson.remoting.Request.call(Request.java:137)
          at hudson.remoting.Channel.call(Channel.java:629)
          at hudson.FilePath.act(FilePath.java:745)
          at hudson.FilePath.act(FilePath.java:738)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:683)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:632)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1171)
          at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:499)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:415)
          at hudson.model.Run.run(Run.java:1362)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:145)
          Caused by: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55
          at hudson.remoting.Request.abort(Request.java:257)
          at hudson.remoting.Channel.terminate(Channel.java:680)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:971)
          Caused by: java.io.StreamCorruptedException: invalid type code: 55
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1355)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:947)
          
          Show
          conf_ua cOnf_ua added a comment - We upgraded to Hudson 1.395 (Master host: Linux Gentoo, Slave host: Linux Ubuntu.), so now our logs are: FATAL: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55 hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55 at hudson.remoting.Request.call(Request.java:137) at hudson.remoting.Channel.call(Channel.java:629) at hudson.FilePath.act(FilePath.java:745) at hudson.FilePath.act(FilePath.java:738) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:683) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:632) at hudson.model.AbstractProject.checkout(AbstractProject.java:1171) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:499) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:415) at hudson.model.Run.run(Run.java:1362) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:145) Caused by: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 55 at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:680) at hudson.remoting.Channel$ReaderThread.run(Channel.java:971) Caused by: java.io.StreamCorruptedException: invalid type code: 55 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1355) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at hudson.remoting.Channel$ReaderThread.run(Channel.java:947)
          Hide
          lacostej lacostej added a comment - - edited

          I was looking at the Proc class, and there seems to be a bug in the StdinCopyThread that could be cause the associated LocalProc to close its stream too early.

          See :

                      @Override
                      public void run() {
                          try {
                              try {
                                  byte[] buf = new byte[8192];
                                  int len;
                                  while ((len = in.read(buf)) > 0) {
                                      out.write(buf, 0, len);
                                      out.flush();
                                  }
                              } finally {
                                  in.close();
                                  out.close();
                              }
                          } catch (IOException e) {
                              // TODO: what to do?
                          }
                      }
          

          If the read returns 0, e.g. because of a buffering delay etc, then the while loop exists before we are done.

          Same for StreamCopyThread. There may be other occurences in the code. I will search for them.

          Not sure how these classes are used but it could definitively affect a long build, explaining the lack of reproducibility.

          I created JENKINS-8686 to track and deal with this issue. We will see if that solves this problem or not. Otherwise we might have to ask for some logs (there are a few well placed log calls in that part of the code).

          Show
          lacostej lacostej added a comment - - edited I was looking at the Proc class, and there seems to be a bug in the StdinCopyThread that could be cause the associated LocalProc to close its stream too early. See : @Override public void run() { try { try { byte [] buf = new byte [8192]; int len; while ((len = in.read(buf)) > 0) { out.write(buf, 0, len); out.flush(); } } finally { in.close(); out.close(); } } catch (IOException e) { // TODO: what to do ? } } If the read returns 0, e.g. because of a buffering delay etc, then the while loop exists before we are done. Same for StreamCopyThread. There may be other occurences in the code. I will search for them. Not sure how these classes are used but it could definitively affect a long build, explaining the lack of reproducibility. I created JENKINS-8686 to track and deal with this issue. We will see if that solves this problem or not. Otherwise we might have to ask for some logs (there are a few well placed log calls in that part of the code).
          Hide
          soundrabbit lukas rytz added a comment - - edited

          [Edit 2: We only had the issues with JNLP-Slaves. Instead, we now installed Cygwin+OpenSSH on the Windows slave and connect using the SSH Slaves plugin. This works fine.]

          We see similar problems.

          Master: Ubuntu x64, 2.6.32-29-server
          Slave: Windows 7 SP1 x64, VM running in VirtualBox 4.0.4, Java 1.6.0_24-x64 (also tried 32bit JVM, same problem).

          [Edit: i disabled antivirus, Windows Defender, UAC, I run the jenkins slave as administrator. These don't help.]

          We also have a Windows XP 32 bit VM, similar configuration, and there it never happens. On the Win7 slave it happens at every build.

          Here are 5 build, the stack traces are exactly identical 1 2 3 4 5

          It looks like it does not happen at random points, but at certain points in the build.

          FATAL: Unable to delete script file C:\Windows\TEMP\hudson182664989132439196.bat
          hudson.util.IOException2: remote file operation failed: C:\Windows\TEMP\hudson182664989132439196.bat at hudson.remoting.Channel@42728b7:win7
          	at hudson.FilePath.act(FilePath.java:753)
          	at hudson.FilePath.act(FilePath.java:739)
          	at hudson.FilePath.delete(FilePath.java:994)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:649)
          	at hudson.model.Build$RunnerImpl.build(Build.java:177)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423)
          	at hudson.model.Run.run(Run.java:1362)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:145)
          Caused by: hudson.remoting.ChannelClosedException: channel is already closed
          	at hudson.remoting.Channel.send(Channel.java:466)
          	at hudson.remoting.Request.call(Request.java:105)
          	at hudson.remoting.Channel.call(Channel.java:630)
          	at hudson.FilePath.act(FilePath.java:746)
          	... 13 more
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:985)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2553)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:979)
          FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request.call(Request.java:137)
          	at hudson.remoting.Channel.call(Channel.java:630)
          	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
          	at $Proxy26.join(Unknown Source)
          	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:850)
          	at hudson.Launcher$ProcStarter.join(Launcher.java:336)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)
          	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
          	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:649)
          	at hudson.model.Build$RunnerImpl.build(Build.java:177)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423)
          	at hudson.model.Run.run(Run.java:1362)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:145)
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Request.abort(Request.java:257)
          	at hudson.remoting.Channel.terminate(Channel.java:681)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1003)
          Caused by: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:985)
          Caused by: java.io.EOFException
          	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2553)
          	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
          	at hudson.remoting.Channel$ReaderThread.run(Channel.java:979)
          
          Show
          soundrabbit lukas rytz added a comment - - edited [Edit 2: We only had the issues with JNLP-Slaves. Instead, we now installed Cygwin+OpenSSH on the Windows slave and connect using the SSH Slaves plugin. This works fine.] We see similar problems. Master: Ubuntu x64, 2.6.32-29-server Slave: Windows 7 SP1 x64, VM running in VirtualBox 4.0.4, Java 1.6.0_24-x64 (also tried 32bit JVM, same problem). [Edit: i disabled antivirus, Windows Defender, UAC, I run the jenkins slave as administrator. These don't help.] We also have a Windows XP 32 bit VM, similar configuration, and there it never happens. On the Win7 slave it happens at every build. Here are 5 build, the stack traces are exactly identical 1 2 3 4 5 It looks like it does not happen at random points, but at certain points in the build. FATAL: Unable to delete script file C:\Windows\TEMP\hudson182664989132439196.bat hudson.util.IOException2: remote file operation failed: C:\Windows\TEMP\hudson182664989132439196.bat at hudson.remoting.Channel@42728b7:win7 at hudson.FilePath.act(FilePath.java:753) at hudson.FilePath.act(FilePath.java:739) at hudson.FilePath.delete(FilePath.java:994) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:649) at hudson.model.Build$RunnerImpl.build(Build.java:177) at hudson.model.Build$RunnerImpl.doRun(Build.java:139) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423) at hudson.model.Run.run(Run.java:1362) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:145) Caused by: hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:466) at hudson.remoting.Request.call(Request.java:105) at hudson.remoting.Channel.call(Channel.java:630) at hudson.FilePath.act(FilePath.java:746) ... 13 more Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:985) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2553) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at hudson.remoting.Channel$ReaderThread.run(Channel.java:979) FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.call(Request.java:137) at hudson.remoting.Channel.call(Channel.java:630) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158) at $Proxy26.join(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:850) at hudson.Launcher$ProcStarter.join(Launcher.java:336) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:649) at hudson.model.Build$RunnerImpl.build(Build.java:177) at hudson.model.Build$RunnerImpl.doRun(Build.java:139) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423) at hudson.model.Run.run(Run.java:1362) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:145) Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:257) at hudson.remoting.Channel.terminate(Channel.java:681) at hudson.remoting.Channel$ReaderThread.run(Channel.java:1003) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:985) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2553) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at hudson.remoting.Channel$ReaderThread.run(Channel.java:979)
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          core/src/main/java/hudson/tasks/CommandInterpreter.java
          http://jenkins-ci.org/commit/jenkins/1b22559581abf87344c57ac0dcf21d1612f2c80b
          Log:
          JENKINS-5073

          Suppressed a confusing 2nd error message.

          Compare: https://github.com/jenkinsci/jenkins/compare/786c1d899367...1b22559581ab

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/tasks/CommandInterpreter.java http://jenkins-ci.org/commit/jenkins/1b22559581abf87344c57ac0dcf21d1612f2c80b Log: JENKINS-5073 Suppressed a confusing 2nd error message. Compare: https://github.com/jenkinsci/jenkins/compare/786c1d899367...1b22559581ab
          Hide
          dogfood dogfood added a comment -

          Integrated in jenkins_main_trunk #2887
          JENKINS-5073 (Revision 1b22559581abf87344c57ac0dcf21d1612f2c80b)

          Result = SUCCESS
          kohsuke : 1b22559581abf87344c57ac0dcf21d1612f2c80b
          Files :

          • core/src/main/java/hudson/tasks/CommandInterpreter.java
          Show
          dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2887 JENKINS-5073 (Revision 1b22559581abf87344c57ac0dcf21d1612f2c80b) Result = SUCCESS kohsuke : 1b22559581abf87344c57ac0dcf21d1612f2c80b Files : core/src/main/java/hudson/tasks/CommandInterpreter.java
          Hide
          jglick Jesse Glick added a comment -

          Essentially a duplicate of JENKINS-1948.

          Show
          jglick Jesse Glick added a comment - Essentially a duplicate of JENKINS-1948 .

            People

            • Assignee:
              Unassigned
              Reporter:
              deccico deccico
            • Votes:
              45 Vote for this issue
              Watchers:
              43 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: