Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19453

Slave launcher fails after NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Fixed
    • Component/s: core
    • Environment:
      Core versions: Jenkins-1.509.2, Jenkins-1.509.3, Jenkins-1.509.4-RC
      Master: Windows server 2007 SP1; RHEL6
      Slave: Linux Mint 64 bits; Windows 7 (JNLP)
    • Similar Issues:

      Description

      I am able to start slave.jar via ssh or via jnlp.
      The log of the connection shows that the node is correctly started.
      In a project, I have 2 main parts:

      • a svn checkout
      • a shell command to start cmake + compilation.

      If I launch my build, it hangs. Then, I disabled the svn checkout and leave only the shell command with a "echo hello".
      If I start the build, it hangs with the following error message:

      Started by user anonymous
      [EnvInject] - Loading node environment variables.
      Building remotely on slave_linux_64 in workspace /home/crystal/jenkins/workspace/TESTLinux
      FATAL: command execution failed
      java.io.IOException: Remote call on slave_linux_64 failed
      at hudson.remoting.Channel.call(Channel.java:723)
      at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862)
      at hudson.Launcher$ProcStarter.start(Launcher.java:353)
      at hudson.Launcher$ProcStarter.join(Launcher.java:360)
      at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91)
      at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:60)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
      at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
      at hudson.model.Build$BuildExecution.build(Build.java:199)
      at hudson.model.Build$BuildExecution.doRun(Build.java:160)
      at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586)
      at hudson.model.Run.execute(Run.java:1603)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:247)
      Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer
      at hudson.Launcher$LocalLauncher.<init>(Launcher.java:755)
      at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:991)
      at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:965)
      at hudson.remoting.UserRequest.perform(UserRequest.java:118)
      at hudson.remoting.UserRequest.perform(UserRequest.java:48)
      at hudson.remoting.Request$2.run(Request.java:326)
      at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at hudson.remoting.Engine$1$1.run(Engine.java:63)
      at java.lang.Thread.run(Thread.java:724)
      Build step 'Execute shell' marked build as failure
      Finished: FAILURE

      When I looked at the jenkins directory I configured on the slave, all the required jar are present, the workspace directory is present and the build directory is correctly created by the jenkins master. But nothing in this directory.

        Attachments

          Issue Links

            Activity

            ycollet yann collette created issue -
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            We experience this issue since the migration to 1.509.2. It is being reproduced in 1.509.3 as well.
            Seems that Windows JNLP slaves periodically fail into the death loop

            [EnvInject] - Loading node environment variables.
            Building remotely on XXX in workspace C:\....
            FATAL: command execution failed
            java.io.IOException: Remote call on ru20-custom-codecs-farm09 failed
            at hudson.remoting.Channel.call(Channel.java:681)
            at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862)
            at hudson.Launcher$ProcStarter.start(Launcher.java:353)
            at hudson.Launcher$ProcStarter.join(Launcher.java:360)
            ...
            at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:237)
            Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer
            at hudson.Launcher$LocalLauncher.<init>(Launcher.java:755)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:991)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:965)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:326)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            at hudson.remoting.Engine$1$1.run(Engine.java:58)
            at java.lang.Thread.run(Unknown Source)
            Build step 'Execute Windows batch command' marked build as failure
            Finished: FAILURE

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited We experience this issue since the migration to 1.509.2. It is being reproduced in 1.509.3 as well. Seems that Windows JNLP slaves periodically fail into the death loop [EnvInject] - Loading node environment variables. Building remotely on XXX in workspace C:\.... FATAL: command execution failed java.io.IOException: Remote call on ru20-custom-codecs-farm09 failed at hudson.remoting.Channel.call(Channel.java:681) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862) at hudson.Launcher$ProcStarter.start(Launcher.java:353) at hudson.Launcher$ProcStarter.join(Launcher.java:360) ... at hudson.matrix.MatrixRun.run(MatrixRun.java:146) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:237) Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer at hudson.Launcher$LocalLauncher.<init>(Launcher.java:755) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:991) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:965) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:58) at java.lang.Thread.run(Unknown Source) Build step 'Execute Windows batch command' marked build as failure Finished: FAILURE
            oleg_nenashev Oleg Nenashev made changes -
            Field Original Value New Value
            Environment Master: Windows server 2007 SP1
            Slave: Linux Mint 64 bits
            Core versions: Jenkins-1.509.2, Jenkins-1.509.3
            Master: Windows server 2007 SP1; RHEL6
            Slave: Linux Mint 64 bits; Windows 7 (JNLP)
            Description I am able to start slave.jar via ssh or via jnlp.
            The log of the connection shows that the node is correctly started.
            In a project, I have 2 main parts:
            - a svn checkout
            - a shell command to start cmake + compilation.

            If I launch my build, it hangs. Then, I disabled the svn checkout and leave only the shell command with a "echo hello".
            If I start the build, it hangs with the following error message:


            Started by user anonymous
            [EnvInject] - Loading node environment variables.
            Building remotely on slave_linux_64 in workspace /home/crystal/jenkins/workspace/TESTLinux
            FATAL: command execution failed
            java.io.IOException: Remote call on slave_linux_64 failed
            at hudson.remoting.Channel.call(Channel.java:723)
            at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862)
            at hudson.Launcher$ProcStarter.start(Launcher.java:353)
            at hudson.Launcher$ProcStarter.join(Launcher.java:360)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:60)
            at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
            at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
            at hudson.model.Build$BuildExecution.build(Build.java:199)
            at hudson.model.Build$BuildExecution.doRun(Build.java:160)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586)
            at hudson.model.Run.execute(Run.java:1603)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:247)
            Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer
            at hudson.Launcher$LocalLauncher.<init>(Launcher.java:755)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:991)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:965)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:326)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
            at java.util.concurrent.FutureTask.run(FutureTask.java:166)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            at hudson.remoting.Engine$1$1.run(Engine.java:63)
            at java.lang.Thread.run(Thread.java:724)
            Build step 'Execute shell' marked build as failure
            Finished: FAILURE


            When I looked at the jenkins directory I configured on the slave, all the required jar are present, the workspace directory is present and the build directory is correctly created by the jenkins master. But nothing in this directory.
            I am able to start slave.jar via ssh or via jnlp.
            The log of the connection shows that the node is correctly started.
            In a project, I have 2 main parts:
            - a svn checkout
            - a shell command to start cmake + compilation.

            If I launch my build, it hangs. Then, I disabled the svn checkout and leave only the shell command with a "echo hello".
            If I start the build, it hangs with the following error message:

            {quote}
            Started by user anonymous
            [EnvInject] - Loading node environment variables.
            Building remotely on slave_linux_64 in workspace /home/crystal/jenkins/workspace/TESTLinux
            FATAL: command execution failed
            java.io.IOException: Remote call on slave_linux_64 failed
            at hudson.remoting.Channel.call(Channel.java:723)
            at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862)
            at hudson.Launcher$ProcStarter.start(Launcher.java:353)
            at hudson.Launcher$ProcStarter.join(Launcher.java:360)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:60)
            at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
            at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
            at hudson.model.Build$BuildExecution.build(Build.java:199)
            at hudson.model.Build$BuildExecution.doRun(Build.java:160)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586)
            at hudson.model.Run.execute(Run.java:1603)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:247)
            Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer
            at hudson.Launcher$LocalLauncher.<init>(Launcher.java:755)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:991)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:965)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:326)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
            at java.util.concurrent.FutureTask.run(FutureTask.java:166)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            at hudson.remoting.Engine$1$1.run(Engine.java:63)
            at java.lang.Thread.run(Thread.java:724)
            Build step 'Execute shell' marked build as failure
            Finished: FAILURE
            {quote}

            When I looked at the jenkins directory I configured on the slave, all the required jar are present, the workspace directory is present and the build directory is correctly created by the jenkins master. But nothing in this directory.
            Component/s core [ 15593 ]
            Component/s slave-setup [ 15929 ]
            oleg_nenashev Oleg Nenashev made changes -
            Labels remoting
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Update to remoting-2.32 in 1.509.4 does not solve the issue.
            Seems that is is being reproduced even more frequently.

            Show
            oleg_nenashev Oleg Nenashev added a comment - Update to remoting-2.32 in 1.509.4 does not solve the issue. Seems that is is being reproduced even more frequently.
            oleg_nenashev Oleg Nenashev made changes -
            Labels remoting class remoting
            Environment Core versions: Jenkins-1.509.2, Jenkins-1.509.3
            Master: Windows server 2007 SP1; RHEL6
            Slave: Linux Mint 64 bits; Windows 7 (JNLP)
            Core versions: Jenkins-1.509.2, Jenkins-1.509.3, Jenkins-1.509.4-RC
            Master: Windows server 2007 SP1; RHEL6
            Slave: Linux Mint 64 bits; Windows 7 (JNLP)
            Priority Major [ 3 ] Critical [ 2 ]
            oleg_nenashev Oleg Nenashev made changes -
            Labels class remoting class core remoting
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            According to monitoring statistics, issue has became twice more frequent after migration to 1.509.4

            Seems that issue is caused by initialization of static channel instance, but I cannot catch it in the debugger.

            Show
            oleg_nenashev Oleg Nenashev added a comment - According to monitoring statistics, issue has became twice more frequent after migration to 1.509.4 Seems that issue is caused by initialization of static channel instance, but I cannot catch it in the debugger.
            oleg_nenashev Oleg Nenashev made changes -
            Priority Critical [ 2 ] Blocker [ 1 ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            I’ve found a reproducible scenario for the issue:

            1. Create a node, which has big network delays / small throughput to a master
            2. Reconnect a node to have a fresh classloader
            3. Run the job, which invokes shell/batch script
            4. When the jobs performs a classloading for “Execute shell/batch”, abort the job (you will see the log below)
            5. After that, each job utilizing remote calls will fail with a “java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer” error

            Seems that issue is in incorrect behavior of remote call's during a termination of classloading processes

            A stacktrace from the step #4

            Run condition [Regular expression match] preventing perform for step [Inject environment variables]
            Regular expression run condition: Expression=[.*linux.*], Label=[64bit cn60 cn60-custom-archw-benchmarking custom haps51 win7]
            Run condition [Regular expression match] preventing perform for step [Inject environment variables]
            FATAL: command execution failed
            java.io.InterruptedIOException
            at hudson.Launcher$RemoteLauncher.launch(Launcher.java:864)
            at com.cloudbees.jenkins.plugins.customtools.DecoratedLauncher.launch(DecoratedLauncher.java:49)
            at com.cloudbees.jenkins.plugins.customtools.CustomToolInstallWrapper$2.launch(CustomToolInstallWrapper.java:218)
            at hudson.Launcher$ProcStarter.start(Launcher.java:353)
            at hudson.Launcher$ProcStarter.join(Launcher.java:360)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:60)
            at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
            at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:780)
            at hudson.model.Build$BuildExecution.build(Build.java:199)
            at hudson.model.Build$BuildExecution.doRun(Build.java:160)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:565)
            at hudson.model.Run.execute(Run.java:1592)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:237)
            Caused by: java.lang.InterruptedException
            at java.lang.Object.wait(Native Method)
            at hudson.remoting.Request.call(Request.java:146)
            at hudson.remoting.Channel.call(Channel.java:714)
            at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862)
            ... 15 more
            Build step 'Execute shell' marked build as failure
            Archiving artifacts
            Finished: FAILURE

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited I’ve found a reproducible scenario for the issue: Create a node, which has big network delays / small throughput to a master Reconnect a node to have a fresh classloader Run the job, which invokes shell/batch script When the jobs performs a classloading for “Execute shell/batch”, abort the job (you will see the log below) After that, each job utilizing remote calls will fail with a “java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer” error Seems that issue is in incorrect behavior of remote call's during a termination of classloading processes A stacktrace from the step #4 Run condition [Regular expression match] preventing perform for step [Inject environment variables] Regular expression run condition: Expression= [.*linux.*] , Label= [64bit cn60 cn60-custom-archw-benchmarking custom haps51 win7] Run condition [Regular expression match] preventing perform for step [Inject environment variables] FATAL: command execution failed java.io.InterruptedIOException at hudson.Launcher$RemoteLauncher.launch(Launcher.java:864) at com.cloudbees.jenkins.plugins.customtools.DecoratedLauncher.launch(DecoratedLauncher.java:49) at com.cloudbees.jenkins.plugins.customtools.CustomToolInstallWrapper$2.launch(CustomToolInstallWrapper.java:218) at hudson.Launcher$ProcStarter.start(Launcher.java:353) at hudson.Launcher$ProcStarter.join(Launcher.java:360) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:60) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:780) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:565) at hudson.model.Run.execute(Run.java:1592) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:237) Caused by: java.lang.InterruptedException at java.lang.Object.wait(Native Method) at hudson.remoting.Request.call(Request.java:146) at hudson.remoting.Channel.call(Channel.java:714) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862) ... 15 more Build step 'Execute shell' marked build as failure Archiving artifacts Finished: FAILURE
            jglick Jesse Glick made changes -
            Summary Slave fails to launch commands Slave launcher fails after NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer
            Labels class core remoting remoting
            Component/s slave-setup [ 15929 ]
            Component/s slave-utilization [ 17575 ]
            Hide
            jglick Jesse Glick added a comment -

            Have not tried to reproduce this yet, but I am guessing that the problem lies in RemoteClassLoader since it rethrows an InterruptedException as a ClassNotFoundException. So if some task which was doing class loading is interrupted by the user, the class loader remembers the fact that the class in progress (Jenkins?) was not found, marks the class referring to it (MasterComputer?) as uninitialized, and thereafter refuses to let any code (LocalLauncher) use the referring class.

            (A similar problem might occur when an IOException is wrapped as a ClassNotFoundException, but I guess this is less likely, since an IOException probably means either that class loading on the master is broken too and Jenkins is hosed, or that the socket used by the remoting channel is down and the slave agent is as good as dead.)

            If that is the right diagnosis, I am unsure how to fix it. As far as I know Java offers no way for a ClassLoader to acknowledge that initialization of a class failed earlier with a NoClassDefFoundError but that this was a transient error and it should be retried. Maybe throwing a more serious error (such as ThreadStop) up the call chain will stop it from caching the unwanted state, but I doubt it—the caching seems to be done in native code on Oracle JREs at least.

            Perhaps it would be possible to notice these transient failures during class loading and simply kill off the whole RemoteClassLoader and recreate it? Not exactly sure how that would work.

            Show
            jglick Jesse Glick added a comment - Have not tried to reproduce this yet, but I am guessing that the problem lies in RemoteClassLoader since it rethrows an InterruptedException as a ClassNotFoundException . So if some task which was doing class loading is interrupted by the user, the class loader remembers the fact that the class in progress ( Jenkins ?) was not found, marks the class referring to it ( MasterComputer ?) as uninitialized, and thereafter refuses to let any code ( LocalLauncher ) use the referring class. (A similar problem might occur when an IOException is wrapped as a ClassNotFoundException , but I guess this is less likely, since an IOException probably means either that class loading on the master is broken too and Jenkins is hosed, or that the socket used by the remoting channel is down and the slave agent is as good as dead.) If that is the right diagnosis, I am unsure how to fix it. As far as I know Java offers no way for a ClassLoader to acknowledge that initialization of a class failed earlier with a NoClassDefFoundError but that this was a transient error and it should be retried. Maybe throwing a more serious error (such as ThreadStop ) up the call chain will stop it from caching the unwanted state, but I doubt it—the caching seems to be done in native code on Oracle JREs at least. Perhaps it would be possible to notice these transient failures during class loading and simply kill off the whole RemoteClassLoader and recreate it? Not exactly sure how that would work.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Is it possible to somehow synchronize Channel::terminate() with ClassLoader operations?
            It may provide a workaround for the described scenario.

            Show
            oleg_nenashev Oleg Nenashev added a comment - Is it possible to somehow synchronize Channel::terminate() with ClassLoader operations? It may provide a workaround for the described scenario.
            Hide
            jglick Jesse Glick added a comment -

            I think I have reproduced this scenario in a unit test, though there are complications that I would need @kohsuke’s assistance with.

            Show
            jglick Jesse Glick added a comment - I think I have reproduced this scenario in a unit test, though there are complications that I would need @kohsuke’s assistance with.
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/hudson/remoting/RemoteClassLoader.java
            src/test/java/hudson/remoting/ClassRemotingTest.java
            src/test/java/hudson/remoting/TestLinkage.java
            http://jenkins-ci.org/commit/remoting/5532ffe6c987278cb0acbb1e6cdce96075a0e52f
            Log:
            JENKINS-19453 Seem to have reproduced the problem in a unit test.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/remoting/RemoteClassLoader.java src/test/java/hudson/remoting/ClassRemotingTest.java src/test/java/hudson/remoting/TestLinkage.java http://jenkins-ci.org/commit/remoting/5532ffe6c987278cb0acbb1e6cdce96075a0e52f Log: JENKINS-19453 Seem to have reproduced the problem in a unit test.
            jglick Jesse Glick made changes -
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/main/java/hudson/remoting/RemoteClassLoader.java
            src/test/java/hudson/remoting/ClassRemotingTest.java
            src/test/java/hudson/remoting/DummyClassLoader.java
            src/test/java/hudson/remoting/TestLinkage.java
            http://jenkins-ci.org/commit/remoting/4efb4e1dd5675a5d22993b09ebb65adab5baa9e0
            Log:
            JENKINS-19453 Propose fix: mask interrupt while loading a class

            I think the crux of the issue is that, in general, a failed class loading
            will never get retried. It is certainly the case when another class loading has triggered a
            recurisve class loading as in this test case.

            In https://github.com/jenkinsci/remoting/pull/19 Jesse proposed to fix
            this by dropping the whole RemoteClassLoader. But this will not work
            since the said RemoteClassLoader cloud have loaded other classes that
            might be already running. Dropping a classloader will not drop these
            classes, and we end up just creating another classloader that loads
            incompatible classes. That is a disaster waiting to happen.

            In this fix, we simply make findClass non-interruptible. That is, if a
            blocking remote operation gets interrupted, we catch that and simply
            retry and refuse to give up. We'll remember to set the interrupt flag
            back on, however, so that the interrupt signal doesn't disappear.

            The net effect is as if the delivery of the interrupt was bit late than
            usual. The way I see it, this is the only way to fix this.

            The nesting of control structures here is horrible, but I can't think of
            any better way to write this.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/remoting/RemoteClassLoader.java src/test/java/hudson/remoting/ClassRemotingTest.java src/test/java/hudson/remoting/DummyClassLoader.java src/test/java/hudson/remoting/TestLinkage.java http://jenkins-ci.org/commit/remoting/4efb4e1dd5675a5d22993b09ebb65adab5baa9e0 Log: JENKINS-19453 Propose fix: mask interrupt while loading a class I think the crux of the issue is that, in general, a failed class loading will never get retried. It is certainly the case when another class loading has triggered a recurisve class loading as in this test case. In https://github.com/jenkinsci/remoting/pull/19 Jesse proposed to fix this by dropping the whole RemoteClassLoader. But this will not work since the said RemoteClassLoader cloud have loaded other classes that might be already running. Dropping a classloader will not drop these classes, and we end up just creating another classloader that loads incompatible classes. That is a disaster waiting to happen. In this fix, we simply make findClass non-interruptible. That is, if a blocking remote operation gets interrupted, we catch that and simply retry and refuse to give up. We'll remember to set the interrupt flag back on, however, so that the interrupt signal doesn't disappear. The net effect is as if the delivery of the interrupt was bit late than usual. The way I see it, this is the only way to fix this. The nesting of control structures here is horrible, but I can't think of any better way to write this.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/hudson/remoting/RemoteClassLoader.java
            src/test/java/hudson/remoting/ClassRemotingTest.java
            src/test/java/hudson/remoting/TestLinkage.java
            http://jenkins-ci.org/commit/remoting/b68e3e92b24942585fff517bb4bad1878883256f
            Log:
            JENKINS-19453 Seem to have reproduced the problem in a unit test.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/remoting/RemoteClassLoader.java src/test/java/hudson/remoting/ClassRemotingTest.java src/test/java/hudson/remoting/TestLinkage.java http://jenkins-ci.org/commit/remoting/b68e3e92b24942585fff517bb4bad1878883256f Log: JENKINS-19453 Seem to have reproduced the problem in a unit test.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/main/java/hudson/remoting/RemoteClassLoader.java
            src/test/java/hudson/remoting/ClassRemotingTest.java
            src/test/java/hudson/remoting/DummyClassLoader.java
            src/test/java/hudson/remoting/TestLinkage.java
            http://jenkins-ci.org/commit/remoting/425301511aeb5a876b224cf4e572b74447e08ebc
            Log:
            [FIXED JENKINS-19453] Propose fix: mask interrupt while loading a class

            I think the crux of the issue is that, in general, a failed class loading
            will never get retried. It is certainly the case when another class loading has triggered a
            recurisve class loading as in this test case.

            In https://github.com/jenkinsci/remoting/pull/19 Jesse proposed to fix
            this by dropping the whole RemoteClassLoader. But this will not work
            since the said RemoteClassLoader cloud have loaded other classes that
            might be already running. Dropping a classloader will not drop these
            classes, and we end up just creating another classloader that loads
            incompatible classes. That is a disaster waiting to happen.

            In this fix, we simply make findClass non-interruptible. That is, if a
            blocking remote operation gets interrupted, we catch that and simply
            retry and refuse to give up. We'll remember to set the interrupt flag
            back on, however, so that the interrupt signal doesn't disappear.

            The net effect is as if the delivery of the interrupt was bit late than
            usual. The way I see it, this is the only way to fix this.

            The nesting of control structures here is horrible, but I can't think of
            any better way to write this.

            Compare: https://github.com/jenkinsci/remoting/compare/bfbc7d693a50...425301511aeb

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/remoting/RemoteClassLoader.java src/test/java/hudson/remoting/ClassRemotingTest.java src/test/java/hudson/remoting/DummyClassLoader.java src/test/java/hudson/remoting/TestLinkage.java http://jenkins-ci.org/commit/remoting/425301511aeb5a876b224cf4e572b74447e08ebc Log: [FIXED JENKINS-19453] Propose fix: mask interrupt while loading a class I think the crux of the issue is that, in general, a failed class loading will never get retried. It is certainly the case when another class loading has triggered a recurisve class loading as in this test case. In https://github.com/jenkinsci/remoting/pull/19 Jesse proposed to fix this by dropping the whole RemoteClassLoader. But this will not work since the said RemoteClassLoader cloud have loaded other classes that might be already running. Dropping a classloader will not drop these classes, and we end up just creating another classloader that loads incompatible classes. That is a disaster waiting to happen. In this fix, we simply make findClass non-interruptible. That is, if a blocking remote operation gets interrupted, we catch that and simply retry and refuse to give up. We'll remember to set the interrupt flag back on, however, so that the interrupt signal doesn't disappear. The net effect is as if the delivery of the interrupt was bit late than usual. The way I see it, this is the only way to fix this. The nesting of control structures here is horrible, but I can't think of any better way to write this. Compare: https://github.com/jenkinsci/remoting/compare/bfbc7d693a50...425301511aeb
            scm_issue_link SCM/JIRA link daemon made changes -
            Status In Progress [ 3 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            oleg_nenashev Oleg Nenashev made changes -
            Labels remoting lts-candidate remoting
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Thanks for fix!
            I've marked the issue as "lts-candidate".

            Show
            oleg_nenashev Oleg Nenashev added a comment - Thanks for fix! I've marked the issue as "lts-candidate".
            Hide
            jglick Jesse Glick added a comment -

            Not actually fixed yet, since core still uses remoting 2.32; fix is in remoting 2.33. @kohsuke including the FIXED text in the commit comment for a subcomponent is a bad idea for this reason—the JIRA updater just assumes the fix is actually integrated when this commit appears in the master branch of its repository.

            Show
            jglick Jesse Glick added a comment - Not actually fixed yet, since core still uses remoting 2.32; fix is in remoting 2.33. @kohsuke including the FIXED text in the commit comment for a subcomponent is a bad idea for this reason—the JIRA updater just assumes the fix is actually integrated when this commit appears in the master branch of its repository.
            jglick Jesse Glick made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            Assignee Kohsuke Kawaguchi [ kohsuke ]
            Hide
            jglick Jesse Glick added a comment -

            Actually fixed just now in https://github.com/jenkinsci/jenkins/commit/4137c46182a0652d2eed2897a1e797bdc3f8c7f9 which did not record any issue IDs in its comment so it would not have been noticed by the JIRA link daemon.

            Show
            jglick Jesse Glick added a comment - Actually fixed just now in https://github.com/jenkinsci/jenkins/commit/4137c46182a0652d2eed2897a1e797bdc3f8c7f9 which did not record any issue IDs in its comment so it would not have been noticed by the JIRA link daemon.
            jglick Jesse Glick made changes -
            Status Reopened [ 4 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            ycollette yann collette added a comment -

            Tested with 1.542 and it works now for me.
            Thanks a lot,

            YC

            Show
            ycollette yann collette added a comment - Tested with 1.542 and it works now for me. Thanks a lot, YC
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oliver Gondža
            Path:
            http://jenkins-ci.org/commit/jenkins/5cf3e28c4885a86a4dad3463952e1aae34932f9c
            Log:
            [FIXED JENKINS-20093 JENKINS-19453 JENKINS-19004 JENKINS-8856] Noting retroactively

            Backported as 23424ea7dbced486963a73d6bf56e8367ab779dd

            Compare: https://github.com/jenkinsci/jenkins/compare/7d602d52b445...5cf3e28c4885

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oliver Gondža Path: http://jenkins-ci.org/commit/jenkins/5cf3e28c4885a86a4dad3463952e1aae34932f9c Log: [FIXED JENKINS-20093 JENKINS-19453 JENKINS-19004 JENKINS-8856] Noting retroactively Backported as 23424ea7dbced486963a73d6bf56e8367ab779dd Compare: https://github.com/jenkinsci/jenkins/compare/7d602d52b445...5cf3e28c4885
            olivergondza Oliver Gondža made changes -
            Labels lts-candidate remoting 1.532.2-fixed remoting
            Show
            jglick Jesse Glick added a comment - Build of this alone applied to 1.532.1: http://maven.jenkins-ci.org:8081/content/repositories/releases/org/jenkins-ci/main/jenkins-war/1.532.1.JENKINS-19453/jenkins-war-1.532.1.JENKINS-19453.war
            Hide
            eguess74 eguess74 added a comment -

            For the record I just saw this error in 1.553 One of the slaves started to fail the builds with this error and i had to disconnect it and then reconnect. That seemed to have resolved the issue for now.

            Show
            eguess74 eguess74 added a comment - For the record I just saw this error in 1.553 One of the slaves started to fail the builds with this error and i had to disconnect it and then reconnect. That seemed to have resolved the issue for now.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            @eguess74
            Please check that slave's version is 2.33 or greater.
            In some cases (e.g. for Windows services) you may have to update the slave.jar manually

            Show
            oleg_nenashev Oleg Nenashev added a comment - @eguess74 Please check that slave's version is 2.33 or greater. In some cases (e.g. for Windows services) you may have to update the slave.jar manually
            Hide
            eguess74 eguess74 added a comment -

            @Oleg
            Unix slave 2.33

            Show
            eguess74 eguess74 added a comment - @Oleg Unix slave 2.33
            Hide
            chotchki chotchki added a comment -

            I am still seeing an issue with a Windows client on 2.33. The stack trace is as follows:

            hudson.remoting.Channel@7c2ac188:Win7x86
            at hudson.FilePath.act(FilePath.java:907)
            at hudson.FilePath.act(FilePath.java:884)
            at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:935)
            at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:870)
            at hudson.model.AbstractProject.checkout(AbstractProject.java:1411)
            at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
            at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561)
            at hudson.model.Run.execute(Run.java:1665)
            at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:246)
            Caused by: java.io.IOException: Remote call on Win7x86 failed
            at hudson.remoting.Channel.call(Channel.java:731)
            at hudson.FilePath.act(FilePath.java:900)
            ... 11 more
            Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins
            at hudson.scm.SubversionSCM.descriptor(SubversionSCM.java:2504)
            at hudson.scm.SubversionSCM.createDefaultSVNOptions(SubversionSCM.java:1084)
            at hudson.scm.SubversionSCM.createClientManager(SubversionSCM.java:1074)
            at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1006)
            at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:986)
            at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:328)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            at hudson.remoting.Engine$1$1.run(Engine.java:63)
            at java.lang.Thread.run(Unknown Source)
            An attempt to send an e-mail to empty list of recipients, ignored.
            Finished: FAILURE

            Show
            chotchki chotchki added a comment - I am still seeing an issue with a Windows client on 2.33. The stack trace is as follows: hudson.remoting.Channel@7c2ac188:Win7x86 at hudson.FilePath.act(FilePath.java:907) at hudson.FilePath.act(FilePath.java:884) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:935) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:870) at hudson.model.AbstractProject.checkout(AbstractProject.java:1411) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561) at hudson.model.Run.execute(Run.java:1665) at hudson.matrix.MatrixRun.run(MatrixRun.java:146) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:246) Caused by: java.io.IOException: Remote call on Win7x86 failed at hudson.remoting.Channel.call(Channel.java:731) at hudson.FilePath.act(FilePath.java:900) ... 11 more Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins at hudson.scm.SubversionSCM.descriptor(SubversionSCM.java:2504) at hudson.scm.SubversionSCM.createDefaultSVNOptions(SubversionSCM.java:1084) at hudson.scm.SubversionSCM.createClientManager(SubversionSCM.java:1074) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1006) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:986) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) An attempt to send an e-mail to empty list of recipients, ignored. Finished: FAILURE
            Hide
            chotchki chotchki added a comment -

            I just upgraded to 1.532.2 and this issue is still occurring. I also manually upgraded to 2.33 on the slave.

            Show
            chotchki chotchki added a comment - I just upgraded to 1.532.2 and this issue is still occurring. I also manually upgraded to 2.33 on the slave.
            chotchki chotchki made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            Hide
            jglick Jesse Glick added a comment -

            @chotchki a particular bug was found, reproduced, and fixed. Whatever you are seeing might be related, or it might not. Better to file it separately, with whatever details can be found (steps to reproduce if possible), and link to this one.

            Show
            jglick Jesse Glick added a comment - @chotchki a particular bug was found, reproduced, and fixed. Whatever you are seeing might be related, or it might not. Better to file it separately, with whatever details can be found (steps to reproduce if possible), and link to this one.
            jglick Jesse Glick made changes -
            Status Reopened [ 4 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            I'm seeing this problem in another instance, and slave log says it's failing to write to local jar file cache. I wonder if that could somehow cause this problem?

            https://gist.github.com/kohsuke/f54d4c40094f2d035cf9

            Show
            kohsuke Kohsuke Kawaguchi added a comment - I'm seeing this problem in another instance, and slave log says it's failing to write to local jar file cache. I wonder if that could somehow cause this problem? https://gist.github.com/kohsuke/f54d4c40094f2d035cf9
            Hide
            lil2006 Igor Tereshchuk added a comment -

            I got such issue on build Jenkins ver. 1.558
            slave connected by jnlp
            Here is stack trace:
            FATAL: command execution failed
            java.io.IOException: Remote call on slave_windows failed
            at hudson.remoting.Channel.call(Channel.java:731)
            at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862)
            at hudson.Launcher$ProcStarter.start(Launcher.java:353)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
            at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
            at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
            at hudson.model.Build$BuildExecution.build(Build.java:198)
            at hudson.model.Build$BuildExecution.doRun(Build.java:159)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:523)
            at hudson.model.Run.execute(Run.java:1689)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:231)
            Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer
            at hudson.Launcher$LocalLauncher.<init>(Launcher.java:755)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:991)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:965)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:328)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            at hudson.remoting.Engine$1$1.run(Engine.java:63)
            at java.lang.Thread.run(Unknown Source)
            Build step 'Execute Windows batch command' marked build as failure
            FATAL: Remote call on adft-temp-s failed
            java.io.IOException: Remote call on adft-temp-s failed
            at hudson.remoting.Channel.call(Channel.java:731)
            at hudson.Launcher$RemoteLauncher.kill(Launcher.java:887)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:546)
            at hudson.model.Run.execute(Run.java:1689)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:231)
            Caused by: java.lang.Error: Unable to load resource winp.dll
            at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:380)
            at java.lang.ClassLoader.getResource(Unknown Source)
            at org.jvnet.winp.Native.load(Native.java:84)
            at org.jvnet.winp.Native.<clinit>(Native.java:55)
            at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:200)
            at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:477)
            at hudson.util.ProcessTree.get(ProcessTree.java:336)
            at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:899)
            at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:890)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:328)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            at hudson.remoting.Engine$1$1.run(Engine.java:63)
            at java.lang.Thread.run(Unknown Source)
            Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Unable to call writeJarTo. Invalid object ID 2
            at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:75)
            at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
            at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:372)
            ... 17 more
            Caused by: java.lang.IllegalStateException: Unable to call writeJarTo. Invalid object ID 2
            at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:293)
            at hudson.remoting.Request$2.run(Request.java:328)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
            at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            ... 1 more

            Show
            lil2006 Igor Tereshchuk added a comment - I got such issue on build Jenkins ver. 1.558 slave connected by jnlp Here is stack trace: FATAL: command execution failed java.io.IOException: Remote call on slave_windows failed at hudson.remoting.Channel.call(Channel.java:731) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:862) at hudson.Launcher$ProcStarter.start(Launcher.java:353) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.Build$BuildExecution.build(Build.java:198) at hudson.model.Build$BuildExecution.doRun(Build.java:159) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:523) at hudson.model.Run.execute(Run.java:1689) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins$MasterComputer at hudson.Launcher$LocalLauncher.<init>(Launcher.java:755) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:991) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:965) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) Build step 'Execute Windows batch command' marked build as failure FATAL: Remote call on adft-temp-s failed java.io.IOException: Remote call on adft-temp-s failed at hudson.remoting.Channel.call(Channel.java:731) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:887) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:546) at hudson.model.Run.execute(Run.java:1689) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Caused by: java.lang.Error: Unable to load resource winp.dll at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:380) at java.lang.ClassLoader.getResource(Unknown Source) at org.jvnet.winp.Native.load(Native.java:84) at org.jvnet.winp.Native.<clinit>(Native.java:55) at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:200) at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:477) at hudson.util.ProcessTree.get(ProcessTree.java:336) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:899) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:890) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Unable to call writeJarTo. Invalid object ID 2 at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:75) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:372) ... 17 more Caused by: java.lang.IllegalStateException: Unable to call writeJarTo. Invalid object ID 2 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:293) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 more
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-15796 [ JENKINS-15796 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is blocking JENKINS-15199 [ JENKINS-15199 ]
            Hide
            vikulin Vadym Vikulin added a comment - - edited

            Reproduced on 1.585:

            Started by user Vadym Vikulin
            [EnvInject] - Loading node environment variables.
            Building remotely on win7-node-01 (Win7) in workspace C:\work\workspace\Win7-ED
            [EnvInject] - Executing scripts and injecting environment variables after the SCM step.
            [EnvInject] - Injecting as environment variables the properties content
            CLASSPATH=C:\work\edtest\common\libs*:C:\work\edtest\common;C:\work\edtest\all\edJava;C:\work\edtest\all\edNative/libs/*

            [EnvInject] - Variables injected successfully.
            [Win7-ED] $ cmd /c call C:\Users\VADYMV~1\AppData\Local\Temp\hudson142281969965340690.bat
            FATAL: command execution failed
            java.io.IOException: Remote call on win7-node-01 failed
            at hudson.remoting.Channel.call(Channel.java:751)
            at hudson.Launcher$RemoteLauncher.launch(Launcher.java:915)
            at hudson.Launcher$ProcStarter.start(Launcher.java:380)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
            at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
            at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770)
            at hudson.model.Build$BuildExecution.build(Build.java:199)
            at hudson.model.Build$BuildExecution.doRun(Build.java:160)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533)
            at hudson.model.Run.execute(Run.java:1759)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
            at hudson.model.ResourceController.execute(ResourceController.java:89)
            at hudson.model.Executor.run(Executor.java:240)
            Caused by: java.lang.NoClassDefFoundError: Could not initialize class hudson.Util
            at hudson.EnvVars$OverrideOrderCalculator.scan(EnvVars.java:281)
            at hudson.EnvVars$OverrideOrderCalculator.<init>(EnvVars.java:223)
            at hudson.EnvVars.overrideExpandingAll(EnvVars.java:329)
            at hudson.Launcher.inherit(Launcher.java:1227)
            at hudson.Launcher.inherit(Launcher.java:1219)
            at hudson.Launcher.access$200(Launcher.java:81)
            at hudson.Launcher$LocalLauncher.launch(Launcher.java:795)
            at hudson.Launcher$ProcStarter.start(Launcher.java:380)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1135)
            at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1100)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:328)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            at hudson.remoting.Engine$1$1.run(Engine.java:63)
            at java.lang.Thread.run(Unknown Source)
            FATAL: Unable to delete script file C:\Users\VADYMV~1\AppData\Local\Temp\hudson142281969965340690.bat
            java.io.IOException: remote file operation failed: C:\Users\VADYMV~1\AppData\Local\Temp\hudson142281969965340690.bat at hudson.remoting.Channel@5dc020b1:win7-node-01
            at hudson.FilePath.act(FilePath.java:914)
            at hudson.FilePath.act(FilePath.java:896)
            at hudson.FilePath.delete(FilePath.java:1321)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:106)
            at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
            at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
            at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770)
            at hudson.model.Build$BuildExecution.build(Build.java:199)
            at hudson.model.Build$BuildExecution.doRun(Build.java:160)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533)
            at hudson.model.Run.execute(Run.java:1759)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
            at hudson.model.ResourceController.execute(ResourceController.java:89)
            at hudson.model.Executor.run(Executor.java:240)
            Caused by: java.io.IOException: Remote call on win7-node-01 failed
            at hudson.remoting.Channel.call(Channel.java:751)
            at hudson.FilePath.act(FilePath.java:907)
            ... 13 more

            Show
            vikulin Vadym Vikulin added a comment - - edited Reproduced on 1.585: Started by user Vadym Vikulin [EnvInject] - Loading node environment variables. Building remotely on win7-node-01 (Win7) in workspace C:\work\workspace\Win7-ED [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content CLASSPATH=C:\work\edtest\common\libs*:C:\work\edtest\common;C:\work\edtest\all\edJava;C:\work\edtest\all\edNative/libs/* [EnvInject] - Variables injected successfully. [Win7-ED] $ cmd /c call C:\Users\VADYMV~1\AppData\Local\Temp\hudson142281969965340690.bat FATAL: command execution failed java.io.IOException: Remote call on win7-node-01 failed at hudson.remoting.Channel.call(Channel.java:751) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:915) at hudson.Launcher$ProcStarter.start(Launcher.java:380) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533) at hudson.model.Run.execute(Run.java:1759) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:89) at hudson.model.Executor.run(Executor.java:240) Caused by: java.lang.NoClassDefFoundError: Could not initialize class hudson.Util at hudson.EnvVars$OverrideOrderCalculator.scan(EnvVars.java:281) at hudson.EnvVars$OverrideOrderCalculator.<init>(EnvVars.java:223) at hudson.EnvVars.overrideExpandingAll(EnvVars.java:329) at hudson.Launcher.inherit(Launcher.java:1227) at hudson.Launcher.inherit(Launcher.java:1219) at hudson.Launcher.access$200(Launcher.java:81) at hudson.Launcher$LocalLauncher.launch(Launcher.java:795) at hudson.Launcher$ProcStarter.start(Launcher.java:380) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1135) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1100) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) FATAL: Unable to delete script file C:\Users\VADYMV~1\AppData\Local\Temp\hudson142281969965340690.bat java.io.IOException: remote file operation failed: C:\Users\VADYMV~1\AppData\Local\Temp\hudson142281969965340690.bat at hudson.remoting.Channel@5dc020b1:win7-node-01 at hudson.FilePath.act(FilePath.java:914) at hudson.FilePath.act(FilePath.java:896) at hudson.FilePath.delete(FilePath.java:1321) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:106) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533) at hudson.model.Run.execute(Run.java:1759) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:89) at hudson.model.Executor.run(Executor.java:240) Caused by: java.io.IOException: Remote call on win7-node-01 failed at hudson.remoting.Channel.call(Channel.java:751) at hudson.FilePath.act(FilePath.java:907) ... 13 more
            Hide
            jglick Jesse Glick added a comment -

            Vadym Vikulin your problem is probably distinct and should be filed separately. Look for an earlier message explaining why hudson.Util could not be loaded.

            Show
            jglick Jesse Glick added a comment - Vadym Vikulin your problem is probably distinct and should be filed separately. Look for an earlier message explaining why hudson.Util could not be loaded.
            vikulin Vadym Vikulin made changes -
            Link This issue is related to JENKINS-25316 [ JENKINS-25316 ]
            Hide
            jeebitesh Jeebitesh Kalantri added a comment -

            Solution :- If you are running slave on windows VM\Server with 64 bit OS
            Make sure of few things
            1) You should have 32 bit JRE Installed
            2) It should be default JRE - you go to configure Java and change the settings if you have 64 bit and 32 bit both change the default to 32 bit path.
            3) Restart the Slave and that's it, Vola

            Show
            jeebitesh Jeebitesh Kalantri added a comment - Solution :- If you are running slave on windows VM\Server with 64 bit OS Make sure of few things 1) You should have 32 bit JRE Installed 2) It should be default JRE - you go to configure Java and change the settings if you have 64 bit and 32 bit both change the default to 32 bit path. 3) Restart the Slave and that's it, Vola
            Hide
            svchipiga Sergey Chipiga added a comment - - edited

            Hi!
            I have still this problem (Jenkins ver. 1.580.2):
            <code>
            FATAL: Remote call on win7-home-premium-x86-ru-1-f8e failed
            20:33:18 java.io.IOException: Remote call on win7-home-premium-x86-ru-1-f8e failed
            20:33:18 at hudson.remoting.Channel.call(Channel.java:760)
            20:33:18 at hudson.Launcher$RemoteLauncher.kill(Launcher.java:941)
            20:33:18 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:556)
            20:33:18 at hudson.model.Run.execute(Run.java:1745)
            20:33:18 at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
            20:33:18 at hudson.model.ResourceController.execute(ResourceController.java:89)
            20:33:18 at hudson.model.Executor.run(Executor.java:240)
            20:33:18 Caused by: java.lang.Error: Unable to load resource winp.dll
            20:33:18 at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:381)
            20:33:18 at java.lang.ClassLoader.getResource(Unknown Source)
            20:33:18 at org.jvnet.winp.Native.load(Native.java:85)
            20:33:18 at org.jvnet.winp.Native.<clinit>(Native.java:56)
            20:33:18 at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:212)
            20:33:18 at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:469)
            20:33:18 at hudson.util.ProcessTree.get(ProcessTree.java:328)
            20:33:18 at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:953)
            20:33:18 at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:944)
            20:33:18 at hudson.remoting.UserRequest.perform(UserRequest.java:121)
            20:33:18 at hudson.remoting.UserRequest.perform(UserRequest.java:49)
            20:33:18 at hudson.remoting.Request$2.run(Request.java:324)
            20:33:18 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
            20:33:18 at java.util.concurrent.FutureTask.run(Unknown Source)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            20:33:18 at hudson.remoting.Engine$1$1.run(Engine.java:63)
            20:33:18 at java.lang.Thread.run(Unknown Source)
            20:33:18 at ......remote call to yabrowser-win7-home-premium-x86-ru-1-f8e(Native Method)
            20:33:18 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356)
            20:33:18 at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
            20:33:18 at hudson.remoting.Channel.call(Channel.java:752)
            20:33:18 ... 6 more
            20:33:18 Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Invalid object ID 2 iota=14
            20:33:18 at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:75)
            20:33:18 at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
            20:33:18 at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:373)
            20:33:18 at java.lang.ClassLoader.getResource(Unknown Source)
            20:33:18 at org.jvnet.winp.Native.load(Native.java:85)
            20:33:18 at org.jvnet.winp.Native.<clinit>(Native.java:56)
            20:33:18 at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:212)
            20:33:18 at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:469)
            20:33:18 at hudson.util.ProcessTree.get(ProcessTree.java:328)
            20:33:18 at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:953)
            20:33:18 at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:944)
            20:33:18 at hudson.remoting.UserRequest.perform(UserRequest.java:121)
            20:33:18 at hudson.remoting.UserRequest.perform(UserRequest.java:49)
            20:33:18 at hudson.remoting.Request$2.run(Request.java:324)
            20:33:18 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
            20:33:18 at java.util.concurrent.FutureTask.run(Unknown Source)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            20:33:18 at hudson.remoting.Engine$1$1.run(Engine.java:63)
            20:33:18 at java.lang.Thread.run(Unknown Source)
            20:33:18 Caused by: java.lang.IllegalStateException: Invalid object ID 2 iota=14
            20:33:18 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:348)
            20:33:18 at hudson.remoting.ExportTable.get(ExportTable.java:324)
            20:33:18 at hudson.remoting.Channel.getExportedObject(Channel.java:604)
            20:33:18 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:311)
            20:33:18 at hudson.remoting.Request$2.run(Request.java:324)
            20:33:18 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
            20:33:18 at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
            20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            20:33:18 at java.lang.Thread.run(Thread.java:744)
            20:33:18 at ......remote call to channel(Native Method)
            20:33:18 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356)
            20:33:18 at hudson.remoting.Request.call(Request.java:171)
            20:33:18 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:176)
            20:33:18 at com.sun.proxy.$Proxy5.writeJarTo(Unknown Source)
            20:33:18 at hudson.remoting.FileSystemJarCache.retrieve(FileSystemJarCache.java:66)
            20:33:18 at hudson.remoting.JarCacheSupport$1.run(JarCacheSupport.java:62)
            20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
            20:33:18 at java.util.concurrent.FutureTask.run(Unknown Source)
            20:33:18 at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:104)
            20:33:18 ... 1 more
            20:33:18 Caused by: java.lang.Exception: Object was recently deallocated
            20:33:18 #2 (ref.0) : [hudson.remoting.JarLoader]
            20:33:18 Created at Wed Dec 24 20:28:20 MSK 2014
            20:33:18 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:93)
            20:33:18 at hudson.remoting.ExportTable.export(ExportTable.java:299)
            20:33:18 at hudson.remoting.Channel.internalExport(Channel.java:600)
            20:33:18 at hudson.remoting.Channel.export(Channel.java:591)
            20:33:18 at hudson.remoting.Channel.<init>(Channel.java:472)
            20:33:18 at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:280)
            20:33:18 at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:290)
            20:33:18 at org.jenkinsci.remoting.nio.NioChannelBuilder.build(NioChannelBuilder.java:36)
            20:33:18 at org.jenkinsci.remoting.nio.NioChannelBuilder.build(NioChannelBuilder.java:52)
            20:33:18 at jenkins.slaves.JnlpSlaveAgentProtocol$Handler.jnlpConnect(JnlpSlaveAgentProtocol.java:120)
            20:33:18 at jenkins.slaves.DefaultJnlpSlaveReceiver.handle(DefaultJnlpSlaveReceiver.java:63)
            20:33:18 at jenkins.slaves.JnlpSlaveAgentProtocol2$Handler2.run(JnlpSlaveAgentProtocol2.java:57)
            20:33:18 at jenkins.slaves.JnlpSlaveAgentProtocol2.handle(JnlpSlaveAgentProtocol2.java:31)
            20:33:18 at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:156)
            20:33:18 Released at Wed Dec 24 20:28:20 MSK 2014
            20:33:18 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:125)
            20:33:18 at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:376)
            20:33:18 at hudson.remoting.Channel.unexport(Channel.java:612)
            20:33:18 at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38)
            20:33:18 at hudson.remoting.Channel$2.handle(Channel.java:483)
            20:33:18 at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61)
            20:33:18 at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:597)
            20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
            20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
            20:33:18 at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
            20:33:18 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
            20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            20:33:18 at java.lang.Thread.run(Thread.java:744)
            20:33:18 Caused by: Command hudson.remoting.UnexportCommand@1787906e created at
            20:33:18 at hudson.remoting.Command.<init>(Command.java:67)
            20:33:18 at hudson.remoting.Command.<init>(Command.java:50)
            20:33:18 at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
            20:33:18 at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:234)
            20:33:18 at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
            20:33:18 at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
            20:33:18 at java.lang.ref.Finalizer.access$100(Unknown Source)
            20:33:18 at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
            20:33:18 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:341)
            20:33:18 at hudson.remoting.ExportTable.get(ExportTable.java:324)
            20:33:18 at hudson.remoting.Channel.getExportedObject(Channel.java:604)
            20:33:18 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:311)
            20:33:18 at hudson.remoting.Request$2.run(Request.java:324)
            20:33:18 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
            20:33:18 at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
            20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            20:33:18 at java.lang.Thread.run(Thread.java:744)
            20:33:18 Caused by: Released at Wed Dec 24 20:28:20 MSK 2014
            20:33:18 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:125)
            20:33:18 at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:376)
            20:33:18 at hudson.remoting.Channel.unexport(Channel.java:612)
            20:33:18 at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38)
            20:33:18 at hudson.remoting.Channel$2.handle(Channel.java:483)
            20:33:18 at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61)
            20:33:18 at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:597)
            20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
            20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
            20:33:18 at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
            20:33:18 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
            20:33:18 ... 4 more
            20:33:18 Caused by: Command hudson.remoting.UnexportCommand@1539730b created at
            20:33:18 at hudson.remoting.Command.<init>(Command.java:67)
            20:33:18 at hudson.remoting.Command.<init>(Command.java:50)
            20:33:18 at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
            20:33:18 at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:234)
            20:33:18 at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
            20:33:18 at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
            20:33:18 at java.lang.ref.Finalizer.access$100(Unknown Source)
            20:33:18 at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
            </code>

            Statistically, one from 10 slaves break connection with such error.

            Show
            svchipiga Sergey Chipiga added a comment - - edited Hi! I have still this problem (Jenkins ver. 1.580.2): <code> FATAL: Remote call on win7-home-premium-x86-ru-1-f8e failed 20:33:18 java.io.IOException: Remote call on win7-home-premium-x86-ru-1-f8e failed 20:33:18 at hudson.remoting.Channel.call(Channel.java:760) 20:33:18 at hudson.Launcher$RemoteLauncher.kill(Launcher.java:941) 20:33:18 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:556) 20:33:18 at hudson.model.Run.execute(Run.java:1745) 20:33:18 at hudson.matrix.MatrixRun.run(MatrixRun.java:146) 20:33:18 at hudson.model.ResourceController.execute(ResourceController.java:89) 20:33:18 at hudson.model.Executor.run(Executor.java:240) 20:33:18 Caused by: java.lang.Error: Unable to load resource winp.dll 20:33:18 at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:381) 20:33:18 at java.lang.ClassLoader.getResource(Unknown Source) 20:33:18 at org.jvnet.winp.Native.load(Native.java:85) 20:33:18 at org.jvnet.winp.Native.<clinit>(Native.java:56) 20:33:18 at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:212) 20:33:18 at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:469) 20:33:18 at hudson.util.ProcessTree.get(ProcessTree.java:328) 20:33:18 at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:953) 20:33:18 at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:944) 20:33:18 at hudson.remoting.UserRequest.perform(UserRequest.java:121) 20:33:18 at hudson.remoting.UserRequest.perform(UserRequest.java:49) 20:33:18 at hudson.remoting.Request$2.run(Request.java:324) 20:33:18 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 20:33:18 at java.util.concurrent.FutureTask.run(Unknown Source) 20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 20:33:18 at hudson.remoting.Engine$1$1.run(Engine.java:63) 20:33:18 at java.lang.Thread.run(Unknown Source) 20:33:18 at ......remote call to yabrowser-win7-home-premium-x86-ru-1-f8e(Native Method) 20:33:18 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356) 20:33:18 at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) 20:33:18 at hudson.remoting.Channel.call(Channel.java:752) 20:33:18 ... 6 more 20:33:18 Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Invalid object ID 2 iota=14 20:33:18 at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:75) 20:33:18 at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) 20:33:18 at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:373) 20:33:18 at java.lang.ClassLoader.getResource(Unknown Source) 20:33:18 at org.jvnet.winp.Native.load(Native.java:85) 20:33:18 at org.jvnet.winp.Native.<clinit>(Native.java:56) 20:33:18 at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:212) 20:33:18 at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:469) 20:33:18 at hudson.util.ProcessTree.get(ProcessTree.java:328) 20:33:18 at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:953) 20:33:18 at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:944) 20:33:18 at hudson.remoting.UserRequest.perform(UserRequest.java:121) 20:33:18 at hudson.remoting.UserRequest.perform(UserRequest.java:49) 20:33:18 at hudson.remoting.Request$2.run(Request.java:324) 20:33:18 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 20:33:18 at java.util.concurrent.FutureTask.run(Unknown Source) 20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 20:33:18 at hudson.remoting.Engine$1$1.run(Engine.java:63) 20:33:18 at java.lang.Thread.run(Unknown Source) 20:33:18 Caused by: java.lang.IllegalStateException: Invalid object ID 2 iota=14 20:33:18 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:348) 20:33:18 at hudson.remoting.ExportTable.get(ExportTable.java:324) 20:33:18 at hudson.remoting.Channel.getExportedObject(Channel.java:604) 20:33:18 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:311) 20:33:18 at hudson.remoting.Request$2.run(Request.java:324) 20:33:18 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 20:33:18 at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) 20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 20:33:18 at java.lang.Thread.run(Thread.java:744) 20:33:18 at ......remote call to channel(Native Method) 20:33:18 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356) 20:33:18 at hudson.remoting.Request.call(Request.java:171) 20:33:18 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:176) 20:33:18 at com.sun.proxy.$Proxy5.writeJarTo(Unknown Source) 20:33:18 at hudson.remoting.FileSystemJarCache.retrieve(FileSystemJarCache.java:66) 20:33:18 at hudson.remoting.JarCacheSupport$1.run(JarCacheSupport.java:62) 20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) 20:33:18 at java.util.concurrent.FutureTask.run(Unknown Source) 20:33:18 at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:104) 20:33:18 ... 1 more 20:33:18 Caused by: java.lang.Exception: Object was recently deallocated 20:33:18 #2 (ref.0) : [hudson.remoting.JarLoader] 20:33:18 Created at Wed Dec 24 20:28:20 MSK 2014 20:33:18 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:93) 20:33:18 at hudson.remoting.ExportTable.export(ExportTable.java:299) 20:33:18 at hudson.remoting.Channel.internalExport(Channel.java:600) 20:33:18 at hudson.remoting.Channel.export(Channel.java:591) 20:33:18 at hudson.remoting.Channel.<init>(Channel.java:472) 20:33:18 at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:280) 20:33:18 at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:290) 20:33:18 at org.jenkinsci.remoting.nio.NioChannelBuilder.build(NioChannelBuilder.java:36) 20:33:18 at org.jenkinsci.remoting.nio.NioChannelBuilder.build(NioChannelBuilder.java:52) 20:33:18 at jenkins.slaves.JnlpSlaveAgentProtocol$Handler.jnlpConnect(JnlpSlaveAgentProtocol.java:120) 20:33:18 at jenkins.slaves.DefaultJnlpSlaveReceiver.handle(DefaultJnlpSlaveReceiver.java:63) 20:33:18 at jenkins.slaves.JnlpSlaveAgentProtocol2$Handler2.run(JnlpSlaveAgentProtocol2.java:57) 20:33:18 at jenkins.slaves.JnlpSlaveAgentProtocol2.handle(JnlpSlaveAgentProtocol2.java:31) 20:33:18 at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:156) 20:33:18 Released at Wed Dec 24 20:28:20 MSK 2014 20:33:18 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:125) 20:33:18 at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:376) 20:33:18 at hudson.remoting.Channel.unexport(Channel.java:612) 20:33:18 at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) 20:33:18 at hudson.remoting.Channel$2.handle(Channel.java:483) 20:33:18 at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61) 20:33:18 at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:597) 20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 20:33:18 at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) 20:33:18 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) 20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 20:33:18 at java.lang.Thread.run(Thread.java:744) 20:33:18 Caused by: Command hudson.remoting.UnexportCommand@1787906e created at 20:33:18 at hudson.remoting.Command.<init>(Command.java:67) 20:33:18 at hudson.remoting.Command.<init>(Command.java:50) 20:33:18 at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) 20:33:18 at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:234) 20:33:18 at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) 20:33:18 at java.lang.ref.Finalizer.runFinalizer(Unknown Source) 20:33:18 at java.lang.ref.Finalizer.access$100(Unknown Source) 20:33:18 at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) 20:33:18 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:341) 20:33:18 at hudson.remoting.ExportTable.get(ExportTable.java:324) 20:33:18 at hudson.remoting.Channel.getExportedObject(Channel.java:604) 20:33:18 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:311) 20:33:18 at hudson.remoting.Request$2.run(Request.java:324) 20:33:18 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 20:33:18 at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) 20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 20:33:18 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 20:33:18 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 20:33:18 at java.lang.Thread.run(Thread.java:744) 20:33:18 Caused by: Released at Wed Dec 24 20:28:20 MSK 2014 20:33:18 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:125) 20:33:18 at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:376) 20:33:18 at hudson.remoting.Channel.unexport(Channel.java:612) 20:33:18 at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) 20:33:18 at hudson.remoting.Channel$2.handle(Channel.java:483) 20:33:18 at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61) 20:33:18 at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:597) 20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 20:33:18 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 20:33:18 at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) 20:33:18 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) 20:33:18 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 20:33:18 ... 4 more 20:33:18 Caused by: Command hudson.remoting.UnexportCommand@1539730b created at 20:33:18 at hudson.remoting.Command.<init>(Command.java:67) 20:33:18 at hudson.remoting.Command.<init>(Command.java:50) 20:33:18 at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) 20:33:18 at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:234) 20:33:18 at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) 20:33:18 at java.lang.ref.Finalizer.runFinalizer(Unknown Source) 20:33:18 at java.lang.ref.Finalizer.access$100(Unknown Source) 20:33:18 at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) </code> Statistically, one from 10 slaves break connection with such error.
            Hide
            jglick Jesse Glick added a comment -

            Sergey Chipiga your problem (IllegalStateException: Invalid object ID 2 iota=14) looks unrelated and should be filed separately.

            Show
            jglick Jesse Glick added a comment - Sergey Chipiga your problem ( IllegalStateException: Invalid object ID 2 iota=14 ) looks unrelated and should be filed separately.
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 150902 ] JNJira + In-Review [ 193728 ]
            olivergondza Oliver Gondža made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            olivergondza Oliver Gondža made changes -
            Link This issue is related to JENKINS-36991 [ JENKINS-36991 ]

              People

              • Assignee:
                kohsuke Kohsuke Kawaguchi
                Reporter:
                ycollet yann collette
              • Votes:
                5 Vote for this issue
                Watchers:
                17 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: