Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61844

Caused: java.io.IOException: remote file operation failed preparing for git checkout

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: ec2-plugin
    • Labels:
    • Environment:
      Jenkins Version 2.107.3
      Git Version: 3.9.1
      Git Client: 2.7.2
      Amazon EC2: 1.39
    • Similar Issues:

      Description

      Hi,

      So we use Amazon EC2 plugin which created Dynamic slaves.  

      I will try to describe the issue:

      • Slave request was made by the server to the EC2 plugin
      • EC2 plugin brought up a slave and attached for use
      • A build picked up the slave and failed with the error below
      • The slave got assigned to another build and that build also failed
      • The slave doesn't get decommissioned when the below issue happens rather it keeps disrupting all the Jenkins jobs it gets assigned too. 

      Workaround: 

      • There is a manual workaround to restart the slave but I am looking for an automated way to detect it and either delete the slave or automatically restart the slave. 
      • My point also is that this issue has happened not once but many times with different people so what can be done to fix it?

       

       19:35:30 [WS-CLEANUP] Deleting project workspace...
       19:35:30 [WS-CLEANUP] Done
       19:35:30 Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to slave (i-123456)
       19:35:30 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1737)
       19:35:30 at hudson.remoting.UserResponse.retrieve(UserRequest.java:313)
       19:35:30 at hudson.remoting.Channel.call(Channel.java:952)
       19:35:30 at hudson.FilePath.act(FilePath.java:1009)
       19:35:30 at hudson.FilePath.act(FilePath.java:998)
       19:35:30 at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:137)
       19:35:30 at hudson.plugins.git.GitSCM.createClient(GitSCM.java:821)
       19:35:30 at hudson.plugins.git.GitSCM.createClient(GitSCM.java:812)
       19:35:30 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1180)
       19:35:30 at hudson.scm.SCM.checkout(SCM.java:504)
       19:35:30 at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
       19:35:30 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
       19:35:30 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
       19:35:30 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
       19:35:30 at hudson.model.Run.execute(Run.java:1727)
       19:35:30 at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
       19:35:30 at hudson.model.ResourceController.execute(ResourceController.java:97)
       19:35:30 at hudson.model.Executor.run(Executor.java:429)
       19:35:30 java.lang.NoClassDefFoundError: Could not initialize class org.jenkinsci.plugins.gitclient.GitClient
       19:35:30 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.<init>(CliGitAPIImpl.java:253)
       19:35:30 at hudson.plugins.git.GitAPI.<init>(GitAPI.java:77)
       19:35:30 at org.jenkinsci.plugins.gitclient.Git$1.invoke(Git.java:134)
       19:35:30 at org.jenkinsci.plugins.gitclient.Git$1.invoke(Git.java:120)
       19:35:30 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2827)
       19:35:30 at hudson.remoting.UserRequest.perform(UserRequest.java:210)
       19:35:30 at hudson.remoting.UserRequest.perform(UserRequest.java:53)
       19:35:30 at hudson.remoting.Request$2.run(Request.java:364)
       19:35:30 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
       19:35:30 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       19:35:30 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       19:35:30 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       19:35:30 at java.lang.Thread.run(Thread.java:748)
       19:35:30 Caused: java.io.IOException: Remote call to slave (i-123456) failed
       19:35:30 at hudson.remoting.Channel.call(Channel.java:958)
       19:35:30 at hudson.FilePath.act(FilePath.java:1009)
       19:35:30 Caused: java.io.IOException: remote file operation failed: $HOME/workspace at hudson.remoting.Channel@678f:slave (i-123456)
       19:35:30 at hudson.FilePath.act(FilePath.java:1016)
       19:35:30 at hudson.FilePath.act(FilePath.java:998)
       19:35:30 at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:137)
       19:35:30 at hudson.plugins.git.GitSCM.createClient(GitSCM.java:821)
       19:35:30 at hudson.plugins.git.GitSCM.createClient(GitSCM.java:812)
       19:35:30 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1180)
       19:35:30 at hudson.scm.SCM.checkout(SCM.java:504)
       19:35:30 at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
       19:35:30 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
       19:35:30 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
       19:35:30 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
       19:35:30 at hudson.model.Run.execute(Run.java:1727)
       19:35:30 at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
       19:35:30 at hudson.model.ResourceController.execute(ResourceController.java:97)
       19:35:30 at hudson.model.Executor.run(Executor.java:429)
       19:35:30 Retrying after 10 seconds 

        Attachments

          Activity

          Hide
          markewaite Mark Waite added a comment - - edited

          I assume that root exception is the Caused: java.io.IOException: remote file operation failed: $HOME/workspace at hudson.remoting.Channel@678f:slave (i-123456) and that the EC2 plugin needs to disable or delete that agent when it detects that it is unable to perform its work.

          I've removed git-plugin from the assignment because I would expect the same failure to affect the Mercurial plugin, the Subversion plugin, and any other SCM plugin. Likewise, if there were an operation configured in the job to occur before the checkout, that operation would likely also fail in the same manner.

          I doubt that it will affect this behavior, but you might also consider a newer version of Jenkins. Jenkins 2.107.3 was released two years ago. The current long term support release is 2.222.1. Newer versions of the git plugin (4.0.0 and later) do not support that Jenkins version.

          Show
          markewaite Mark Waite added a comment - - edited I assume that root exception is the Caused: java.io.IOException: remote file operation failed: $HOME/workspace at hudson.remoting.Channel@678f:slave (i-123456) and that the EC2 plugin needs to disable or delete that agent when it detects that it is unable to perform its work. I've removed git-plugin from the assignment because I would expect the same failure to affect the Mercurial plugin, the Subversion plugin, and any other SCM plugin. Likewise, if there were an operation configured in the job to occur before the checkout, that operation would likely also fail in the same manner. I doubt that it will affect this behavior, but you might also consider a newer version of Jenkins. Jenkins 2.107.3 was released two years ago. The current long term support release is 2.222.1. Newer versions of the git plugin (4.0.0 and later) do not support that Jenkins version.

            People

            • Assignee:
              Unassigned
              Reporter:
              cliffhanger Jason Hunt
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: