Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61341

FileNotFoundException if withCredentials([sshUserPrivateKey]) is called twice

    Details

    • Similar Issues:

      Description

      Hi.

      I am currently working on a Jenkins pipeline and trying to move all credential handling in the domain of the jenkins instance.

      To be able to execute an ssh command on another machine we use private key authentication. I tried to accomplish this by combining credentials-binding and ssh-steps but ran into a problem I couldn't find a solution to.

      I basicly do this:
       

      node {
          def remote = [:]
          remote.name = "integration_server"
          remote.host = integration_server
          remote.allowAnyHosts = true
      
          withCredentials([sshUserPrivateKey(credentialsId: 'myCredentialId', keyFileVariable: 'identity', passphraseVariable: '', usernameVariable: 'userName')]) {
              remote.user = userName
              remote.identityFile = identity
              sshCommand remote: remote, command: "do stuff"
          }
      
      ...
      
          withCredentials([sshUserPrivateKey(credentialsId: 'myCredentialId', keyFileVariable: 'identity', passphraseVariable: '', usernameVariable: 'userName')]) {
              remote.user = userName
              remote.identityFile = identity
              sshCommand remote: remote, command: "do some other stuff"
          }
      }
      

      The first block runs fine. The second block fails with a FileNotFoundException looking for a temporary "secretFile".
      My guess is, that credential-binding reuses the reference to the temporary file from the first block during execution of the second block while simultaniously deleting it after the first block ends.

      I would like to be able to define multiple indipendent withCredential steps to omit wrapping the whole script in one large block.

        Attachments

          Activity

          Hide
          julian_alarcon Julian Alarcon added a comment - - edited

          I updated my Jenkins to 2.330, same error. I'm out of ideas.

          This is my list of installed plugins:

          ace-editor	1.1	true
          ansicolor	0.7.0	true
          antisamy-markup-formatter	2.0	true
          apache-httpcomponents-client-4-api	4.5.10-2.0	true
          authentication-tokens	1.4	true
          bouncycastle-api	2.18	true
          branch-api	2.5.6	true
          build-user-vars-plugin	1.5	true
          cloudbees-folder	6.14	true
          command-launcher	1.4	true
          credentials	2.3.10	true
          credentials-binding	1.23	true
          display-url-api	2.3.2	true
          docker-commons	1.16	true
          docker-workflow	1.23	true
          durable-task	1.34	true
          email-ext	2.69	true
          extended-read-permission	3.2	true
          git	4.3.0	true
          git-client	3.3.0	true
          git-server	1.9	true
          github	1.30.0	true
          github-api	1.114.3	true
          handlebars	1.1.1	true
          htmlpublisher	1.23	true
          jackson2-api	2.11.0	true
          jaxb	2.3.0.1	true
          jdk-tool	1.4	true
          jquery-detached	1.2.1	true
          jsch	0.1.55.2	true
          junit	1.29	true
          lockable-resources	2.8	true
          mailer	1.32	true
          matrix-auth	2.6.1	true
          matrix-project	1.16	true
          momentjs	1.1.1	true
          okhttp-api	3.14.9	true
          pipeline-build-step	2.12	true
          pipeline-config-history	1.6	true
          pipeline-graph-analysis	1.10	true
          pipeline-input-step	2.11	true
          pipeline-milestone-step	1.3.1	true
          pipeline-model-api	1.7.0	true
          pipeline-model-declarative-agent	1.1.1	true
          pipeline-model-definition	1.7.0	true
          pipeline-model-extensions	1.7.0	true
          pipeline-rest-api	2.13	true
          pipeline-stage-step	2.5	true
          pipeline-stage-tags-metadata	1.7.0	true
          pipeline-stage-view	2.13	true
          plain-credentials	1.7	true
          publish-over	0.22	true
          publish-over-ssh	1.20.1	true
          resource-disposer	0.14	true
          robot	2.1.1	true
          role-strategy	3.0	true
          saml	1.1.6	true
          scm-api	2.6.3	true
          script-security	1.73	true
          slack	2.40	true
          ssh-credentials	1.18.1	true
          ssh-steps	2.0.0	true
          structs	1.20	true
          timestamper	1.11.3	true
          token-macro	2.12	true
          trilead-api	1.0.8	true
          workflow-aggregator	2.6	true
          workflow-api	2.40	true
          workflow-basic-steps	2.20	true
          workflow-cps	2.80	true
          workflow-cps-global-lib	2.16	true
          workflow-durable-task-step	2.35	true
          workflow-job	2.39	true
          workflow-multibranch	2.21	true
          workflow-scm-step	2.11	true
          workflow-step-api	2.22	true
          workflow-support	3.5	true
          ws-cleanup	0.38	true
          Show
          julian_alarcon Julian Alarcon added a comment - - edited I updated my Jenkins to 2.330, same error. I'm out of ideas. This is my list of installed plugins: ace-editor 1.1 true ansicolor 0.7.0 true antisamy-markup-formatter 2.0 true apache-httpcomponents-client-4-api 4.5.10-2.0 true authentication-tokens 1.4 true bouncycastle-api 2.18 true branch-api 2.5.6 true build-user-vars-plugin 1.5 true cloudbees-folder 6.14 true command-launcher 1.4 true credentials 2.3.10 true credentials-binding 1.23 true display-url-api 2.3.2 true docker-commons 1.16 true docker-workflow 1.23 true durable-task 1.34 true email-ext 2.69 true extended-read-permission 3.2 true git 4.3.0 true git-client 3.3.0 true git-server 1.9 true github 1.30.0 true github-api 1.114.3 true handlebars 1.1.1 true htmlpublisher 1.23 true jackson2-api 2.11.0 true jaxb 2.3.0.1 true jdk-tool 1.4 true jquery-detached 1.2.1 true jsch 0.1.55.2 true junit 1.29 true lockable-resources 2.8 true mailer 1.32 true matrix-auth 2.6.1 true matrix-project 1.16 true momentjs 1.1.1 true okhttp-api 3.14.9 true pipeline-build-step 2.12 true pipeline-config-history 1.6 true pipeline-graph-analysis 1.10 true pipeline-input-step 2.11 true pipeline-milestone-step 1.3.1 true pipeline-model-api 1.7.0 true pipeline-model-declarative-agent 1.1.1 true pipeline-model-definition 1.7.0 true pipeline-model-extensions 1.7.0 true pipeline-rest-api 2.13 true pipeline-stage-step 2.5 true pipeline-stage-tags-metadata 1.7.0 true pipeline-stage-view 2.13 true plain-credentials 1.7 true publish-over 0.22 true publish-over-ssh 1.20.1 true resource-disposer 0.14 true robot 2.1.1 true role-strategy 3.0 true saml 1.1.6 true scm-api 2.6.3 true script-security 1.73 true slack 2.40 true ssh-credentials 1.18.1 true ssh-steps 2.0.0 true structs 1.20 true timestamper 1.11.3 true token-macro 2.12 true trilead-api 1.0.8 true workflow-aggregator 2.6 true workflow-api 2.40 true workflow-basic-steps 2.20 true workflow-cps 2.80 true workflow-cps-global-lib 2.16 true workflow-durable-task-step 2.35 true workflow-job 2.39 true workflow-multibranch 2.21 true workflow-scm-step 2.11 true workflow-step-api 2.22 true workflow-support 3.5 true ws-cleanup 0.38 true
          Hide
          antoniocfranco Antonio Franco added a comment -

          I am getting the same error. I am using Jenkins 2.235.3. Running Jenkins under windows 10 and agent under Centos 7.6.

          Running my jobs all under the Centos agent, here is the log.

          java.io.FileNotFoundException: C:/Jenkins/workspace/pipeline_test@tmp/secretFiles/a338eb6c-9e3e-4c8d-8adc-5d30f3be374a/ssh-key-keyFileName (No such file or directory)
          	at java.io.FileInputStream.open0(Native Method)
          	at java.io.FileInputStream.open(FileInputStream.java:195)
          	at java.io.FileInputStream.<init>(FileInputStream.java:138)
          	at java.io.FileInputStream.<init>(FileInputStream.java:93)
          	at com.jcraft.jsch.Util.fromFile(Util.java:508)
          	at com.jcraft.jsch.KeyPair.load(KeyPair.java:540)
          Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to Test_agent
          		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
          		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
          		at hudson.remoting.Channel.call(Channel.java:998)
          		at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution.run(CommandStep.java:72)
          		at org.jenkinsci.plugins.sshsteps.util.SSHStepExecution.lambda$start$0(SSHStepExecution.java:84)
          		at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
          		at java.util.concurrent.FutureTask.run(Unknown Source)
          		at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          		at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          		at java.lang.Thread.run(Unknown Source)
          Caused: com.jcraft.jsch.JSchException
          	at com.jcraft.jsch.KeyPair.load(KeyPair.java:543)
          	at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40)
          	at com.jcraft.jsch.JSch.addIdentity(JSch.java:406)
          	at com.jcraft.jsch.JSch.addIdentity(JSch.java:387)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:192)
          	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
          	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
          	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
          	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
          	at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper.configureUserAuthentication(UserAuthentication.groovy:36)
          	at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper$configureUserAuthentication$2.call(Unknown Source)
          	at org.hidetake.groovy.ssh.connection.ConnectionManager.configureUserAuthentication(ConnectionManager.groovy)
          	at org.hidetake.groovy.ssh.connection.UserAuthentication$configureUserAuthentication$1.callCurrent(Unknown Source)
          	at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:104)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
          	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
          	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384)
          	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
          	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
          	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190)
          	at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy:85)
          	at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
          	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
          	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
          	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
          	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
          	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
          	at org.hidetake.groovy.ssh.util.Utility.retry(Utility.groovy:52)
          	at org.hidetake.groovy.ssh.util.Utility$retry.callStatic(Unknown Source)
          	at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:83)
          	at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
          	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
          	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
          	at org.hidetake.groovy.ssh.connection.ConnectionManager.connect(ConnectionManager.groovy:59)
          	at org.hidetake.groovy.ssh.connection.ConnectionManager$connect$0.call(Unknown Source)
          	at org.hidetake.groovy.ssh.session.SessionTask.wetRun(SessionTask.groovy:61)
          	at org.hidetake.groovy.ssh.session.SessionTask.call(SessionTask.groovy:48)
          	at java_util_concurrent_Callable$call$0.call(Unknown Source)
          	at org.hidetake.groovy.ssh.core.Service.run(Service.groovy:81)
          	at org.hidetake.groovy.ssh.core.Service$run$1.call(Unknown Source)
          	at org.jenkinsci.plugins.sshsteps.SSHService.executeCommand(SSHService.groovy:177)
          	at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution$CommandCallable.execute(CommandStep.java:84)
          	at org.jenkinsci.plugins.sshsteps.util.SSHMasterToSlaveCallable.call(SSHMasterToSlaveCallable.java:32)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
          	at hudson.remoting.Request$2.run(Request.java:369)
          	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          Finished: FAILURE
          

          Let me know what can I do to help diagnose this issue. It is 100% reproducible.

           

          Show
          antoniocfranco Antonio Franco added a comment - I am getting the same error. I am using Jenkins 2.235.3. Running Jenkins under windows 10 and agent under Centos 7.6. Running my jobs all under the Centos agent, here is the log. java.io.FileNotFoundException: C:/Jenkins/workspace/pipeline_test@tmp/secretFiles/a338eb6c-9e3e-4c8d-8adc-5d30f3be374a/ssh-key-keyFileName (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at java.io.FileInputStream.<init>(FileInputStream.java:93) at com.jcraft.jsch.Util.fromFile(Util.java:508) at com.jcraft.jsch.KeyPair.load(KeyPair.java:540) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Test_agent at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:998) at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution.run(CommandStep.java:72) at org.jenkinsci.plugins.sshsteps.util.SSHStepExecution.lambda$start$0(SSHStepExecution.java:84) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang. Thread .run(Unknown Source) Caused: com.jcraft.jsch.JSchException at com.jcraft.jsch.KeyPair.load(KeyPair.java:543) at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40) at com.jcraft.jsch.JSch.addIdentity(JSch.java:406) at com.jcraft.jsch.JSch.addIdentity(JSch.java:387) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:192) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper.configureUserAuthentication(UserAuthentication.groovy:36) at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper$configureUserAuthentication$2.call(Unknown Source) at org.hidetake.groovy.ssh.connection.ConnectionManager.configureUserAuthentication(ConnectionManager.groovy) at org.hidetake.groovy.ssh.connection.UserAuthentication$configureUserAuthentication$1.callCurrent(Unknown Source) at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190) at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy:85) at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117) at org.hidetake.groovy.ssh.util.Utility.retry(Utility.groovy:52) at org.hidetake.groovy.ssh.util.Utility$retry.callStatic(Unknown Source) at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:83) at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166) at org.hidetake.groovy.ssh.connection.ConnectionManager.connect(ConnectionManager.groovy:59) at org.hidetake.groovy.ssh.connection.ConnectionManager$connect$0.call(Unknown Source) at org.hidetake.groovy.ssh.session.SessionTask.wetRun(SessionTask.groovy:61) at org.hidetake.groovy.ssh.session.SessionTask.call(SessionTask.groovy:48) at java_util_concurrent_Callable$call$0.call(Unknown Source) at org.hidetake.groovy.ssh.core.Service.run(Service.groovy:81) at org.hidetake.groovy.ssh.core.Service$run$1.call(Unknown Source) at org.jenkinsci.plugins.sshsteps.SSHService.executeCommand(SSHService.groovy:177) at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution$CommandCallable.execute(CommandStep.java:84) at org.jenkinsci.plugins.sshsteps.util.SSHMasterToSlaveCallable.call(SSHMasterToSlaveCallable.java:32) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE Let me know what can I do to help diagnose this issue. It is 100% reproducible.  
          Hide
          julian_alarcon Julian Alarcon added a comment - - edited

          Seems that I found the reason of the error. Every time that a new block withCredentials is invoked, a new temporal folder is created. But when a sshCommand or any other option from ssh-steps-plugin is used it locks the original temporal directory/file to be used, but as after every withCredentials run that file is deleted, it will not find it, ssh-steps-plugin should use the new temporal directory created and not be fixed with the first temporal path from the first withCredentials block.
          I tested this code and also check the files, and checking the error that's what is happening.

          Jenkinsfile code:

          def fileName
          def remote = [:]
          remote.name = "integration_server"
          remote.user = "ubuntu"
          remote.host = "10.20.156.167"
          remote.allowAnyHosts = truepipeline {
              agent any
              stages {
                  stage('APP_01') {
                      steps {
                          withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) {
                              script {
                                  fileName = identityFileName
                                  def data = readFile(file: fileName)
                                  println data // <--- This print the secret ssh key with no issues
                                  remote.identityFile = identityFileName
                                  sshCommand remote: remote, command: "echo first-app01-ok"
                              }
                              echo 'after first withCredentials block'
                              sh 'sleep 30'
                          }
                          withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) {
                              script {
                                  fileName = identityFileName
                                  def data = readFile(file: fileName)
                                  println data // <--- This print the secret ssh key again with no issues
                                  remote.identityFile = identityFileName
                                  //sshCommand remote: remote, command: "echo this-fails"
                              }
                              echo 'after second withCredentials block'
                              sh 'sleep 30'
                          }
                      }
                  }
                  stage('APP_01_AGAIN') { //this is never run, but if you comment the second withCredentials above it will run an fail
                      steps {
                          withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) {
                              script {
                                  fileName = identityFileName
                                  def data = readFile(file: fileName)
                                  println data
                                  remote.identityFile = identityFileName
                                  sshCommand remote: remote, command: "echo this-one-fails-too"
                              }
                          }
                      }
                  }
              }
          }
          

          Jenkins Error (Look, it's trying to find the 6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName but it doesn't exists anymore):

          16:49:37  [Pipeline] sshCommand
          16:49:37  Executing command on integration_server[10.20.156.167]: echo this-one-fails-too sudo: false
          16:49:37  [Pipeline] }
          16:49:37  [Pipeline] // script
          16:49:37  [Pipeline] }
          16:49:37  [Pipeline] // withCredentials
          16:49:37  [Pipeline] }
          16:49:37  [Pipeline] // stage
          16:49:37  [Pipeline] }
          16:49:37  [Pipeline] // node
          16:49:37  [Pipeline] End of Pipeline
          16:49:37  java.io.FileNotFoundException: /var/lib/jenkins/workspace/test-bug-JENKINS-61341@tmp/secretFiles/6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName (No such file or directory)
          16:49:37  	at java.base/java.io.FileInputStream.open0(Native Method)
          16:49:37  	at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
          16:49:37  	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
          16:49:37  	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)
          16:49:37  	at com.jcraft.jsch.Util.fromFile(Util.java:508)
          16:49:37  	at com.jcraft.jsch.KeyPair.load(KeyPair.java:540)
          16:49:37  Caused: com.jcraft.jsch.JSchException
          16:49:37  	at com.jcraft.jsch.KeyPair.load(KeyPair.java:543)
          16:49:37  	at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40)
          16:49:37  	at com.jcraft.jsch.JSch.addIdentity(JSch.java:406)
          16:49:37  	at com.jcraft.jsch.JSch.addIdentity(JSch.java:387)
          16:49:37  	at jdk.internal.reflect.GeneratedMethodAccessor4870.invoke(Unknown Source)
          16:49:37  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          

          Checking the real files in the server, the files with the key exists but every time than a withCredentials block is used it deletes the files and recreates them, with another tmp name. 

          jenkins@ip-10-20-150-196:~/workspace/test-bug-JENKINS-61341@tmp$ cat secretFiles/6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName
          -----BEGIN RSA PRIVATE KEY-----
          MY_SSH_KEY
          .
          .
          .
          -----END RSA PRIVATE KEY-----
          jenkins@ip-10-20-150-196:~/workspace/test-bug-JENKINS-61341@tmp$ cat secretFiles/4b169093-f5a3-4316-b7e2-e940149b361f/ssh-key-identityFileName
          -----BEGIN RSA PRIVATE KEY-----
          MY_SSH_KEY
          .
          .
          .
          -----END RSA PRIVATE KEY-----
          

          Can you take a look Naresh Rayapati?

          Show
          julian_alarcon Julian Alarcon added a comment - - edited Seems that I found the reason of the error. Every time that a new block withCredentials is invoked, a new temporal folder is created. But when a sshCommand or any other option from ssh-steps-plugin is used it locks the original temporal directory/file to be used, but as after every withCredentials run that file is deleted, it will not find it, ssh-steps-plugin should use the new temporal directory created and not be fixed with the first temporal path from the first withCredentials block. I tested this code and also check the files, and checking the error that's what is happening. Jenkinsfile code: def fileName def remote = [:] remote.name = "integration_server" remote.user = "ubuntu" remote.host = "10.20.156.167" remote.allowAnyHosts = truepipeline { agent any stages { stage( 'APP_01' ) { steps { withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key' , keyFileVariable: 'identityFileName' , passphraseVariable: 'password' , usernameVariable: 'username' )]) { script { fileName = identityFileName def data = readFile(file: fileName) println data // <--- This print the secret ssh key with no issues remote.identityFile = identityFileName sshCommand remote: remote, command: "echo first-app01-ok" } echo 'after first withCredentials block' sh 'sleep 30' } withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key' , keyFileVariable: 'identityFileName' , passphraseVariable: 'password' , usernameVariable: 'username' )]) { script { fileName = identityFileName def data = readFile(file: fileName) println data // <--- This print the secret ssh key again with no issues remote.identityFile = identityFileName //sshCommand remote: remote, command: "echo this -fails" } echo 'after second withCredentials block' sh 'sleep 30' } } } stage( 'APP_01_AGAIN' ) { // this is never run, but if you comment the second withCredentials above it will run an fail steps { withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key' , keyFileVariable: 'identityFileName' , passphraseVariable: 'password' , usernameVariable: 'username' )]) { script { fileName = identityFileName def data = readFile(file: fileName) println data remote.identityFile = identityFileName sshCommand remote: remote, command: "echo this -one-fails-too" } } } } } } Jenkins Error (Look, it's trying to find the 6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName but it doesn't exists anymore): 16:49:37 [Pipeline] sshCommand 16:49:37 Executing command on integration_server[10.20.156.167]: echo this -one-fails-too sudo: false 16:49:37 [Pipeline] } 16:49:37 [Pipeline] // script 16:49:37 [Pipeline] } 16:49:37 [Pipeline] // withCredentials 16:49:37 [Pipeline] } 16:49:37 [Pipeline] // stage 16:49:37 [Pipeline] } 16:49:37 [Pipeline] // node 16:49:37 [Pipeline] End of Pipeline 16:49:37 java.io.FileNotFoundException: / var /lib/jenkins/workspace/test-bug-JENKINS-61341@tmp/secretFiles/6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName (No such file or directory) 16:49:37 at java.base/java.io.FileInputStream.open0(Native Method) 16:49:37 at java.base/java.io.FileInputStream.open(FileInputStream.java:219) 16:49:37 at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) 16:49:37 at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112) 16:49:37 at com.jcraft.jsch.Util.fromFile(Util.java:508) 16:49:37 at com.jcraft.jsch.KeyPair.load(KeyPair.java:540) 16:49:37 Caused: com.jcraft.jsch.JSchException 16:49:37 at com.jcraft.jsch.KeyPair.load(KeyPair.java:543) 16:49:37 at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40) 16:49:37 at com.jcraft.jsch.JSch.addIdentity(JSch.java:406) 16:49:37 at com.jcraft.jsch.JSch.addIdentity(JSch.java:387) 16:49:37 at jdk.internal.reflect.GeneratedMethodAccessor4870.invoke(Unknown Source) 16:49:37 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Checking the real files in the server, the files with the key exists but every time than a withCredentials block is used it deletes the files and recreates them, with another tmp name.  jenkins@ip-10-20-150-196:~/workspace/test-bug-JENKINS-61341@tmp$ cat secretFiles/6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName -----BEGIN RSA PRIVATE KEY----- MY_SSH_KEY . . . -----END RSA PRIVATE KEY----- jenkins@ip-10-20-150-196:~/workspace/test-bug-JENKINS-61341@tmp$ cat secretFiles/4b169093-f5a3-4316-b7e2-e940149b361f/ssh-key-identityFileName -----BEGIN RSA PRIVATE KEY----- MY_SSH_KEY . . . -----END RSA PRIVATE KEY----- Can you take a look Naresh Rayapati ?
          Hide
          antoniocfranco Antonio Franco added a comment -

          That looks like a good find Julian Alarcon, please let me know if I can help in anything. I am using a scripted pipeline.

          Show
          antoniocfranco Antonio Franco added a comment - That looks like a good find Julian Alarcon , please let me know if I can help in anything. I am using a scripted pipeline.
          Hide
          julian_alarcon Julian Alarcon added a comment - - edited

          Hi, as I found the reason (not changing the value of the map value remote.identityFile), I was able to repeat the execution using two workarounds, you can find my code here:

           

          server_01 = [:]
          server_01.name = "integration_server"
          server_01.host = "10.20.11.1"
          
          def remotename = "integration_server"
          def remotehost = "10.20.11.1"
          
          pipeline {
              agent any
              stages {
                  stage('APP_01') {
                      steps {
                          withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) {
                              script {
                                  def remote = [ name: server_01.name, host: server_01.host, user: username, identityFile: identityFileName, allowAnyHosts: true]
                                  sshCommand remote: remote, command: "echo First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block"
                              }
                          }
                          withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) {
                              script {
                                  sshCommand remote: [ name: remotename, host: remotehost, allowAnyHosts: true, user: username, identityFile: identityFileName ], command: "echo works too, but seems ugly, as it is too large"
                              }
                          }
                          withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) {
                              script {
                                  def remote = [ name: server_01.name, host: server_01.host, user: username, identityFile: identityFileName, allowAnyHosts: true]
                                  sshCommand remote: remote, command: "echo First workaround. It keeps working"
                              }
                          }
                          withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) {
                              script {
                                  sshCommand remote: [ name: remotename, host: remotehost, allowAnyHosts: true, user: username, identityFile: identityFileName ], command: "echo Second workaround. It keeps working too"
                              }
                          }
                      }
                  }
              }
          }
          

           

          Job Log:

          Running in Durability level: MAX_SURVIVABILITY
           [Pipeline] Start of Pipeline
           [Pipeline] node
           Running on Jenkins in /var/lib/jenkins/workspace/test-bug-JENKINS-61341
           [Pipeline] {
           [Pipeline] stage
           [Pipeline] { (APP_01)
           [Pipeline] withCredentials
           Masking supported pattern matches of $identityFileName or $password or $username
           [Pipeline] {
           [Pipeline] script
           [Pipeline] {
           [Pipeline] sshCommand
           Executing command on integration_server[10.20.11.1]: echo First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block sudo: false
           First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block
           [Pipeline] }
           [Pipeline] // script
           [Pipeline] }
           [Pipeline] // withCredentials
           [Pipeline] withCredentials
           Masking supported pattern matches of $identityFileName or $password or $username
           [Pipeline] {
           [Pipeline] script
           [Pipeline] {
           [Pipeline] sshCommand
           Executing command on integration_server[10.20.156.167]: echo works too, but seems ugly, as it is too large sudo: false
           works too, but seems ugly, as it is too large
           [Pipeline] }
           [Pipeline] // script
           [Pipeline] }
           [Pipeline] // withCredentials
           [Pipeline] withCredentials
           Masking supported pattern matches of $identityFileName or $password or $username
           [Pipeline] {
           [Pipeline] script
           [Pipeline] {
           [Pipeline] sshCommand
           Executing command on integration_server[10.20.11.1]: echo First workaround. It keeps working sudo: false
           First workaround. It keeps working
           [Pipeline] }
           [Pipeline] // script
           [Pipeline] }
           [Pipeline] // withCredentials
           [Pipeline] withCredentials
           Masking supported pattern matches of $identityFileName or $password or $username
           [Pipeline] {
           [Pipeline] script
           [Pipeline] {
           [Pipeline] sshCommand
           Executing command on integration_server[10.20.156.167]: echo Second workaround. It keeps working too sudo: false Second workaround. It keeps working too
           [Pipeline] }
           [Pipeline] // script
           [Pipeline] }
           [Pipeline] // withCredentials
           [Pipeline] }
           [Pipeline] // stage
           [Pipeline] }
           [Pipeline] // node
           [Pipeline] End of Pipeline
           Finished: SUCCESS
          
          Show
          julian_alarcon Julian Alarcon added a comment - - edited Hi, as I found the reason (not changing the value of the map value remote.identityFile), I was able to repeat the execution using two workarounds , you can find my code here:   server_01 = [:] server_01.name = "integration_server" server_01.host = "10.20.11.1" def remotename = "integration_server" def remotehost = "10.20.11.1" pipeline { agent any stages { stage( 'APP_01' ) { steps { withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key' , keyFileVariable: 'identityFileName' , passphraseVariable: 'password' , usernameVariable: 'username' )]) { script { def remote = [ name: server_01.name, host: server_01.host, user: username, identityFile: identityFileName, allowAnyHosts: true ] sshCommand remote: remote, command: "echo First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block" } } withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key' , keyFileVariable: 'identityFileName' , passphraseVariable: 'password' , usernameVariable: 'username' )]) { script { sshCommand remote: [ name: remotename, host: remotehost, allowAnyHosts: true , user: username, identityFile: identityFileName ], command: "echo works too, but seems ugly, as it is too large" } } withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key' , keyFileVariable: 'identityFileName' , passphraseVariable: 'password' , usernameVariable: 'username' )]) { script { def remote = [ name: server_01.name, host: server_01.host, user: username, identityFile: identityFileName, allowAnyHosts: true ] sshCommand remote: remote, command: "echo First workaround. It keeps working" } } withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key' , keyFileVariable: 'identityFileName' , passphraseVariable: 'password' , usernameVariable: 'username' )]) { script { sshCommand remote: [ name: remotename, host: remotehost, allowAnyHosts: true , user: username, identityFile: identityFileName ], command: "echo Second workaround. It keeps working too" } } } } } }   Job Log: Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in / var /lib/jenkins/workspace/test-bug-JENKINS-61341 [Pipeline] { [Pipeline] stage [Pipeline] { (APP_01) [Pipeline] withCredentials Masking supported pattern matches of $identityFileName or $password or $username [Pipeline] { [Pipeline] script [Pipeline] { [Pipeline] sshCommand Executing command on integration_server[10.20.11.1]: echo First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block sudo: false First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withCredentials [Pipeline] withCredentials Masking supported pattern matches of $identityFileName or $password or $username [Pipeline] { [Pipeline] script [Pipeline] { [Pipeline] sshCommand Executing command on integration_server[10.20.156.167]: echo works too, but seems ugly, as it is too large sudo: false works too, but seems ugly, as it is too large [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withCredentials [Pipeline] withCredentials Masking supported pattern matches of $identityFileName or $password or $username [Pipeline] { [Pipeline] script [Pipeline] { [Pipeline] sshCommand Executing command on integration_server[10.20.11.1]: echo First workaround. It keeps working sudo: false First workaround. It keeps working [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withCredentials [Pipeline] withCredentials Masking supported pattern matches of $identityFileName or $password or $username [Pipeline] { [Pipeline] script [Pipeline] { [Pipeline] sshCommand Executing command on integration_server[10.20.156.167]: echo Second workaround. It keeps working too sudo: false Second workaround. It keeps working too [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS

            People

            • Assignee:
              nrayapati Naresh Rayapati
              Reporter:
              theck Timo Heck
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: