Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56846

filesystem scm load stuck in jenkinfile-runner

    Details

    • Similar Issues:

      Description

      I need to run jenkinsfile via the jenkinsdile-runner.

      My jenkinsfile contains load shared library via filesystem scm plugin.

       

      pipeline {    
         agent any    
         stages {
             stage('Load Library') {
                steps {
                  library identifier: 'my@TestCase',
                             retriever: legacySCM(
                             filesystem(clearWorkspace: false, copyHidden: false, path: "/workspace/my"))
                  }
             }
           stage('Test') {
              steps {
                  my_method_to_run a: "aaaa"
              }
           }
         }
       }
      
      

      Via docker execution the load shared lib is stuck.

       

      Loading library my@TestCase
      FSSCM.checkout /workspace/my_shared_library to /tmp/jenkinsfileRunner.tmp/jfr5221246900865917223.run/workspace/job@libs/my
      

        Attachments

          Activity

          Hide
          egutierrez Evaristo Gutierrez added a comment -

          Hi Shurik Gefter

          thanks for using Jenkinsfile Runner. I've noticed how the path is different in the pipeline ("workspace/my") and in the log message ("workspace/my_shared_library"). Maybe it was only that you change it in order to put it here, but could you please make sure the path is correct? Also the path is your host while the Jenkinsfile is run into the Docker image? Then you need to make sure you are sharing the volume.

          Probably you could also configure the shared library though the Configuration As Code plugin. See this example.

          Show
          egutierrez Evaristo Gutierrez added a comment - Hi Shurik Gefter ,  thanks for using Jenkinsfile Runner. I've noticed how the path is different in the pipeline ("workspace/my") and in the log message ("workspace/my_shared_library"). Maybe it was only that you change it in order to put it here, but could you please make sure the path is correct? Also the path is your host while the Jenkinsfile is run into the Docker image? Then you need to make sure you are sharing the volume. Probably you could also configure the shared library though the Configuration As Code plugin. See this example .
          Hide
          shurikg Shurik Gefter added a comment -

          Hi Evaristo,

          You are right, I change the message only in this ticket.

          I align the example

          pipeline {
                          agent any
                          options { skipDefaultCheckout() }
                          stages {
                              stage('Load Library') {
                                  steps {
                                      library identifier: 'my@master', 
                                              retriever: legacySCM(
                                              filesystem(clearWorkspace: true, copyHidden: false, path: "/workspace/my_shared_library"))
                                  }
                              }
                              stage('Test') {
                                  steps {
                                           my_method param1: "aaa"
                                  }
                              }
                          }
                      }
          

          I run it via docker command

          docker run --rm -v /users/shurik/jira_workspace:/workspace  shurikg/jenkinsfile-runner  
          

          The output

            16.932 [id=25]        WARNING i.j.p.c.i.s.VaultSecretSource#getCommaSeparatedVariables: [Deprecation Warning] CASC_VAULT_PATH will be deprecated. Please use CASC_VAULT_PATHS instead.
          Started
          Running in Durability level: PERFORMANCE_OPTIMIZED
          [Pipeline] Start of Pipeline
          [Pipeline] node
          Running on Jenkins in /tmp/jenkinsfileRunner.tmp/jfr2108196765485727374.run/workspace/job
          [Pipeline] {
          [Pipeline] stage
          [Pipeline] { (Load Library)
          [Pipeline] library
          Loading library my@master
          FSSCM.checkout /workspace/my_shared_library to /tmp/jenkinsfileRunner.tmp/jfr2108196765485727374.run/workspace/job@libs/my
          FSSCM.clearWorkspace...
          
          

          My jira_workspace folder contains

          jira_workspace/my_shared_library/vars
          jira_workspace/my_shared_library/vars/my_method.groovy
          jira_workspace/Jenkinsfile
          

          Please note, when I run with the same pipe and workspace via cli runner (not docker), it's working ok.

          Show
          shurikg Shurik Gefter added a comment - Hi Evaristo, You are right, I change the message only in this ticket. I align the example pipeline { agent any options { skipDefaultCheckout() } stages { stage( 'Load Library' ) { steps { library identifier: 'my@master' , retriever: legacySCM( filesystem(clearWorkspace: true , copyHidden: false , path: "/workspace/my_shared_library" )) } } stage( 'Test' ) { steps { my_method param1: "aaa" } } } } I run it via docker command docker run --rm -v /users/shurik/jira_workspace:/workspace shurikg/jenkinsfile-runner The output 16.932 [id=25] WARNING i.j.p.c.i.s.VaultSecretSource#getCommaSeparatedVariables: [Deprecation Warning] CASC_VAULT_PATH will be deprecated. Please use CASC_VAULT_PATHS instead. Started Running in Durability level: PERFORMANCE_OPTIMIZED [Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in /tmp/jenkinsfileRunner.tmp/jfr2108196765485727374.run/workspace/job [Pipeline] { [Pipeline] stage [Pipeline] { (Load Library) [Pipeline] library Loading library my@master FSSCM.checkout /workspace/my_shared_library to /tmp/jenkinsfileRunner.tmp/jfr2108196765485727374.run/workspace/job@libs/my FSSCM.clearWorkspace... My jira_workspace folder contains jira_workspace/my_shared_library/vars jira_workspace/my_shared_library/vars/my_method.groovy jira_workspace/Jenkinsfile Please note, when I run with the same pipe and workspace via cli runner (not docker), it's working ok.
          Hide
          egutierrez Evaristo Gutierrez added a comment -

          Can you please make a couple things and let me know the results?

          • sh 'ls /workspace' inside the pipeline so that we can see the content of such folder the pipeline is able to view (before the shared library loading of course).
          • FSSCM.checkout /workspace/my_shared_library to /tmp/jenkinsfileRunner.tmp/jfr2108196765485727374.run/workspace/job@libs/my
            I can still see only "my" at the end. Could you change the folder to something without the '_' character to validate that is not the issue?

          Thanks!

          Show
          egutierrez Evaristo Gutierrez added a comment - Can you please make a couple things and let me know the results? sh 'ls /workspace' inside the pipeline so that we can see the content of such folder the pipeline is able to view (before the shared library loading of course). FSSCM.checkout /workspace/my_shared_library to /tmp/jenkinsfileRunner.tmp/jfr2108196765485727374.run/workspace/job@libs/my I can still see only "my" at the end. Could you change the folder to something without the '_' character to validate that is not the issue? Thanks!
          Hide
          shurikg Shurik Gefter added a comment -

          Hi,

          The "ls" command show all the relevant files under /workspace 

          + ls -la /workspace
          total 4
          drwxr-xr-x. 3 1001 1001  50 Apr  7 11:53 .
          drwxr-xr-x. 1 root root  68 Apr  7 11:53 ..
          -rw-r--r--. 1 1001 1001 717 Apr  7 11:53 Jenkinsfile
          drwxr-xr-x. 3 1001 1001  18 Apr  2 20:43 my_shared_library
          

          The "my" string is define in the library command, the identifier define the string.
          When I changed to the below:

              ......
              library identifier: 'my_share@master',

          In output you will get:
              Loading library my_share@master
              FSSCM.checkout /workspace/my_shared_library to         /tmp/jenkinsfileRunner.tmp/jfr408537468992298463.run/workspace/job@libs/my_share

           

          I also tried to remove the _ character, but the result the same, stuck.

           

          Note, you can run the same command

          docker run --rm -v /users/shurik/jira_workspace:/workspace shurikg/jenkinsfile-runner
          

          the image exists in docker hub

          Show
          shurikg Shurik Gefter added a comment - Hi, The "ls" command show all the relevant files under /workspace  + ls -la /workspace total 4 drwxr-xr-x. 3 1001 1001 50 Apr 7 11:53 . drwxr-xr-x. 1 root root 68 Apr 7 11:53 .. -rw-r--r--. 1 1001 1001 717 Apr 7 11:53 Jenkinsfile drwxr-xr-x. 3 1001 1001 18 Apr 2 20:43 my_shared_library The "my" string is define in the library command, the identifier define the string. When I changed to the below:     ......     library identifier: ' my_share @master', In output you will get:     Loading library my_share@master     FSSCM.checkout /workspace/my_shared_library to         /tmp/jenkinsfileRunner.tmp/jfr408537468992298463.run/workspace/job@libs/ my_share   I also tried to remove the _ character, but the result the same, stuck.   Note, you can run the same command docker run --rm -v /users/shurik/jira_workspace:/workspace shurikg/jenkinsfile-runner the image exists in docker hub
          Hide
          egutierrez Evaristo Gutierrez added a comment -

          Thanks for the clarification regarding the name of the folder. I have actually managed to reproduce the issue with your image, it seems somehow related to https://issues.jenkins-ci.org/browse/JENKINS-49560

           

          Could you please try to use CasC as I stated in my first comment to verify it could be a valid workaround for you? Thanks!

          Show
          egutierrez Evaristo Gutierrez added a comment - Thanks for the clarification regarding the name of the folder. I have actually managed to reproduce the issue with your image, it seems somehow related to https://issues.jenkins-ci.org/browse/JENKINS-49560   Could you please try to use CasC as I stated in my first comment to verify it could be a valid workaround for you? Thanks!
          Hide
          shurikg Shurik Gefter added a comment -

          I found WA.

          Before I call the library command I manually create the /tmp/jenkinsfileRunner.tmp/jfr408537468992298463.run/workspace/job@libs folder and copy files to there.

          After it my jenkinsfile run successfully via docker.

                              stage('Load Library') {
                                  steps {
                                      sh "echo 'WA - when running via docker the load library stuck, do the preparation manually'"
                                      sh "mkdir -p ${WORKSPACE}/../job@libs/"
                                      sh "rm -rf /workspace/my_shared_library/.git"
                                      sh "cp -r /workspace/my_shared_library ${WORKSPACE}/../job@libs"
          
                                      library identifier: 'my_shared_library@main', 
                                              retriever: legacySCM(
                                              filesystem(clearWorkspace: false, copyHidden: false, path: '/workspace/my_shared_library'))
                                  }
                              }
          

           

          Show
          shurikg Shurik Gefter added a comment - I found WA. Before I call the library command I manually create the /tmp/jenkinsfileRunner.tmp/jfr408537468992298463.run/workspace/job@libs folder and copy files to there. After it my jenkinsfile run successfully via docker. stage( 'Load Library' ) { steps { sh "echo 'WA - when running via docker the load library stuck, do the preparation manually' " sh "mkdir -p ${WORKSPACE}/../job@libs/" sh "rm -rf /workspace/my_shared_library/.git" sh "cp -r /workspace/my_shared_library ${WORKSPACE}/../job@libs" library identifier: 'my_shared_library@main' , retriever: legacySCM( filesystem(clearWorkspace: false , copyHidden: false , path: '/workspace/my_shared_library' )) } }  

            People

            • Assignee:
              egutierrez Evaristo Gutierrez
              Reporter:
              shurikg Shurik Gefter
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: