Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50975

Concurrent Groovy Shared Library syncs on different jobs use same workspace root

    Details

    • Type: Bug
    • Status: Reopened (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Environment:
    • Similar Issues:

      Description

      We're encountering an issue where multiple jobs syncing a groovy shared library concurrently results in the wrong sync root being used for some of the jobs. For example, job1 and job2 are triggered at the same time and use the p4 plugin for syncing the groovy shared library. job1 will have the correct workspace root (e.g.  /var/lib/jenkins/jobs/job1/workspace%40libs/LIBRARYNAME) for the groovy shared library sync, but job2 will sync the groovy shared library to the same workspace root as job1 (e.g.  /var/lib/jenkins/jobs/job1/workspace%40libs/LIBRARYNAME). When job2 loads the groovy shared library, it references what the workspace root should be for job2 (e.g.  /var/lib/jenkins/jobs/job2/workspace%40libs/LIBRARYNAME), which was not sync'd to, and causes job2 to run based off of an old groovy shared library sync.

        Attachments

        1. error_logs.txt
          2 kB
          Sven Schubert
        2. job1.txt
          4 kB
          Kurt Routley
        3. job2.txt
          4 kB
          Kurt Routley

          Activity

          Hide
          p4paul Paul Allen added a comment -

          Released in 1.9.3

          Show
          p4paul Paul Allen added a comment - Released in 1.9.3
          Hide
          jazzyjayx Jay Spang added a comment - - edited

          I know this issue is old, but I think it's repro'ing for me on Jenkins 2.164.3 with P4 Plugin 1.9.7.

          Here is an example pipeline that repros the error. Note that it DYNAMICALLY loads a pipeline.

          library(
              identifier: 'dynamically-loaded-library@now', retriever: legacySCM(
                  [
                      $class: 'PerforceScm', credential: 'p4creds',
                      populate: [
                          $class: 'AutoCleanImpl', 
                          delete: true, 
                          modtime: false, 
                          parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], 
                          pin: '', 
                          quiet: true, 
                          replace: true, 
                          tidy: false
                      ], 
                      workspace: [
                          $class: 'ManualWorkspaceImpl', 
                          charset: 'none', 
                          name: "jenkins-${env.JOB_NAME}-dynamically-loaded-library",
                          pinHost: false, 
                          spec: [allwrite: true, clobber: false, compress: false, line: 'LOCAL', locked: false, modtime: false, rmdir: false, streamName: '', 
                              view: "//depot/build-library/... //jenkins-${env.JOB_NAME}-dynamically-loaded-library/..."
                          ]
                      ]
                  ]
              )
          )
          
          node() {
              stage('Echo') {
                  echo("If you hit 'Build Now' rapidly enough, a build wiill fail before it even gets here.")
              }
          }
          
          

          If you hit "Build Now" once, it works great. If you rapidly hit it a few times, several builds will fail with this error.

          P4 Task: reverting all pending and shelved revisions.
          ... p4 revert d:\JenkinsHome\jobs\stream_name\workspac___
           -
          p4 revert d:\JenkinsHome\jobs\stream_name\jobs\stream_name\workspace_libs\dynamically-loaded-library_2/...
          
          ERROR: P4: Task Exception: hudson.AbortException: P4JAVA: Error(s):
          Path 'd:\JenkinsHome\jobs\stream_name\workspace_libs\dynamically-loaded-library_2/...' is not under client's root 'd:\JenkinsHome\jobs\stream_name\workspace_libs\dynamically-loaded-library'.
          

          It seems like when two builds are run in a VERY close timeframe, the plugin will append "_2" to the library's foldername, but re-uses the old p4 workspace.

          EDIT: Edited my original comment to be more concise, with a working example.

          Show
          jazzyjayx Jay Spang added a comment - - edited I know this issue is old, but I think it's repro'ing for me on Jenkins 2.164.3 with P4 Plugin 1.9.7. Here is an example pipeline that repros the error. Note that it DYNAMICALLY loads a pipeline. library( identifier: 'dynamically-loaded-library@now' , retriever: legacySCM( [ $class: 'PerforceScm' , credential: 'p4creds' , populate: [ $class: 'AutoCleanImpl' , delete: true , modtime: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: true , replace: true , tidy: false ], workspace: [ $class: 'ManualWorkspaceImpl' , charset: 'none' , name: "jenkins-${env.JOB_NAME}-dynamically-loaded-library" , pinHost: false , spec: [allwrite: true , clobber: false , compress: false , line: 'LOCAL' , locked: false , modtime: false , rmdir: false , streamName: '', view: " //depot/build-library/... //jenkins-${env.JOB_NAME}-dynamically-loaded-library/..." ] ] ] ) ) node() { stage( 'Echo' ) { echo( "If you hit 'Build Now' rapidly enough, a build wiill fail before it even gets here." ) } } If you hit "Build Now" once, it works great. If you rapidly hit it a few times, several builds will fail with this error. P4 Task: reverting all pending and shelved revisions. ... p4 revert d:\JenkinsHome\jobs\stream_name\workspac___ - p4 revert d:\JenkinsHome\jobs\stream_name\jobs\stream_name\workspace_libs\dynamically-loaded-library_2/... ERROR: P4: Task Exception: hudson.AbortException: P4JAVA: Error(s): Path 'd:\JenkinsHome\jobs\stream_name\workspace_libs\dynamically-loaded-library_2/...' is not under client 's root ' d:\JenkinsHome\jobs\stream_name\workspace_libs\dynamically-loaded-library'. It seems like when two builds are run in a VERY close timeframe, the plugin will append "_2" to the library's foldername, but re-uses the old p4 workspace. EDIT: Edited my original comment to be more concise, with a working example.
          Hide
          msmeeth Matthew Smeeth added a comment -

          Hi Jay, 

          I'm currently trying to reproduce your issue, so far I have been unable to. If possible could you share your library file?

          Thanks,

          Matthew

          Show
          msmeeth Matthew Smeeth added a comment - Hi Jay,  I'm currently trying to reproduce your issue, so far I have been unable to. If possible could you share your library file? Thanks, Matthew
          Hide
          jazzyjayx Jay Spang added a comment - - edited

          To be extra clear, it only repros for me with dynamically loaded libraries.

          If you import your library with...

          library 'static-global-library@now'

          it doesn't happen. But if you load your library with...

          library(identifier: 'dynamically-loaded-library@now', retriever: legacySCM( // ... ) )

          it happens. Just click "Build Now" as rapidly as you you can 4-5 times.

          I can still send you a library if you'd like, but the failure happens before it even loads, so its contents don't matter.

          Show
          jazzyjayx Jay Spang added a comment - - edited To be extra clear, it only repros for me with dynamically loaded libraries. If you import your library with... library ' static -global-library@now' it doesn't happen. But if you load your library with... library(identifier: 'dynamically-loaded-library@now' , retriever: legacySCM( // ... ) ) it happens. Just click "Build Now" as rapidly as you you can 4-5 times. I can still send you a library if you'd like, but the failure happens before it even loads, so its contents don't matter.
          Hide
          jazzyjayx Jay Spang added a comment -

          Hey Matthew Smeeth, I was wondering if you had any luck repro'ing the error? The key is to use "dynamically loaded" libraries (where you specify the `retriever` property). You also have to hit "Build Now" very rapidly to get it to repro.

          I hypothesize that p4groovy isn't changing the workspace root when it decides to append "_2" to the folder name. Since it works properly when specifying global libraries (that were previously loaded into the Jenkins global config), maybe it uses a slightly different code path.

          Show
          jazzyjayx Jay Spang added a comment - Hey Matthew Smeeth , I was wondering if you had any luck repro'ing the error? The key is to use "dynamically loaded" libraries (where you specify the `retriever` property). You also have to hit "Build Now" very rapidly to get it to repro. I hypothesize that p4groovy isn't changing the workspace root when it decides to append "_2" to the folder name. Since it works properly when specifying global libraries (that were previously loaded into the Jenkins global config), maybe it uses a slightly different code path.

            People

            • Assignee:
              p4paul Paul Allen
              Reporter:
              kroutley Kurt Routley
            • Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated: