Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60358

Support Root=null value in Template workspace - Currently it is overridden, failing sync

    Details

    • Similar Issues:

      Description

      P4 Windows use Root:null to support mapping sync to different drives. 
      this value is always overwritten when using p4 plugin, thus failing the sync. 
      p4 sync should use .../@ instead, and the workspace Root should not change. 

        Attachments

          Activity

          Hide
          p4karl Karl Wirth added a comment -

          Hi Shai Shapira - Thanks for the feedback.

          Can you give me an example of why your build systems are using different drives for the Perforce data so that I can pass that to the developers. Please also give me screen shots of your job definition and/or Jenkinsfile.

          Jenkins usually controls where we should sync the files which is why p4-plugin is designed to overwrite the root with the location provided by Jenkins. However this does not work well with the model of different files on different Windows drive or fixed locations on the disk.

          To work around the problem you could use multiple 'p4sync' or 'checkout' steps to each drive in a Jenkinsfile but would need to change 'ws' path before each sync.  Another way is to use P4Groovy or a batch file and 'p4.exe' but you would lose the power of polling and change reporting.

           

          Reproduction Steps:

          (1) Setup a build slave on Windows.

          (2) Create a workspace with a null root. For example:

          Client:	ws_root_set
          Root:	null
          View:
          	//depot/Project1/... //ws_root_set/C:/filestore/ws/JENKINS/...

          This would sync the files from '//depot/project1/...' to 'C:\filestore\ws\JENKINS'.

          (3) If you use the workspace behavior 'Static' we will try to sync the path 'null/...'. If you use Template the '\C:\' would be inserted into the path. Each way will cause a bad build.

          See related job JENKINS-47005.

           

          Show
          p4karl Karl Wirth added a comment - Hi Shai Shapira - Thanks for the feedback. Can you give me an example of why your build systems are using different drives for the Perforce data so that I can pass that to the developers. Please also give me screen shots of your job definition and/or Jenkinsfile. Jenkins usually controls where we should sync the files which is why p4-plugin is designed to overwrite the root with the location provided by Jenkins. However this does not work well with the model of different files on different Windows drive or fixed locations on the disk. To work around the problem you could use multiple 'p4sync' or 'checkout' steps to each drive in a Jenkinsfile but would need to change 'ws' path before each sync.  Another way is to use P4Groovy or a batch file and 'p4.exe' but you would lose the power of polling and change reporting.   Reproduction Steps: (1) Setup a build slave on Windows. (2) Create a workspace with a null root. For example: Client: ws_root_set Root: null View: //depot/Project1/... //ws_root_set/C:/filestore/ws/JENKINS/... This would sync the files from '//depot/project1/...' to 'C:\filestore\ws\JENKINS'. (3) If you use the workspace behavior 'Static' we will try to sync the path 'null/...'. If you use Template the '\C:\' would be inserted into the path. Each way will cause a bad build. See related job JENKINS-47005 .  
          Hide
          guy_catz Guy Catz added a comment -

          Hi Karl,

          I'm Guy, working Shai Shapira on that.

          To answer your question "....why your build systems are using different drives for the Perforce data".

          Our Windows build is comprised of a large amount of very small files (10^5). To make the cleanup faster after a build, we simply format that disk.

          Moreover, a big chunk of the sources (~ 30GB) are SDKs that are rarely being changed. Instead of forced-syncing it on every build, we keep it in a separated drive, (non-forced) sync it and logically combined both drive into a one partition.

          For that, we keep the root null.

          The job below is a test for that matter.

          Could u explain more about the workaround? How do we achieve that while still using the p4 plugin?

          As a solution, can you introduce a checkbox "override root path" (default checked)?

          Thanks

          Show
          guy_catz Guy Catz added a comment - Hi Karl, I'm Guy, working Shai Shapira on that. To answer your question "....why your build systems are using different drives for the Perforce data". Our Windows build is comprised of a large amount of very small files (10^5). To make the cleanup faster after a build, we simply format that disk. Moreover, a big chunk of the sources (~ 30GB) are SDKs that are rarely being changed. Instead of forced-syncing it on every build, we keep it in a separated drive, (non-forced) sync it and logically combined both drive into a one partition. For that, we keep the root null. The job below is a test for that matter. Could u explain more about the workaround? How do we achieve that while still using the p4 plugin? As a solution, can you introduce a checkbox "override root path" (default checked)? Thanks
          Hide
          p4karl Karl Wirth added a comment -

          Hi Guy Catz, Yes I'll put the checkbox for null root in as an enhancement request.

          The workaround is to use a Pipeline job and do separate sync's for each drive.

          Below is an example of a piece of code that runs on a slave called 'Windows10' and syns code from '//depot/folder1/...' to C:\Jenkins1 and '//depot/folder2/...' to 'E:\Jenkins2':

           

          pipeline {
            
            agent { label "Windows10" }
            
            stages {
              stage("Sync to C:\\") {
                steps {
                  ws("C:\\Jenkins1") {
                     p4sync charset: 'none', credential: 'JenkinsMaster', populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false, replace: true, tidy: false), source: depotSource('//depot/folder1/...')
                  }
                }
              }
              stage("Sync to D:\\") {
                steps {
                  ws("E:\\Jenkins2") {
                     p4sync charset: 'none', credential: 'JenkinsMaster', populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false, replace: true, tidy: false), source: depotSource('//depot/folder2/...')
                  }
                }
              }
            }
          }
          

           

           

           

          Show
          p4karl Karl Wirth added a comment - Hi Guy Catz , Yes I'll put the checkbox for null root in as an enhancement request. The workaround is to use a Pipeline job and do separate sync's for each drive. Below is an example of a piece of code that runs on a slave called 'Windows10' and syns code from '//depot/folder1/...' to C:\Jenkins1 and '//depot/folder2/...' to 'E:\Jenkins2':   pipeline { agent { label "Windows10" } stages { stage( "Sync to C:\\" ) { steps { ws( "C:\\Jenkins1" ) { p4sync charset: 'none' , credential: 'JenkinsMaster' , populate: autoClean(delete: true , modtime: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: false , replace: true , tidy: false ), source: depotSource(' //depot/folder1/...') } } } stage( "Sync to D:\\" ) { steps { ws( "E:\\Jenkins2" ) { p4sync charset: 'none' , credential: 'JenkinsMaster' , populate: autoClean(delete: true , modtime: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: false , replace: true , tidy: false ), source: depotSource(' //depot/folder2/...') } } } } }      
          Hide
          shai_shap Shai Shapira added a comment -

          Thanks, but this breaks current simplicity for allowing multi-config management as well (using Jenkins multi config add-on) - Use of pipeline job affects UI simplification for users, which we are currently looking for.

          Show
          shai_shap Shai Shapira added a comment - Thanks, but this breaks current simplicity for allowing multi-config management as well (using Jenkins multi config add-on) - Use of pipeline job affects UI simplification for users, which we are currently looking for.

            People

            • Assignee:
              Unassigned
              Reporter:
              shai_shap Shai Shapira
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: