Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51525

Pipeline: Poll per change syncs to head when using 'checkout' or 'p4 sync' step.

    Details

    • Similar Issues:

      Description

      When using Poll per change the system will correctly run individual builds for each new changelist but if using 'p4sync' or 'checkout' steps are used they will sync to #HEAD instead of the changelist to be built.

      For example if the Jenkinsfile relies on the implicit sync that happens automatically the behavior is:

       

      Setup job with a 30 second sleep at start
      Submit CL 1, 2 ,3, 4
      Build 1 runs that syncs up to changelist 1 and reports that only CL 1 has built.
      Build 2 runs that syncs up to changelist 2 and reports that only CL 2 has built.
      Build 3 runs that syncs up to changelist 3 and reports that only CL 3 has built.
      Build 4 runs that syncs up to changelist 4 and reports that only CL 4 has built.

       

       

      If instead the implicit sync is switched off and an explicit  'p4sync' or  checkout' is used in the Jenkinsfile the behavior is:

       

      Setup job with a 30 second sleep at start
      Submit CL 1, 2 ,3, 4
      Build 1 runs that syncs up to changelist 4 but reports that only CL 1 has built.
      Build 2 runs that syncs up to changelist 4 but reports that only CL 2 has built.
      Build 3 runs that syncs up to changelist 4 but reports that only CL 3 has built.
      Build 4 runs that syncs up to changelist 4 but reports that only CL 4 has built.

       

      Reproduction Step:

      (1) Create a pipeline job with Jenkinsfile in SCM and polling build filter of 'Polling Per Change'.

      (2) Create a dummy Jenkinsfile with a sleep and submit to Perforce. For example:

      pipeline {
          //agent any
          agent{label 'MySlave'}
          //options { skipDefaultCheckout() }  
        stages {
          stage("PollPerChange") {
            steps {
              script {
                  echo "In script."
                  echo "Sleeping..."
                  sleep 30
                  //checkout perforce(credential: 'MasterServer', filter: [incremental(true)], populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: true, replace: true, tidy: false), workspace: manualSpec(charset: 'none', name: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}', pinHost: false, spec: clientSpec(allwrite: false, backup: false, clobber: true, compress: false, line: 'LOCAL', locked: false, modtime: false, rmdir: false, serverID: '', streamName: '', type: 'WRITABLE', view: '//depot/pollperchange/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}/...')))
                  sh 'ls'  
              }
            }
          }
        }
      }
      

      (3) Create 3 changelists in relevant path then run 'poll now' on job (customised to your environment) as it is.

      (4) Uncomment 3 lines in Jenkinsfile and submit to Perforce.

      (5) Create 3 changelists in relevant path then run 'poll now' on job.

       

       

       

        Attachments

          Issue Links

            Activity

            Hide
            p4paul Paul Allen added a comment - - edited

            The Jenkinsfile in my test setup (I also set Poll per Change on the Pipeline Script from SCM):

            pipeline {
             agent any 
             stages {
              stage('Build') { 
               steps {
                checkout perforce(credential: 'phooey', filter: [incremental(true)], populate: autoClean(quiet: true), workspace: streamSpec(format: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}', streamName: '//deep/subX/main'))
               }
              }
             }
            } 

            //deep/subX/main is a stream in a different location to the Jenkinsfile defined in the Pipeline Script from SCM (also a stream at //deep/projA/main).  Changes in //deep/subX/main seem to sync as expected with Poll per Change.

            Changes to //deep/projA/main are not seen by polling as Lightweight checkout was selected (client is thrown away and p4 print is used so no have list). 

            Show
            p4paul Paul Allen added a comment - - edited The Jenkinsfile in my test setup (I also set Poll per Change on the Pipeline Script from SCM): pipeline { agent any stages { stage( 'Build' ) { steps { checkout perforce(credential: 'phooey' , filter: [incremental( true )], populate: autoClean(quiet: true ), workspace: streamSpec(format: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}' , streamName: ' //deep/subX/main' )) } } } } //deep/subX/main is a stream in a different location to the Jenkinsfile defined in the Pipeline Script from SCM (also a stream at //deep/projA/main).  Changes in //deep/subX/main seem to sync as expected with Poll per Change. Changes to //deep/projA/main are not seen by polling as Lightweight checkout was selected (client is thrown away and p4 print is used so no have list). 
            Hide
            danielresolutiongames Daniel Hagström added a comment -

            Any updates on this? Our organization is "stuck" on 1.8.9 until this and the (AFAICT) very much related JENKINS-52066 is resolved.

            Show
            danielresolutiongames Daniel Hagström added a comment - Any updates on this? Our organization is "stuck" on 1.8.9 until this and the (AFAICT) very much related  JENKINS-52066  is resolved.
            Hide
            p4karl Karl Wirth added a comment -

            Hi Daniel Hagström. I need to test if this is fixed by a related change we made. I will let you know.

             

            Show
            p4karl Karl Wirth added a comment - Hi Daniel Hagström . I need to test if this is fixed by a related change we made. I will let you know.  
            Hide
            p4karl Karl Wirth added a comment - - edited

            Hi Daniel Hagström. I've tested with P4-Plugin 1.9.4 and Jenkins 2.138.1 and the above now correctly works.

            Important Note: There is a known bug in 1.9.4 and documented workaround. Please see:

            JENKINS-54695

            Below is the example output when implicit checkout is disabled and I have submitted changes 90, 91 and 92:

            P4 Task: establishing connection.
            ... server: vm-swarm182-xenial:1666
            ... node: vm-swarm182-xenial
            P4: Polling with range: 89,now
            ... p4 changes -m20 //jenkins-master-PollPerChange-0/...@89,now +... p4 change -o 92 +... p4 describe -s 92 +... found change: 92
            ... p4 change -o 91 +... p4 describe -s 91 +... found change: 91
            ... p4 change -o 90 +... p4 describe -s 90 +... found change: 90
            ... p4 repos -C +Building on Node: master
            ... p4 client -o jenkins-master-PollPerChange-0 +... p4 info +... p4 info +... p4 client -i +...   View: +
            P4 Task: establishing connection.
            ... server: vm-swarm182-xenial:1666
            ... node: vm-swarm182-xenial
            
            P4 Task: reverting all pending and shelved revisions.
            ... p4 revert /var/lib/jenkins/workspace/PollPerChange/... +... rm [abandoned files]
            duration: (2ms)
            
            P4 Task: cleaning workspace to match have list.
            ... p4 reconcile -w -f /var/lib/jenkins/workspace/PollPerChange/... +duration: (2ms)
            
            P4 Task: syncing files at change: 90
            ... p4 sync -q /var/lib/jenkins/workspace/PollPerChange/...@90 +duration: (2ms)
            

            Note that it finds all the changes but only syncs changelist '90'.

             

            I will look into JENKINS-52066 next.

             

            Show
            p4karl Karl Wirth added a comment - - edited Hi Daniel Hagström . I've tested with P4-Plugin 1.9.4 and Jenkins 2.138.1 and the above now correctly works. Important Note: There is a known bug in 1.9.4 and documented workaround. Please see: JENKINS-54695 Below is the example output when implicit checkout is disabled and I have submitted changes 90, 91 and 92: P4 Task: establishing connection. ... server: vm-swarm182-xenial:1666 ... node: vm-swarm182-xenial P4: Polling with range: 89,now ... p4 changes -m20 //jenkins-master-PollPerChange-0/...@89,now +... p4 change -o 92 +... p4 describe -s 92 +... found change: 92 ... p4 change -o 91 +... p4 describe -s 91 +... found change: 91 ... p4 change -o 90 +... p4 describe -s 90 +... found change: 90 ... p4 repos -C +Building on Node: master ... p4 client -o jenkins-master-PollPerChange-0 +... p4 info +... p4 info +... p4 client -i +... View: + P4 Task: establishing connection. ... server: vm-swarm182-xenial:1666 ... node: vm-swarm182-xenial P4 Task: reverting all pending and shelved revisions. ... p4 revert / var /lib/jenkins/workspace/PollPerChange/... +... rm [abandoned files] duration: (2ms) P4 Task: cleaning workspace to match have list. ... p4 reconcile -w -f / var /lib/jenkins/workspace/PollPerChange/... +duration: (2ms) P4 Task: syncing files at change: 90 ... p4 sync -q / var /lib/jenkins/workspace/PollPerChange/...@90 +duration: (2ms) Note that it finds all the changes but only syncs changelist '90'.   I will look into JENKINS-52066 next.  
            Hide
            danielresolutiongames Daniel Hagström added a comment -

            All right, Karl Wirth. Thanks.

            Show
            danielresolutiongames Daniel Hagström added a comment - All right, Karl Wirth . Thanks.

              People

              • Assignee:
                p4paul Paul Allen
                Reporter:
                p4karl Karl Wirth
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: