Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38508

git scm poll doesn't ignore changes with pipeline

    Details

    • Similar Issues:

      Description

      We are using the job as attached with the groovy test.groovy file. We poll the SCM for changes but want to ignore any change that is in our jenkins-jobs git repo. This is because in a full setup we poll more repositories which do change. In the jenkins pipeline we use .* as exclude region. This does work unless we clone the same repository in our test.groovy. There we say poll=false, but the repository is polled and at the moment we make a commit to jenkins-jobs.git the job is run. With these configuration we would expect this job to never run, even when we schedule it frequrently

        Attachments

        1. config.xml
          2 kB
        2. test.groovy
          0.2 kB

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            Would need to study your example more closely since so far I do not understand it. (Generally, steps to reproduce from scratch are more helpful than explanations.) Possibly same as JENKINS-38659.

            Show
            jglick Jesse Glick added a comment - Would need to study your example more closely since so far I do not understand it. (Generally, steps to reproduce from scratch are more helpful than explanations.) Possibly same as JENKINS-38659 .
            Hide
            jwillemsen Johnny Willemsen added a comment -

            I also don't 100% understand it. Mostly it seems the polling does work, but in some cases we still see jobs to run even when nothing changed

            Show
            jwillemsen Johnny Willemsen added a comment - I also don't 100% understand it. Mostly it seems the polling does work, but in some cases we still see jobs to run even when nothing changed
            Hide
            0x89 Martin Sander added a comment - - edited

            JENKINS-39621 might also be related.

            Show
            0x89 Martin Sander added a comment - - edited JENKINS-39621 might also be related.
            Hide
            jlehtnie Jukka Lehtniemi added a comment - - edited

            I can reproduce this (Jenkins 2.43, Git plugin 3.0.1, Pipeline plugin 2.4).

            Steps to reproduce:

            1. Create a test repository:
              git init /tmp/testrepo; cd /tmp/testrepo; echo foo > foo; git add foo; git commit -m foo
              
            2. Create a new pipeline job with script:
              properties([pipelineTriggers([pollSCM('* * * * *')])])
              node("master") {
                  git(url: '/tmp/testrepo', poll:false)
              }
              
            3. Trigger the pipeline once manually
            4. Commit more content to the test repository
               
              echo foo2 >> foo; git commit -a -m foo2
              
            5. Observe how the build is being triggered in a minute even if the poll=false is set
            Show
            jlehtnie Jukka Lehtniemi added a comment - - edited I can reproduce this (Jenkins 2.43, Git plugin 3.0.1, Pipeline plugin 2.4). Steps to reproduce: Create a test repository: git init /tmp/testrepo; cd /tmp/testrepo; echo foo > foo; git add foo; git commit -m foo Create a new pipeline job with script: properties([pipelineTriggers([pollSCM( '* * * * *' )])]) node( "master" ) { git(url: '/tmp/testrepo' , poll: false ) } Trigger the pipeline once manually Commit more content to the test repository echo foo2 >> foo; git commit -a -m foo2 Observe how the build is being triggered in a minute even if the poll=false is set
            Hide
            pgeorgiev Pavel Georgiev added a comment - - edited

            I'm not sure if this is the correct defect but im my case when i checkout a git repo using the checkout step anc configure poll: false i see in the git pooling log that this repo is being checked for changes....

            node('build') {

            checkout changelog: false, poll: false,
            scm: [$class: 'GitSCM',
            branches: [[name: '*/master']],
            doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [],
            userRemoteConfigs: [[credentialsId: 'git_ci_credentials', url: 'https://hidden']]]
            }

            The pooling log:

            Using strategy: Default
            [poll] Last Built Revision: Revision hidden (refs/remotes/origin/master)
            using GIT_ASKPASS to set credentials git_ci_credentials
            > git ls-remote -h https://hidden # timeout=10
            Found 6 remote heads on https://hidden
            [poll] Latest remote head revision on refs/heads/master is: hidden - already built by 171
            Done. Took 5.8 sec
            No changes

            Show
            pgeorgiev Pavel Georgiev added a comment - - edited I'm not sure if this is the correct defect but im my case when i checkout a git repo using the checkout step anc configure poll: false i see in the git pooling log that this repo is being checked for changes.... node('build') { checkout changelog: false, poll: false, scm: [$class: 'GitSCM', branches: [ [name: '*/master'] ], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [ [credentialsId: 'git_ci_credentials', url: 'https:// hidden '] ]] } The pooling log: Using strategy: Default [poll] Last Built Revision: Revision hidden (refs/remotes/origin/master) using GIT_ASKPASS to set credentials git_ci_credentials > git ls-remote -h https:// hidden # timeout=10 Found 6 remote heads on https:// hidden [poll] Latest remote head revision on refs/heads/master is: hidden - already built by 171 Done. Took 5.8 sec No changes
            Hide
            ansib ansib added a comment - - edited

            I have here a small Explanation on StackOverflow with the same/similar case. Maybe it could help you to understand what the problem with the polling is.

            http://stackoverflow.com/questions/40017034/jenkins-pipeline-poll-scm-with-different-repo-for-jenkinsfile-and-application-re

            I try to summarize the problem we have at our company.

            We have two repositories. One for the configuration of Jenkinsfiles (config-repo) and on for our Development (dev-repo), which is included in the Jenkinsfile.
            In Jenkins we have 3 Jobs for deployment of the dev-repo. One for dev branch, one for hotfix branch and one for release branch. --> Jenkinsfiles are located in the config-repo. the git step in the Jenkinsfile check out the dev-repo.

            If i check new code into the config-repo, the builds aren't triggered, but if we check new code into dev-repo after a checkin/change in the config-repo all jobs configured with the config-repo and using the dev-repo starts a build.

            The Jobs should never start a build after checkins into config-repo and only starts a build, if new code is checked into their specific branch at the dev-repo.

            Show
            ansib ansib added a comment - - edited I have here a small Explanation on StackOverflow with the same/similar case. Maybe it could help you to understand what the problem with the polling is. http://stackoverflow.com/questions/40017034/jenkins-pipeline-poll-scm-with-different-repo-for-jenkinsfile-and-application-re I try to summarize the problem we have at our company. We have two repositories. One for the configuration of Jenkinsfiles (config-repo) and on for our Development (dev-repo), which is included in the Jenkinsfile. In Jenkins we have 3 Jobs for deployment of the dev-repo. One for dev branch, one for hotfix branch and one for release branch. --> Jenkinsfiles are located in the config-repo. the git step in the Jenkinsfile check out the dev-repo. If i check new code into the config-repo, the builds aren't triggered, but if we check new code into dev-repo after a checkin/change in the config-repo all jobs configured with the config-repo and using the dev-repo starts a build. The Jobs should never start a build after checkins into config-repo and only starts a build, if new code is checked into their specific branch at the dev-repo.
            Hide
            ehaselwanter Edmund Haselwanter added a comment - - edited

            Update:

                      // poll false does not count
                      [$class: 'DisableRemotePoll'],
                      [$class: 'PathRestriction', excludedRegions: '', includedRegions: 'foo/.*']
            

            works as a workaround

            ------ old report -----

            same here.

                checkout(
                  poll: true,
                  scm: [
                    $class: 'GitSCM',
                    userRemoteConfigs: [[
                      url: 'git@xxxxxx/deployed-service-versions.git',
                      credentialsId: xxxxxx,
                    ]],
                    branches: [[name: '*/master']],
                    extensions: [
                      [$class: 'RelativeTargetDirectory', relativeTargetDir: 'deployed-service-versions'],
                      [$class: 'CleanCheckout'],
                    ],
                    doGenerateSubmoduleConfigurations: false,
                    submoduleCfg: [],
                  ],
                )
            
                checkout(
                  poll: false,
                  scm: [
                    $class: 'GitSCM',
                    userRemoteConfigs: [[
                      url: 'git@8xxxxxxxxr/images-reporting.git',
                      credentialsId: 'xxxxx',
                    ]],
                    branches: [[name: '*/master']],
                    extensions: [
                      [$class: 'RelativeTargetDirectory', relativeTargetDir: 'images-reporting'],
                      // prune everything in workspace
                      [$class: 'CleanCheckout'],
                      // avoid detached head. it calculates a local branch derived from the clone (mater in this case)
                      [$class: 'LocalBranch'],
                    ],
                    doGenerateSubmoduleConfigurations: false,
                    submoduleCfg: [],
                  ],
                )
            
            

            does poll and trigger on the `images-reporting.git` repo, where `poll: false` is set

            Started on Feb 17, 2017 11:27:00 AM
            Using strategy: Default
            [poll] Last Built Revision: Revision da056b8e354ad819942980d620f35751b52fad6a (refs/remotes/origin/master)
             > git ls-remote -h git@xxxxxxxx/deployed-service-versions.git # timeout=10
            Found 1 remote heads on git@xxxxxxxxxxxxx/deployed-service-versions.git
            [poll] Latest remote head revision on refs/heads/master is: da056b8e354ad819942980d620f35751b52fad6a - already built by 202
            Using strategy: Default
            [poll] Last Built Revision: Revision 724e92bee2268fe431d86239d97ce04ed673a047 (refs/remotes/origin/master)
            using GIT_SSH to set credentials gitlab
             > git ls-remote -h git@xxxxxxxx/images-reporting.git # timeout=10
            Found 1 remote heads on git@xxxxxx/-images-reporting.git
            [poll] Latest remote head revision on refs/heads/master is: e9cfddb95d27a46e3048d12e0a83fedd9a3dc2ae
            Done. Took 1.6 sec
            Changes found
            

            so, how can I ignore changes in `/images-reporting.git` but build on changes in the the `deployed-service-versions.git' repo?

            Show
            ehaselwanter Edmund Haselwanter added a comment - - edited Update: // poll false does not count [$class: 'DisableRemotePoll' ], [$class: 'PathRestriction' , excludedRegions: '', includedRegions: ' foo/.*'] works as a workaround ------ old report ----- same here. checkout( poll: true , scm: [ $class: 'GitSCM' , userRemoteConfigs: [[ url: 'git@xxxxxx/deployed-service-versions.git' , credentialsId: xxxxxx, ]], branches: [[name: '*/master' ]], extensions: [ [$class: 'RelativeTargetDirectory' , relativeTargetDir: 'deployed-service-versions' ], [$class: 'CleanCheckout' ], ], doGenerateSubmoduleConfigurations: false , submoduleCfg: [], ], ) checkout( poll: false , scm: [ $class: 'GitSCM' , userRemoteConfigs: [[ url: 'git@8xxxxxxxxr/images-reporting.git' , credentialsId: 'xxxxx' , ]], branches: [[name: '*/master' ]], extensions: [ [$class: 'RelativeTargetDirectory' , relativeTargetDir: 'images-reporting' ], // prune everything in workspace [$class: 'CleanCheckout' ], // avoid detached head. it calculates a local branch derived from the clone (mater in this case ) [$class: 'LocalBranch' ], ], doGenerateSubmoduleConfigurations: false , submoduleCfg: [], ], ) does poll and trigger on the `images-reporting.git` repo, where `poll: false` is set Started on Feb 17, 2017 11:27:00 AM Using strategy: Default [poll] Last Built Revision: Revision da056b8e354ad819942980d620f35751b52fad6a (refs/remotes/origin/master) > git ls-remote -h git@xxxxxxxx/deployed-service-versions.git # timeout=10 Found 1 remote heads on git@xxxxxxxxxxxxx/deployed-service-versions.git [poll] Latest remote head revision on refs/heads/master is: da056b8e354ad819942980d620f35751b52fad6a - already built by 202 Using strategy: Default [poll] Last Built Revision: Revision 724e92bee2268fe431d86239d97ce04ed673a047 (refs/remotes/origin/master) using GIT_SSH to set credentials gitlab > git ls-remote -h git@xxxxxxxx/images-reporting.git # timeout=10 Found 1 remote heads on git@xxxxxx/-images-reporting.git [poll] Latest remote head revision on refs/heads/master is: e9cfddb95d27a46e3048d12e0a83fedd9a3dc2ae Done. Took 1.6 sec Changes found so, how can I ignore changes in `/images-reporting.git` but build on changes in the the `deployed-service-versions.git' repo?
            Hide
            furiousd D B added a comment - - edited

            Edmund Haselwanter, Could you explain a bit how to use your workaround?  I'm not sure where to put DisableRemotePoll or PathRestriction.  I tried putting it under the extensions property of checkout scm but it's still polling the secondary repo.

            Update: You add them between the extensions: [] brackets.

            Show
            furiousd D B added a comment - - edited Edmund Haselwanter , Could you explain a bit how to use your workaround?  I'm not sure where to put DisableRemotePoll or PathRestriction.  I tried putting it under the extensions property of checkout scm but it's still polling the secondary repo. Update: You add them between the extensions: [] brackets.
            Hide
            ygorth Ygor Almeida added a comment - - edited

            Thanks to Edmund Haselwanter this works for me:

            checkout(
            poll: false,
            scm: [
              $class: 'GitSCM', branches: [[name: '*/master']],
              userRemoteConfigs: [[
                url: MY_URL.git,
                credentialsId: CREDENTIALS_ID]],
              extensions: [
                [$class: 'DisableRemotePoll'],
                [$class: 'PathRestriction', excludedRegions: '', includedRegions: '*']]
            ])
            

             
            GitSCM poll does ignore changes now.
             
             

            Show
            ygorth Ygor Almeida added a comment - - edited Thanks to Edmund Haselwanter this works for me: checkout( poll: false , scm: [ $class: 'GitSCM' , branches: [[name: '*/master' ]], userRemoteConfigs: [[ url: MY_URL.git, credentialsId: CREDENTIALS_ID]], extensions: [ [$class: 'DisableRemotePoll' ], [$class: 'PathRestriction' , excludedRegions: '', includedRegions: ' *']] ])   GitSCM poll does ignore changes now.    
            Hide
            amoreira Alfredo Moreira added a comment - - edited

            Are you guys using: 
            triggers {
            pollSCM 'H/5 * * * *'
            }
            to trigger the builds? Edmund Haselwanter Ygor Almeida
             
            I have that and it is still polling the repo I do not wan to be polled. This is my set up:
             

            // pipeline{
            //Pipeline Options
            options{
            buildDiscarder(logRotator(numToKeepStr:'5'))
            disableConcurrentBuilds()
            skipDefaultCheckout()
            }
            //define agent/node to run job on on
            agent {
            node{
            label 'windows'
            }
            }
            triggers {
            pollSCM 'H/2 * * * *'
            }
            //Stages/Build Steps
            stages{
            //Set Source Code Management of code
            stage('Set SCM'){
            steps{
            checkout(
            poll:true,
            scm: [
            $class:'GitSCM',
            branches: [[name:branch]],
            doGenerateSubmoduleConfigurations: false,
            clean:true,
            extensions: [[$class: 'DisableRemotePoll'],
            [$class: 'PathRestriction', excludedRegions: '',
            includedRegions: '*']],
            submoduleCfg: [],
            userRemoteConfigs: [[url: 'link to Repo']],
            credentialsId:'Jenkins Creds',
            ])
            }
            }
            //Pull Groovy Function Library
            stage('Load Jenkins-Code Files'){
            steps{
            dir('CI-CD'){
            checkout(
            poll:false,
            scm: [
            $class:'GitSCM',
            branches: [[name:'master']],
            clean:true,
            extensions: [
            [$class: 'DisableRemotePoll'],
            [$class: 'PathRestriction', excludedRegions: '',
            includedRegions: '*']],
            submoduleCfg: [],
            userRemoteConfigs: [[url: 'GitHub Repo']],
            credentialsId:'Jenkins Creds',
            ])
            }
            }
            }

             

            The Load Jenkins Code is the repo i do not want to poll. Like this, both repos are being polled, and if i remove trigger nothing gets polled. Any help would be much appreciated. Thanks in advance.

            Show
            amoreira Alfredo Moreira added a comment - - edited Are you guys using:  triggers { pollSCM 'H/5 * * * *' } to trigger the builds?  Edmund Haselwanter Ygor Almeida   I have that and it is still polling the repo I do not wan to be polled. This is my set up:   // pipeline{ //Pipeline Options options{ buildDiscarder(logRotator(numToKeepStr: '5' )) disableConcurrentBuilds() skipDefaultCheckout() } //define agent/node to run job on on agent { node{ label 'windows' } } triggers { pollSCM 'H/2 * * * *' } //Stages/Build Steps stages{ //Set Source Code Management of code stage( 'Set SCM' ){ steps{ checkout( poll: true , scm: [ $class: 'GitSCM' , branches: [[name:branch]], doGenerateSubmoduleConfigurations: false , clean: true , extensions: [[$class: 'DisableRemotePoll' ], [$class: 'PathRestriction' , excludedRegions: '', includedRegions: '*' ]], submoduleCfg: [], userRemoteConfigs: [[url: 'link to Repo' ]], credentialsId: 'Jenkins Creds' , ]) } } //Pull Groovy Function Library stage( 'Load Jenkins-Code Files' ){ steps{ dir( 'CI-CD' ){ checkout( poll: false , scm: [ $class: 'GitSCM' , branches: [[name: 'master' ]], clean: true , extensions: [ [$class: 'DisableRemotePoll' ], [$class: 'PathRestriction' , excludedRegions: '', includedRegions: '*' ]], submoduleCfg: [], userRemoteConfigs: [[url: 'GitHub Repo' ]], credentialsId: 'Jenkins Creds' , ]) } } }   The Load Jenkins Code is the repo i do not want to poll. Like this, both repos are being polled, and if i remove trigger nothing gets polled. Any help would be much appreciated. Thanks in advance.
            Hide
            michaelw Michael Wong added a comment -

            It seems like polling is always done if changelog is set to true (by default it does get set to true). It should work if you set both to false.

            You also might need to restart your Jenkins server to get it to work (if the job was polling before, since the cached polling baseline never gets reset).

            Show
            michaelw Michael Wong added a comment - It seems like polling is always done if changelog is set to true (by default it does get set to true). It should work if you set both to false. You also might need to restart your Jenkins server to get it to work (if the job was polling before, since the cached polling baseline never gets reset).
            Hide
            sradi sradi added a comment -

            For me, the workaround with "DisableRemotePoll" and "PathRestriction" does not work.
            Did anyone successfully use this workaround?

            Is there anybody, who plans to fix this old issue anytime soon?

            Show
            sradi sradi added a comment - For me, the workaround with "DisableRemotePoll" and "PathRestriction" does not work. Did anyone successfully use this workaround? Is there anybody, who plans to fix this old issue anytime soon?
            Hide
            amoreira Alfredo Moreira added a comment -

            sradi use the option "changelog: false", i had used the Disable with no success, but adding the changelog option fixed everything. You might have to restart Jenkins for the polling to take effect correctly and not use the cache. 

            Here is my git checkout that i dont want to be polled, this repo is cloned into the CI-CD directory inside the project:

            checkout(
            poll:false,
            changelog:false,
            scm: [
            $class:'GitSCM',
            branches: [[name:'master']],
            clean:true,
            extensions: [[$class: 'DisableRemotePoll'],
            [$class: 'PathRestriction', excludedRegions: '*']],
            userRemoteConfigs: [[url: 'GithubURL]],
            credentialsId:'creds-ID,
            ])

            And the polling will look like this:

            Started on Jan 22, 2018 9:43:15 AM Using strategy: Default [poll] Last Built Revision: Revision a2a141227f71769af6e69a0b5d1dacfb6478effe (origin/develop) > git --version # timeout=10 > git ls-remote -h

            ""Github URL"

            1. timeout=10 Found 36 remote heads on

            "Github URL"

            [poll] Latest remote head revision on refs/heads/develop is: a2a141227f71769af6e69a0b5d1dacfb6478effe - already built by 272

            no polling baseline in /Users/mcoeadm/.jenkins/jobs/pe-dev- Davita Dev - Android/workspace/CI-CD on Done. Took 0.69 sec No changes .  --> "THIS IS THE LINE THAT MAKES THE DIFFERENCE"

             

            Hopefully this helps.

            Show
            amoreira Alfredo Moreira added a comment - sradi use the option "changelog: false", i had used the Disable with no success, but adding the changelog option fixed everything. You might have to restart Jenkins for the polling to take effect correctly and not use the cache.  Here is my git checkout that i dont want to be polled, this repo is cloned into the CI-CD directory inside the project: checkout( poll:false, changelog:false, scm: [ $class:'GitSCM', branches: [ [name:'master'] ], clean:true, extensions: [ [$class: 'DisableRemotePoll'] , [$class: 'PathRestriction', excludedRegions: '*'] ], userRemoteConfigs: [ [url: 'GithubURL] ], credentialsId:'creds-ID, ]) And the polling will look like this: Started on Jan 22, 2018 9:43:15 AM Using strategy: Default [poll] Last Built Revision: Revision a2a141227f71769af6e69a0b5d1dacfb6478effe (origin/develop) > git --version # timeout=10 > git ls-remote -h ""Github URL" timeout=10 Found 36 remote heads on "Github URL" [poll] Latest remote head revision on refs/heads/develop is: a2a141227f71769af6e69a0b5d1dacfb6478effe - already built by 272 no polling baseline in /Users/mcoeadm/.jenkins/jobs/pe-dev- Davita Dev - Android/workspace/CI-CD on Done. Took 0.69 sec No changes .  --> "THIS IS THE LINE THAT MAKES THE DIFFERENCE"   Hopefully this helps.
            Hide
            aaronling Aaron Ling added a comment -

            I hit this issue as well. Setting changelog to false and restarting Jenkins seems to bypass the issue.

            Show
            aaronling Aaron Ling added a comment - I hit this issue as well. Setting changelog to false and restarting Jenkins seems to bypass the issue.
            Hide
            nddeshmukh16 Nandlal Deshmukh added a comment - - edited

            @Edmund Haselwanter ,  Thanks a lot, you saved me ..

             Below workaround worked for me 

            extensions: [[$class: 'DisableRemotePoll']]
            
             def scmVars1 = checkout([def scmVars1 = checkout([ $class : 'GitSCM', branches:  [[name: '*/' + "${BRANCH_PARAM}"]], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'DisableRemotePoll']], submoduleCfg : [], userRemoteConfigs: [[url: 'https://github.com/zbcdeshmukhxys/framework.git']] ])
            
            Show
            nddeshmukh16 Nandlal Deshmukh added a comment - - edited @ Edmund Haselwanter  ,  Thanks a lot, you saved me ..  Below workaround worked for me  extensions: [[$class: 'DisableRemotePoll' ]] def scmVars1 = checkout([def scmVars1 = checkout([ $class : 'GitSCM' , branches:  [[name: '*/' + "${BRANCH_PARAM}" ]], doGenerateSubmoduleConfigurations: false , extensions: [[$class: 'DisableRemotePoll' ]], submoduleCfg : [], userRemoteConfigs: [[url: 'https: //github.com/zbcdeshmukhxys/framework.git' ]] ])
            Hide
            igora Igor Fraa added a comment -

            According to Git plugin source 
            [$class: 'DisableRemotePoll']
            is not related to disabling polling. It is for repositories with 'ls-remote' disabled and acts as an alias for requiresWorkspaceForPolling parameter. 

            the only workaround to disable polling is to set 

            [$class: 'PathRestriction', excludedRegions: '', includedRegions: 'no/such/path/exists']
            Show
            igora Igor Fraa added a comment - According to Git plugin source   [$class: 'DisableRemotePoll'] is not related to disabling polling. It is for repositories with 'ls-remote' disabled and acts as an alias for requiresWorkspaceForPolling parameter.  the only workaround to disable polling is to set  [$class: 'PathRestriction' , excludedRegions: '', includedRegions: ' no/such/path/exists']

              People

              • Assignee:
                Unassigned
                Reporter:
                jwillemsen Johnny Willemsen
              • Votes:
                16 Vote for this issue
                Watchers:
                25 Start watching this issue

                Dates

                • Created:
                  Updated: