Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56037

SCM Polling with p4-plugin triggers builds several times

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Fixed
    • Component/s: p4-plugin
    • Labels:
    • Environment:
      Latest Jenkins version, latest p4-plugin version, scripted pipeline jobs
    • Similar Issues:

      Description

      We have 2 jobs with different workspaces that do SCM polling every 2 minutes (with different filters) that could potentially be triggered at roughly the same time (as polling has a random fudge factor). If the longer job (5 minutes) runs first and then the other one will be polling as well, then it could happen that it polls one or two more times before it can actually run, the end result being that once the longer job finishes, the short job runs 3 times in a row, once with the change it should have been building, then two more times with no changes, but the logs will still say it was triggered because it did detect changes. Then as a bonus sometimes even the long job will run again with no changes, but a polling log saying that there were changes detected even though not saying which ones.

      It gets worse still, as in this scenario sometimes it even builds a job when it should not have detected changes due to the filter we supplied and then to make it worse still also build that multiple times with no changes (same changelist number).

      Then last of all, when a job does get triggered due to one out of many changes being filtered as positive, still all changes are reported to the job and not just the change that triggered it. Which is inconvenient for our slack notification code showing changes that were unrelated.

      It is also unclear how to achieve the desired results if we were to switch to perforce triggers instead of polling, as the documentation is vague about the filters applying only to polling or also to p4 triggering.

      We use a Jenkinsfile with scripted pipeline (not declarative) and as Jenkins UI for editing the script is not great and the serialization of the pipeline script to XML seems to be verbatim which seems dangerous for the XML parser to get confused, we have to (and prefer to) store the Jenkinsfile(s) in perforce as well. However the documentation on how polling and p4 workspaces work with "Pipeline script from SCM" is lacking or ambiguous and seems to have been tested and documented with declarative pipeline scripts only.

      The end result is that in addition to jobs getting built when we do not want them to, our slack channel gets more notifications than it should which is disorienting to the team.

        Attachments

          Activity

          Hide
          p4karl Karl Wirth added a comment -

          Hi Jeffrey Exterkate. I have sent you a direct email to discuss the topics above. Please let me know here if you don't receive it.

          Show
          p4karl Karl Wirth added a comment - Hi Jeffrey Exterkate . I have sent you a direct email to discuss the topics above. Please let me know here if you don't receive it.
          Hide
          lystor Mykola Ulianytskyi added a comment -

          We have the same issue.

          Jenkins triggers jobs with enabled polling while they are running even without new commits in perforce.

          • Jenkins: 2.150.3 (official docker image)
          • P4 plugin: 1.9.6
          Show
          lystor Mykola Ulianytskyi added a comment - We have the same issue. Jenkins triggers jobs with enabled polling while they are running even without new commits in perforce. Jenkins: 2.150.3 (official docker image) P4 plugin: 1.9.6
          Hide
          dpn982 Dave Nichols added a comment - - edited

          Same issue here as well.

          Jenkins 2.150.2 (Official Docker Image)

          P4 Plugin: 1.9.6

           

          Seeing SCM polling trigger jobs with no changes in perforce.

          Is there any kind of work-around to this issue?

          Show
          dpn982 Dave Nichols added a comment - - edited Same issue here as well. Jenkins 2.150.2 (Official Docker Image) P4 Plugin: 1.9.6   Seeing SCM polling trigger jobs with no changes in perforce. Is there any kind of work-around to this issue?
          Hide
          lystor Mykola Ulianytskyi added a comment -

          > Is there any kind of work-around to this issue?
           
          https://issues.jenkins-ci.org/browse/JENKINS-56286

          Show
          lystor Mykola Ulianytskyi added a comment - > Is there any kind of work-around to this issue?   https://issues.jenkins-ci.org/browse/JENKINS-56286
          Hide
          p4karl Karl Wirth added a comment -

          Linking this to 'JENKINS-56286' for now. If the fix for that case does not solve it we will look at this one again.

          Show
          p4karl Karl Wirth added a comment - Linking this to ' JENKINS-56286 ' for now. If the fix for that case does not solve it we will look at this one again.
          Hide
          p4karl Karl Wirth added a comment -

          Also linking to this one - JENKINS-56295. Again if that does not solve it either we will come back to this one.

          Show
          p4karl Karl Wirth added a comment - Also linking to this one -  JENKINS-56295 . Again if that does not solve it either we will come back to this one.
          Hide
          p4karl Karl Wirth added a comment - - edited

          Changed my mind and decided to make this the master job.

          Reproduction steps:

          (1) Create a pipeline job.
          (2) Set it to poll every minute (or use the Poll Now Plugin)
          (3) Disable all executors.
          (4) Add a changelist that would trigger the job.
          (5) Wait 1 minute or click 'Poll Now'.
          (6) Wait 1 minute or click 'Poll Now'.
          (7) Wait 1 minute or click 'Poll Now'.
          (8) Wait 1 minute or click 'Poll Now'.
          (9) Enable the executor.
          (10) The job will build the same changelist at least 2 times (note there is something else going on that I don't understand where the number of times it builds can be more than 2 times).
          

          At the top of the console log you see 'Started by an SCM change' for each missed poll. For example:

           
          Started by an SCM change
          Started by an SCM change
          ...
          Started by an SCM change
          Started by an SCM change
          (25 times)
          

          And the 'build.xml' shows:

                    <hudson.triggers.SCMTrigger_-SCMTriggerCause/>
                    <int>25</int>
          

          Passing this occurrence of unexpected retries to development. Note there are other causes I am still investigating.

          Show
          p4karl Karl Wirth added a comment - - edited Changed my mind and decided to make this the master job. Reproduction steps: (1) Create a pipeline job. (2) Set it to poll every minute (or use the Poll Now Plugin) (3) Disable all executors. (4) Add a changelist that would trigger the job. (5) Wait 1 minute or click 'Poll Now' . (6) Wait 1 minute or click 'Poll Now' . (7) Wait 1 minute or click 'Poll Now' . (8) Wait 1 minute or click 'Poll Now' . (9) Enable the executor. (10) The job will build the same changelist at least 2 times (note there is something else going on that I don't understand where the number of times it builds can be more than 2 times). At the top of the console log you see 'Started by an SCM change' for each missed poll. For example: Started by an SCM change Started by an SCM change ... Started by an SCM change Started by an SCM change (25 times) And the 'build.xml' shows: <hudson.triggers.SCMTrigger_-SCMTriggerCause/> < int >25</ int > Passing this occurrence of unexpected retries to development. Note there are other causes I am still investigating.
          Hide
          p4paul Paul Allen added a comment -

          Karl Wirth - I think this issue is now fixed in release 1.10.5

          Show
          p4paul Paul Allen added a comment - Karl Wirth  - I think this issue is now fixed in release 1.10.5

            People

            • Assignee:
              p4paul Paul Allen
              Reporter:
              logician Jeffrey Exterkate
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: