Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15160

earliest git hash passed by parameterized build trigger when downstream job was triggered multiple times

    Details

    • Similar Issues:

      Description

      Job A triggers job B and passes its git hash along. Job A is fast but job B is slow. So there are times when job A finishes and triggers job B while job B is still running and processing the earlier git hash. When job B finishes its current run, the next run would be triggered by multiple runs of job A. The parameterized trigger passes the git hash from the earliest trigger by job A. As a result, git hashes from the later triggers are never processed by job B. The parameterized trigger should pass the git hash from the latest trigger by job A.

        Attachments

          Activity

          Hide
          cjo9900 cjo9900 added a comment -

          Which version of jenkins/ parameterized trigger are you using?

          How is Job A configured?
          Is the parameterized trigger done as a post build trigger?
          Are the git hashes parameters passed using "Pass-Through Git commit that was built" Parameters?

          If the answers to above are both yes, then the issue seems to be that the action RevisionParameterAction[1] that the git-plugin is providing via the GitRevisionBuildParameters[2] class does not support the QueueAction[3] interface that would allow the Queue class to add separate builds for the different revisions.

          Alternatively the issue is that the RevisionParameterAction[1] does not implement the FoldableAction[4] interface, that would allow the multiple revisions to be merged together so only the latest revision is passed to the next build, as shown in the Queue Class[5].

          In most use cases the behaviour would want to be trigger the downstream job for every commit Job A built.
          In a smaller number of cases there might be the need for the downstream job to run only for the latest commit in Job A, (not sure how that would work if Job A worked on multiple branches).

          [1] https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/RevisionParameterAction.java
          [2] https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
          [3] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/Queue.java#L1467
          [4] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/queue/FoldableAction.java
          [5] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/Queue.java#L526

          Show
          cjo9900 cjo9900 added a comment - Which version of jenkins/ parameterized trigger are you using? How is Job A configured? Is the parameterized trigger done as a post build trigger? Are the git hashes parameters passed using "Pass-Through Git commit that was built" Parameters? If the answers to above are both yes, then the issue seems to be that the action RevisionParameterAction [1] that the git-plugin is providing via the GitRevisionBuildParameters [2] class does not support the QueueAction [3] interface that would allow the Queue class to add separate builds for the different revisions. Alternatively the issue is that the RevisionParameterAction [1] does not implement the FoldableAction [4] interface, that would allow the multiple revisions to be merged together so only the latest revision is passed to the next build, as shown in the Queue Class [5] . In most use cases the behaviour would want to be trigger the downstream job for every commit Job A built. In a smaller number of cases there might be the need for the downstream job to run only for the latest commit in Job A, (not sure how that would work if Job A worked on multiple branches). [1] https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/RevisionParameterAction.java [2] https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java [3] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/Queue.java#L1467 [4] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/queue/FoldableAction.java [5] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/Queue.java#L526
          Hide
          hlau hlau added a comment -

          I have experienced this issue in various versions of jenkins and parameterized trigger plugin. I have been upgrading both in the far chance that it might have been fixed from work on other bugs for the plugin. I am now using the latest versions: 1.486 for jenkins and 3.16 for parameterized trigger plugin and I'm still seeing the problem.

          Indeed Job A triggers job B through the post build parameterized trigger plugin using Pass-through Git commit that was built. This is one example of the problem that I saw as recent as in the last hour:

          Job B was triggered by both #414 and #415 of job A:

          Started by upstream project A build number 414
          originally caused by:

          Started by remote host 127.0.0.1
          Started by upstream project A build number 415
          originally caused by:

          Started by remote host 127.0.0.1
          Revision: 495765555ce4db911825ea0b40bca6278669ae72

          Note that the git commit hash of 4957 was actually from #414 of job A:

          Build #414 (Oct 22, 2012 12:28:07 PM)
          ...
          Started by remote host 127.0.0.1

          Revision: 495765555ce4db911825ea0b40bca6278669ae72

          The git commit hash of 4c41 from #415 was not what the triggered job B is building ... in fact, this commit never will get built if no more pushes come in:

          Build #415 (Oct 22, 2012 1:04:17 PM)
          ...
          Revision: 4c41dd60bdff7557133ebfe5229d7f9d06722cef

          As a result, the developers (justifiably) complain that the latest build artifacts often do not reflect the latest push. Since the continuous integration system is broken, I have to continually monitor the jobs and manually kick them when necessary.

          Show
          hlau hlau added a comment - I have experienced this issue in various versions of jenkins and parameterized trigger plugin. I have been upgrading both in the far chance that it might have been fixed from work on other bugs for the plugin. I am now using the latest versions: 1.486 for jenkins and 3.16 for parameterized trigger plugin and I'm still seeing the problem. Indeed Job A triggers job B through the post build parameterized trigger plugin using Pass-through Git commit that was built. This is one example of the problem that I saw as recent as in the last hour: Job B was triggered by both #414 and #415 of job A: Started by upstream project A build number 414 originally caused by: Started by remote host 127.0.0.1 Started by upstream project A build number 415 originally caused by: Started by remote host 127.0.0.1 Revision: 495765555ce4db911825ea0b40bca6278669ae72 Note that the git commit hash of 4957 was actually from #414 of job A: Build #414 (Oct 22, 2012 12:28:07 PM) ... Started by remote host 127.0.0.1 Revision: 495765555ce4db911825ea0b40bca6278669ae72 The git commit hash of 4c41 from #415 was not what the triggered job B is building ... in fact, this commit never will get built if no more pushes come in: Build #415 (Oct 22, 2012 1:04:17 PM) ... Revision: 4c41dd60bdff7557133ebfe5229d7f9d06722cef As a result, the developers (justifiably) complain that the latest build artifacts often do not reflect the latest push. Since the continuous integration system is broken, I have to continually monitor the jobs and manually kick them when necessary.
          Hide
          hlau hlau added a comment -

          Ideally, there would be a checkbox for the user to indicate whether s/he wants every commit built, or just the latest commit built. In my case, job B runs many times slower than job A. If every commit is built, it would take days for job B to catch up. That would mean I would need to manually kill some runs in job B to help it catching up. constant manual intervention is very undesirable.

          Show
          hlau hlau added a comment - Ideally, there would be a checkbox for the user to indicate whether s/he wants every commit built, or just the latest commit built. In my case, job B runs many times slower than job A. If every commit is built, it would take days for job B to catch up. That would mean I would need to manually kill some runs in job B to help it catching up. constant manual intervention is very undesirable.
          Hide
          cjo9900 cjo9900 added a comment -

          Pull request created to solve the issue

          https://github.com/jenkinsci/git-plugin/pull/107

          Includes the option to combine commits, if required.
          The default is to create separate builds for each build triggered with a git hash.

          Show
          cjo9900 cjo9900 added a comment - Pull request created to solve the issue https://github.com/jenkinsci/git-plugin/pull/107 Includes the option to combine commits, if required. The default is to create separate builds for each build triggered with a git hash.
          Hide
          hlau hlau added a comment -

          Thanks. Will this also solve the problem that only the git commit from the earliest trigger in a multiply triggered downstream job gets processed? In my example above, commit 4c41 from #415 never gets built (until I manually kick the job).

          Show
          hlau hlau added a comment - Thanks. Will this also solve the problem that only the git commit from the earliest trigger in a multiply triggered downstream job gets processed? In my example above, commit 4c41 from #415 never gets built (until I manually kick the job).
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: cjo9900
          Path:
          src/main/java/hudson/plugins/git/RevisionParameterAction.java
          src/test/java/hudson/plugins/git/RevisionParameterActionTest.java
          http://jenkins-ci.org/commit/git-plugin/0d21dcc3cbfce9e14b117560e11703a267cc0f1f
          Log:
          [FIXED JENKINS-15160] Earliest git hash passed by parameter trigger.

          Correction is to create a seperate build for each git hash rather than
          allowing them to be combined into a single build.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: cjo9900 Path: src/main/java/hudson/plugins/git/RevisionParameterAction.java src/test/java/hudson/plugins/git/RevisionParameterActionTest.java http://jenkins-ci.org/commit/git-plugin/0d21dcc3cbfce9e14b117560e11703a267cc0f1f Log: [FIXED JENKINS-15160] Earliest git hash passed by parameter trigger. Correction is to create a seperate build for each git hash rather than allowing them to be combined into a single build.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: cjo9900
          Path:
          src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
          src/main/java/hudson/plugins/git/RevisionParameterAction.java
          src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/config.jelly
          src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/help-combineQueuedCommits.html
          src/test/java/hudson/plugins/git/RevisionParameterActionTest.java
          http://jenkins-ci.org/commit/git-plugin/e442a7904904256ff9aa79e885c085cacd514ede
          Log:
          JENKINS-15160 add support for combining git hashs when passed downstream

          Don't combine with manually started builds

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: cjo9900 Path: src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java src/main/java/hudson/plugins/git/RevisionParameterAction.java src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/config.jelly src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/help-combineQueuedCommits.html src/test/java/hudson/plugins/git/RevisionParameterActionTest.java http://jenkins-ci.org/commit/git-plugin/e442a7904904256ff9aa79e885c085cacd514ede Log: JENKINS-15160 add support for combining git hashs when passed downstream Don't combine with manually started builds
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Nicolas De loof
          Path:
          src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
          src/main/java/hudson/plugins/git/RevisionParameterAction.java
          src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/config.jelly
          src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/help-combineQueuedCommits.html
          src/test/java/hudson/plugins/git/RevisionParameterActionTest.java
          http://jenkins-ci.org/commit/git-plugin/17132ba7e8b2323cdb5b09ee363afbe9c3042b65
          Log:
          Merge pull request #107 from cjo9900/JENKINS-15160

          Jenkins 15160

          Compare: https://github.com/jenkinsci/git-plugin/compare/513d89b4a8a9...17132ba7e8b2

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De loof Path: src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java src/main/java/hudson/plugins/git/RevisionParameterAction.java src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/config.jelly src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/help-combineQueuedCommits.html src/test/java/hudson/plugins/git/RevisionParameterActionTest.java http://jenkins-ci.org/commit/git-plugin/17132ba7e8b2323cdb5b09ee363afbe9c3042b65 Log: Merge pull request #107 from cjo9900/ JENKINS-15160 Jenkins 15160 Compare: https://github.com/jenkinsci/git-plugin/compare/513d89b4a8a9...17132ba7e8b2
          Hide
          hlau hlau added a comment -

          Many thanks. Fix Version/s for this bug says None. Which version contains the fix?

          Show
          hlau hlau added a comment - Many thanks. Fix Version/s for this bug says None. Which version contains the fix?
          Hide
          inger Gergely Nagy added a comment - - edited

          Should combineQueuedCommits=true result in a single long downstream build (even if its upstream ran multiple times)? If so, it does not appear to work for me.

          My simple test with Jenkins 1.480.2 (LTE), Git plugin 1.1.26, Paramtrigger plugin: 2.16:

          1) new job: try-combinecommits-downstream
          scm: a tiny git repo
          build step: shell "sleep 120"

          2) new job: try-combinecommits-base,
          scm: the same repo, trigger by push notification
          build step: shell "sleep 10"
          trigger parametrized build: try-combinecommits-downstream,
          param: Git SHA1
          combineQueuedCommits: yes

          3) a simple bash loop to pushed dummy changes dozens of times

          Result: the short upstream job triggers the downsteam one for each build, flooding the queue - no sign of "combining" them, ie. to pick the last build. I'd expect a second trigger should remove the first one off the queue.

          Do I miss anything? Thanks.

          Show
          inger Gergely Nagy added a comment - - edited Should combineQueuedCommits=true result in a single long downstream build (even if its upstream ran multiple times)? If so, it does not appear to work for me. My simple test with Jenkins 1.480.2 (LTE), Git plugin 1.1.26, Paramtrigger plugin: 2.16: 1) new job: try-combinecommits-downstream scm: a tiny git repo build step: shell "sleep 120" 2) new job: try-combinecommits-base, scm: the same repo, trigger by push notification build step: shell "sleep 10" trigger parametrized build: try-combinecommits-downstream, param: Git SHA1 combineQueuedCommits: yes 3) a simple bash loop to pushed dummy changes dozens of times Result: the short upstream job triggers the downsteam one for each build, flooding the queue - no sign of "combining" them, ie. to pick the last build. I'd expect a second trigger should remove the first one off the queue. Do I miss anything? Thanks.
          Hide
          inger Gergely Nagy added a comment -

          combineQueuedCommits=true does not seem to work => reopening.
          See my previous comment.

          Show
          inger Gergely Nagy added a comment - combineQueuedCommits=true does not seem to work => reopening. See my previous comment.
          Hide
          cjo9900 cjo9900 added a comment -

          Apologies, looks like I missed passing that flag to the Action that is created, so it never took effect.

          https://github.com/jenkinsci/git-plugin/pull/129

          Show
          cjo9900 cjo9900 added a comment - Apologies, looks like I missed passing that flag to the Action that is created, so it never took effect. https://github.com/jenkinsci/git-plugin/pull/129
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: cjo9900
          Path:
          src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
          http://jenkins-ci.org/commit/git-plugin/951517ccd2b99a8a4464b444e03976a1ff958239
          Log:
          [FIXED JENKINS-15160] Pass combineCommits to action

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: cjo9900 Path: src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java http://jenkins-ci.org/commit/git-plugin/951517ccd2b99a8a4464b444e03976a1ff958239 Log: [FIXED JENKINS-15160] Pass combineCommits to action
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Nicolas De loof
          Path:
          src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
          http://jenkins-ci.org/commit/git-plugin/1349eaba9c5767e9bf52ab27b6c31681f0c13733
          Log:
          Merge pull request #129 from cjo9900/JENKINS-15160-2

          [FIXED JENKINS-15160] Pass combineCommits to action

          Compare: https://github.com/jenkinsci/git-plugin/compare/c72b83216954...1349eaba9c57


          You received this message because you are subscribed to the Google Groups "Jenkins Commits" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-commits+unsubscribe@googlegroups.com.
          For more options, visit https://groups.google.com/groups/opt_out.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De loof Path: src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java http://jenkins-ci.org/commit/git-plugin/1349eaba9c5767e9bf52ab27b6c31681f0c13733 Log: Merge pull request #129 from cjo9900/ JENKINS-15160 -2 [FIXED JENKINS-15160] Pass combineCommits to action Compare: https://github.com/jenkinsci/git-plugin/compare/c72b83216954...1349eaba9c57 – You received this message because you are subscribed to the Google Groups "Jenkins Commits" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-commits+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out .
          Hide
          inger Gergely Nagy added a comment - - edited

          Thank you very much for this - it would fix an important regression for me (my long running tests are flooding the queue with the current version - this started to happen with the latest upgrade).
          Any way to release it soon?

          Show
          inger Gergely Nagy added a comment - - edited Thank you very much for this - it would fix an important regression for me (my long running tests are flooding the queue with the current version - this started to happen with the latest upgrade). Any way to release it soon?
          Hide
          hlau hlau added a comment -

          Indeed I'm also still having this problem. Have to manually kill some of the jobs in the long queue. Quite a hassle. Please release the fix soon.

          Show
          hlau hlau added a comment - Indeed I'm also still having this problem. Have to manually kill some of the jobs in the long queue. Quite a hassle. Please release the fix soon.

            People

            • Assignee:
              cjo9900 cjo9900
              Reporter:
              hlau hlau
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: