Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28452

Push event web hook triggers all jobs regardless of Branch Specifier

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Fixed
    • Component/s: gitlab-hook-plugin
    • Labels:
      None
    • Environment:
      AWS Linux AMI, Version 2015.03
    • Similar Issues:

      Description

      This started happening since 1.4.0.

      I have had to downgrade to 1.3.1 as this is a blocker.
      We have 2 jobs for each repository.
      The 2 jobs have the following Branch Specifiers:

      • origin/feature/*
      • origin/master

      Jenkins 1.613 is running on Amazon Linux Image. This issue seems related to the plugin itself rather than Jenkins, but please let me know if you need any more details if you cannot replicate.

      Thanks,
      Michael

        Attachments

          Activity

          mg_zpg Michael Giuliano created issue -
          Hide
          javiplx Javier Palacios added a comment -

          Could you supply a piece of log for one of those unwated builds? Piece from 'INFO: gitlab web hook triggered for' until the build success/failure if possible.

          Show
          javiplx Javier Palacios added a comment - Could you supply a piece of log for one of those unwated builds? Piece from 'INFO: gitlab web hook triggered for' until the build success/failure if possible.
          Hide
          mg_zpg Michael Giuliano added a comment -

          Below is the Jenkins log for the push event from GitLab.
          Push is done on master, which should only match the 'test-build' project, not the 'test-branch'

          INFO: gitlab web hook triggered for
             - repo url: git@gitlab.zoopla.co.uk:mgiuliano/test-jenkins.git
             - branch: master
             - with payload:
          {
            "object_kind": "push",
            "before": "658871fc969682b05a8f8f896b3d9601c9ddb32b",
            "after": "163214aa43fc118d958a26b697128bfbadd9ae4b",
            "ref": "refs/heads/master",
            "checkout_sha": "163214aa43fc118d958a26b697128bfbadd9ae4b",
            "message": null,
            "user_id": 2,
            "user_name": "Michael Giuliano",
            "user_email": "michael.giuliano@zpg.co.uk",
            "project_id": 117,
            "repository": {
              "name": "test-jenkins",
              "url": "git@gitlab.zoopla.co.uk:mgiuliano/test-jenkins.git",
              "description": "",
              "homepage": "https://gitlab.zoopla.co.uk/mgiuliano/test-jenkins",
              "git_http_url": "https://gitlab.zoopla.co.uk/mgiuliano/test-jenkins.git",
              "git_ssh_url": "git@gitlab.zoopla.co.uk:mgiuliano/test-jenkins.git",
              "visibility_level": 0
            },
            "commits": [
              {
                "id": "163214aa43fc118d958a26b697128bfbadd9ae4b",
                "message": "Bump version: 1.0.1 → 1.0.2\n",
                "timestamp": "2015-05-18T11:12:39+01:00",
                "url": "https://gitlab.zoopla.co.uk/mgiuliano/test-jenkins/commit/163214aa43fc118d958a26b697128bfbadd9ae4b",
                "author": {
                  "name": "Michael Giuliano",
                  "email": "michael.giuliano@zpg.co.uk"
                }
              }
            ],
            "total_commits_count": 1
          }
          INFO: matching projects:
             - test-branch
             - test-build
          INFO: test-build scheduled for build
          INFO: test-build #4 main build action completed: SUCCESS
          

          The build itself does nothing but echo "Hello" in a shell script.

          I hope this helps.

          Thanks,
          Michael

          Show
          mg_zpg Michael Giuliano added a comment - Below is the Jenkins log for the push event from GitLab. Push is done on master, which should only match the 'test-build' project, not the 'test-branch' INFO: gitlab web hook triggered for - repo url: git@gitlab.zoopla.co.uk:mgiuliano/test-jenkins.git - branch: master - with payload: { "object_kind" : "push" , "before" : "658871fc969682b05a8f8f896b3d9601c9ddb32b" , "after" : "163214aa43fc118d958a26b697128bfbadd9ae4b" , "ref" : "refs/heads/master" , "checkout_sha" : "163214aa43fc118d958a26b697128bfbadd9ae4b" , "message" : null , "user_id" : 2, "user_name" : "Michael Giuliano" , "user_email" : "michael.giuliano@zpg.co.uk" , "project_id" : 117, "repository" : { "name" : "test-jenkins" , "url" : "git@gitlab.zoopla.co.uk:mgiuliano/test-jenkins.git" , "description" : "", "homepage" : "https: //gitlab.zoopla.co.uk/mgiuliano/test-jenkins" , "git_http_url" : "https: //gitlab.zoopla.co.uk/mgiuliano/test-jenkins.git" , "git_ssh_url" : "git@gitlab.zoopla.co.uk:mgiuliano/test-jenkins.git" , "visibility_level" : 0 }, "commits" : [ { "id" : "163214aa43fc118d958a26b697128bfbadd9ae4b" , "message" : "Bump version: 1.0.1 → 1.0.2\n" , "timestamp" : "2015-05-18T11:12:39+01:00" , "url" : "https: //gitlab.zoopla.co.uk/mgiuliano/test-jenkins/commit/163214aa43fc118d958a26b697128bfbadd9ae4b" , "author" : { "name" : "Michael Giuliano" , "email" : "michael.giuliano@zpg.co.uk" } } ], "total_commits_count" : 1 } INFO: matching projects: - test-branch - test-build INFO: test-build scheduled for build INFO: test-build #4 main build action completed: SUCCESS The build itself does nothing but echo "Hello" in a shell script. I hope this helps. Thanks, Michael
          Hide
          javiplx Javier Palacios added a comment -

          Is there an "INFO: test-branch scheduled for build" message anywhere in the logs? From what you have pasted looks like a misleading text: when it says "INFO: matching projects", the long meaning is "projects tracking git@gitlab.zoopla.co.uk:mgiuliano/test-jenkins.git". The 'schedule for build' message is the trace that the build has already been triggered, and I see it only for master job.

          The reason behind is that to make cleaner the implementation of some features (mainly, template based creation), the project matching is done in two phases, one for overall matching (enabled, git & git-url), and a fine-grained round that actually checks for the tracked branch. So, in your case, both projects must succeed on the first round, but only one will actually run after looking for the second match.

          Show
          javiplx Javier Palacios added a comment - Is there an "INFO: test-branch scheduled for build" message anywhere in the logs? From what you have pasted looks like a misleading text: when it says "INFO: matching projects", the long meaning is "projects tracking git@gitlab.zoopla.co.uk:mgiuliano/test-jenkins.git". The 'schedule for build' message is the trace that the build has already been triggered, and I see it only for master job. The reason behind is that to make cleaner the implementation of some features (mainly, template based creation), the project matching is done in two phases, one for overall matching (enabled, git & git-url), and a fine-grained round that actually checks for the tracked branch. So, in your case, both projects must succeed on the first round, but only one will actually run after looking for the second match.
          Hide
          mg_zpg Michael Giuliano added a comment -

          Hi, sorry this was the logs after I downgraded.

          The ones using 1.4.0 read:

          INFO: matching projects:
             - test-branch
             - test-build
          INFO: project test-branch merge target matches master
          INFO: test-branch scheduled for build
          test-build scheduled for build
          INFO: test-build #3 main build action completed: SUCCESS
          
          Show
          mg_zpg Michael Giuliano added a comment - Hi, sorry this was the logs after I downgraded. The ones using 1.4.0 read: INFO: matching projects: - test-branch - test-build INFO: project test-branch merge target matches master INFO: test-branch scheduled for build test-build scheduled for build INFO: test-build #3 main build action completed: SUCCESS
          Hide
          javiplx Javier Palacios added a comment -

          There is the point:

          INFO: project test-branch merge target matches master
          

          The fine grained match I comment before does also checks for branches merged in the job, not only the configured refspec. The push to master triggers the test-build as expected, but also the test-branch jobs because master is merged into them. This is actually a non-configurable behaviour, as I did consider it highly desirable. Is that behaviour actually a not-feature in your case? And did you want to use any of the (other) new features on 1.4.0? It could be a good point to add to future releases, but it you answer affirmatively to both questions, I'll try to add into a near-future release.

          Show
          javiplx Javier Palacios added a comment - There is the point: INFO: project test-branch merge target matches master The fine grained match I comment before does also checks for branches merged in the job, not only the configured refspec. The push to master triggers the test-build as expected, but also the test-branch jobs because master is merged into them. This is actually a non-configurable behaviour, as I did consider it highly desirable. Is that behaviour actually a not-feature in your case? And did you want to use any of the (other) new features on 1.4.0? It could be a good point to add to future releases, but it you answer affirmatively to both questions, I'll try to add into a near-future release.
          Hide
          mg_zpg Michael Giuliano added a comment -

          In my scenario, the branch job is a test-run job. The job is triggered when a feature branch is pushed to GitLab. It then attempts a merge into the integration branch (in this case master) to check if it would merge cleanly if the merge request was accepted.

          So this job is a pre-check to tell the feature branch developer that his work won't integrate cleanly, without having to get the reviewer involved yet.

          Only when this branch job passes, can the developer create a merge request and submit it for code review. This allow for less time wasted on the reviewers.

          The reviewer, once happy, merges the branch to master, and deletes the feature branch. This triggers the build job.

          At this stage, when the branch job is triggered again, it fails because the feature branch no longer exists. So indeed, I would have liked to have this behaviour as an option, not as a default. The way it works now is very implicit and not very obvious. It would be better to be able to configure this behaviour.

          I am not using any of the 1.4.0 features specifically, I was upgrading to stay up to date.

          So if this doesn't get patched or added to the next release, I will be stuck to the 1.3 release forever, which might be an issue in the future.

          So yes, if you could add it as soon as you can, it would be greatly appreciated.

          Thanks for your help with this.

          Best,
          Michael

          Show
          mg_zpg Michael Giuliano added a comment - In my scenario, the branch job is a test-run job. The job is triggered when a feature branch is pushed to GitLab. It then attempts a merge into the integration branch (in this case master) to check if it would merge cleanly if the merge request was accepted. So this job is a pre-check to tell the feature branch developer that his work won't integrate cleanly, without having to get the reviewer involved yet. Only when this branch job passes, can the developer create a merge request and submit it for code review. This allow for less time wasted on the reviewers. The reviewer, once happy, merges the branch to master, and deletes the feature branch. This triggers the build job. At this stage, when the branch job is triggered again, it fails because the feature branch no longer exists. So indeed, I would have liked to have this behaviour as an option, not as a default. The way it works now is very implicit and not very obvious. It would be better to be able to configure this behaviour. I am not using any of the 1.4.0 features specifically, I was upgrading to stay up to date. So if this doesn't get patched or added to the next release, I will be stuck to the 1.3 release forever, which might be an issue in the future. So yes, if you could add it as soon as you can, it would be greatly appreciated. Thanks for your help with this. Best, Michael
          Hide
          javiplx Javier Palacios added a comment -

          Although it is not exactly your scenario, there is a feature from 1.4 that you might already one, which is the ability of automatically create jobs for merge requests. In short, if you enable MR webhooks, the plugin is able to process it and create a job to build exactly that merge request, reporting back to gitlab the result.

          Show
          javiplx Javier Palacios added a comment - Although it is not exactly your scenario, there is a feature from 1.4 that you might already one, which is the ability of automatically create jobs for merge requests. In short, if you enable MR webhooks, the plugin is able to process it and create a job to build exactly that merge request, reporting back to gitlab the result.
          Hide
          mg_zpg Michael Giuliano added a comment -

          I looked at this feature to see if it could be useful. But the branch job not only checks whether the feature branch would merge cleanly, it also runs the test suite and sometime a fake build as well. Sometime the clean merge is not enough for a feature branch to be valid.

          Show
          mg_zpg Michael Giuliano added a comment - I looked at this feature to see if it could be useful. But the branch job not only checks whether the feature branch would merge cleanly, it also runs the test suite and sometime a fake build as well. Sometime the clean merge is not enough for a feature branch to be valid.
          Hide
          javiplx Javier Palacios added a comment -

          What the plugin actually does is to "clone" the job that build the MR destination branch. So if you request merging anything into master, whatever your jenkins job for master does will get done with the source branch merged.

          Show
          javiplx Javier Palacios added a comment - What the plugin actually does is to "clone" the job that build the MR destination branch. So if you request merging anything into master, whatever your jenkins job for master does will get done with the source branch merged.
          Hide
          mg_zpg Michael Giuliano added a comment -

          Except that the "clone" is slightly different. The build project will publish the artifacts, ready to be deployed, for successful builds.
          The branch/"clone" one should skip those extra post-build steps.

          Fro what I can gather from the documentation, I do not believe this can be done.
          And if so, I would love to hear how.

          Show
          mg_zpg Michael Giuliano added a comment - Except that the "clone" is slightly different. The build project will publish the artifacts, ready to be deployed, for successful builds. The branch/"clone" one should skip those extra post-build steps. Fro what I can gather from the documentation, I do not believe this can be done. And if so, I would love to hear how.
          Hide
          javiplx Javier Palacios added a comment -

          Yep, you're completely right. I'm working on selective removal of that kind of configuration items (even emailing can be annoying), but that's definitely not a matter of days ...

          Show
          javiplx Javier Palacios added a comment - Yep, you're completely right. I'm working on selective removal of that kind of configuration items (even emailing can be annoying), but that's definitely not a matter of days ...
          Hide
          mg_zpg Michael Giuliano added a comment -

          Heh, I bet! That's fine!
          As long as you're happy adding these features to your roadmap, I'm happy to wait for them.

          Thanks

          Show
          mg_zpg Michael Giuliano added a comment - Heh, I bet! That's fine! As long as you're happy adding these features to your roadmap, I'm happy to wait for them. Thanks
          Hide
          javiplx Javier Palacios added a comment -

          If you want to give another chance, try the latest snapshot. It allows to configure whether triggering is done also for merged branches, and defaults to false, which should cause less surprises.
          I've done a fast test and seems to work, although it is not yet fully merged because I want to add some acceptance test for this.

          Show
          javiplx Javier Palacios added a comment - If you want to give another chance, try the latest snapshot . It allows to configure whether triggering is done also for merged branches, and defaults to false, which should cause less surprises. I've done a fast test and seems to work, although it is not yet fully merged because I want to add some acceptance test for this.
          javiplx Javier Palacios made changes -
          Field Original Value New Value
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          rlagoue rlagoue added a comment -

          Hello

          When do you expect to publish the next release. Since I am waiting this fix for long time now

          Thanks for the great work

          Show
          rlagoue rlagoue added a comment - Hello When do you expect to publish the next release. Since I am waiting this fix for long time now Thanks for the great work
          Hide
          javiplx Javier Palacios added a comment -

          I sould fix write permission on the plugins store, but 1.4.1 will be available in a few days.

          Show
          javiplx Javier Palacios added a comment - I sould fix write permission on the plugins store, but 1.4.1 will be available in a few days.
          javiplx Javier Palacios made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 163292 ] JNJira + In-Review [ 208790 ]

            People

            • Assignee:
              javiplx Javier Palacios
              Reporter:
              mg_zpg Michael Giuliano
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: