Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34727

WebHook events are not always successfully triggering Jenkins pipeline

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Labels:
      None
    • Environment:
      core:1.642.4.2
      github:1.17.1
      github-api:1.75
      github-branch-source:1.6
      branch-api:1.7
    • Similar Issues:

      Description

      When committing multiple changes successively to the same branch and repo, some builds are never triggered.

      How to reproduce this (Assuming you have a GitHub Organization folder with WebHooks already setup):

      • Commit multiple changes successively with a minimum interval (few seconds). Usually 5 are enough.
      • You should see that a build is trigger containing the changes until a certain revision/sha, but not the last one.

      I have tried many times to do this and I believe this happen when changes are pushed while the Branch Indexing is running. Hence it seems easier to hit that issue when you have lots of branches on your repository.

      In my example, I have the following commit in that order:

      1. fb1de9187286a1c815dcda7bf143c18f4953a9e0 (included in the build)
      2. 933fd9547927aa545f48dfab0021ea69458ea6f7 (included in the build)
      3. 06cc4d2cc70a4a27b2004de84d50dbdc3b602bd4 (never built)
      4. ca5fbd5854b629beb6e407c0ed6356e222bcf638 (never built)

      Branch Indexing logs when the commit is detected shows:

      Changes detected in feature/feature1 (7e45d8ffaafb3c16567f82769ab6c2400d98068a → fb1de9187286a1c815dcda7bf143c18f4953a9e0)
      Scheduled build for branch: feature/feature1
      

      Next 3 branch indexing cannot detect the changes and show:

      No changes detected in feature/feature1 (still at fb1de9187286a1c815dcda7bf143c18f4953a9e0)
      

      This is where the revision check fails: https://github.com/jenkinsci/branch-api-plugin/blob/master/src/main/java/jenkins/branch/MultiBranchProject.java#L312

      I attached relevant logs (jenkins.log, branch-source.log, github-webhook.log and folder-queue-executor.log).

        Attachments

        1. branch-source.log
          1 kB
        2. folder-queue-executor-logs.log
          36 kB
        3. github-webhook.log
          2 kB
        4. jenkins.log
          2 kB

          Issue Links

            Activity

            Hide
            amuniz Antonio Muñiz added a comment -

            There is another level of cache at client side. PR updated disabling the client side cache also.

            Show
            amuniz Antonio Muñiz added a comment - There is another level of cache at client side. PR updated disabling the client side cache also.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Antonio
            Path:
            src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java
            src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubWebhookListenerImpl.java
            src/main/java/org/jenkinsci/plugins/github_branch_source/PullRequestGHEventSubscriber.java
            http://jenkins-ci.org/commit/github-branch-source-plugin/096eb3867c90fbfcd4066720918ed8e0f0b5da91
            Log:
            JENKINS-34727 Delaying branch indexing (#54)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Antonio Path: src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubWebhookListenerImpl.java src/main/java/org/jenkinsci/plugins/github_branch_source/PullRequestGHEventSubscriber.java http://jenkins-ci.org/commit/github-branch-source-plugin/096eb3867c90fbfcd4066720918ed8e0f0b5da91 Log: JENKINS-34727 Delaying branch indexing (#54) JENKINS-34727 Delaying branch indexing JENKINS-34727 Disabling GH API client cache JENKINS-34727 Reduce indexing delay to 5 seconds
            Hide
            amuniz Antonio Muñiz added a comment -

            Merged in master. Will be released soon.

            Show
            amuniz Antonio Muñiz added a comment - Merged in master. Will be released soon.
            Hide
            recena Manuel Recena Soto added a comment - - edited

            Released as part of 1.7

            Show
            recena Manuel Recena Soto added a comment - - edited Released as part of 1.7
            Hide
            dawg Martin Nowak added a comment -

            Solving the trust issues is easy, as GH webhooks can be signed with a shared secret. So you just need to enforce that a secret is used, Jenkins already runs verification IIRC.

            Show
            dawg Martin Nowak added a comment - Solving the trust issues is easy, as GH webhooks can be signed with a shared secret. So you just need to enforce that a secret is used, Jenkins already runs verification IIRC.

              People

              • Assignee:
                amuniz Antonio Muñiz
                Reporter:
                allan_burdajewicz Allan BURDAJEWICZ
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: