Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42458

Rate limit reached after plugin update

    Details

    • Similar Issues:

      Description

      We have a team repository with about 300 repositories which is scanned for jenkins files.

      We updated Bitbucket-branch-source-plugin to 2.1.0 (working version 1.9) yesterday and now we hit the rate limit every time.

      This means that repositories that are last in the arbitrarily sorted list is never added. We think that this plugin should know about the rate limit and throttle it self. It should always handle newly updated repositories first.

      The best solution would be if Atlasian created a webhook in the team resource for repository creation. But we need a workaround in the mean time.

      [Fri Mar 03 05:29:24 UTC 2017] Finished organization scan. Scan took 8 min 23 sec
      FATAL: Failed to recompute children of {Removed Job name}
      com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: HTTP request error. Status: 429: Unknown Status Code.
      Rate limit for this resource has been exceeded
      	at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.getRequest(BitbucketCloudApiClient.java:568)
      	at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.getRepository(BitbucketCloudApiClient.java:232)
      	at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.isPrivate(BitbucketCloudApiClient.java:405)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrievePullRequests(BitbucketSCMSource.java:334)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:325)
      	at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300)
      	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:254)
      	at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria.recognizes(MultiBranchProjectFactory.java:263)
      	at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.recognizes(OrganizationFolder.java:1266)
      	at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.complete(OrganizationFolder.java:1281)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator.add(BitbucketSCMNavigator.java:212)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator.visitSources(BitbucketSCMNavigator.java:187)
      	at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:399)
      	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219)
      	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:154)
      	at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:850)
      	at hudson.model.ResourceController.execute(ResourceController.java:98)
      	at hudson.model.Executor.run(Executor.java:404)
      Finished: FAILURE
      

        Attachments

          Issue Links

            Activity

            Hide
            lordmatanza Melvyn de Kort added a comment -

            I have created the pull request and created a link from this issue.

            Github complains that the issue cannot be merged, but that is because my commit was based on an earlier commit from master, not the current head.

            In case you would consider merging the change I would be happy to modify my pull request so that it is based on the current head.

            Show
            lordmatanza Melvyn de Kort added a comment - I have created the pull request and created a link from this issue. Github complains that the issue cannot be merged, but that is because my commit was based on an earlier commit from master, not the current head. In case you would consider merging the change I would be happy to modify my pull request so that it is based on the current head.
            Hide
            stephenconnolly Stephen Connolly added a comment -

            Melvyn de Kort I disagree with the hacky solution you proposed in your PR.

            I do not currently have time to implement a rate limiting solution, but if you want to take a stab at it, you are more than welcome to try.

            Should probably be easy to localize within the Bitbucket Cloud client classes... I'm thinking something like:

            Show
            stephenconnolly Stephen Connolly added a comment - Melvyn de Kort I disagree with the hacky solution you proposed in your PR. I do not currently have time to implement a rate limiting solution, but if you want to take a stab at it, you are more than welcome to try. Should probably be easy to localize within the Bitbucket Cloud client classes... I'm thinking something like: Add fields to BitbucketCloudApiClien  that record the last rate limit header from the last response Modify the BitbucketCloudApiClient.getRequest(...)  and similar methods to both populate the last response rate limit header fields as well as sleep when the rate limit is over-target. (if you want a solid if probably over-engineered sleep algorithm, see Connector.java#L454-L540 )
            Hide
            lordmatanza Melvyn de Kort added a comment - - edited

            Stephen Connolly I agree with you on the hacky solution. I immediately stated that this fix would probably not be accepted.

            I would like to implement the solution you've suggested, but my time is currently limited as well. And since our current hacky solution works, it's difficult for me to spend time on the issue during work hours, even though our fix is far from a wanted solution.

            Maybe in the upcoming weeks I might get some time to implement your suggestion, but I can't make any promises.

            Let's update this issue when one of us (or somebody else) starts to work on this.

            Show
            lordmatanza Melvyn de Kort added a comment - - edited Stephen Connolly I agree with you on the hacky solution. I immediately stated that this fix would probably not be accepted. I would like to implement the solution you've suggested, but my time is currently limited as well. And since our current hacky solution works, it's difficult for me to spend time on the issue during work hours, even though our fix is far from a wanted solution. Maybe in the upcoming weeks I might get some time to implement your suggestion, but I can't make any promises. Let's update this issue when one of us (or somebody else) starts to work on this.
            Hide
            lordmatanza Melvyn de Kort added a comment -

            It seems that Bitbucket cloud does not have support to see what the current state of your rate limits are.

            This is the page that explains the current limits, which is very minimal information.

            I've found a developer support question here where an Atlassian team member explains that there currently is no way to predict when rate limiting will occur.

            Therefore I've created a simple fix in the BitbucketCloudApiClient.getRequest(...) which does a sleep and a retry when status code 429 is returned.

            I've created a new pull request and linked it to this issue.

            Show
            lordmatanza Melvyn de Kort added a comment - It seems that Bitbucket cloud does not have support to see what the current state of your rate limits are. This is the page that explains the current limits, which is very minimal information. I've found a developer support question here  where an Atlassian team member explains that there currently is no way to predict when rate limiting will occur. Therefore I've created a simple fix in the BitbucketCloudApiClient.getRequest(...)  which does a sleep and a retry when status code 429  is returned. I've created a new pull request and linked it to this issue.
            Hide
            lordmatanza Melvyn de Kort added a comment -

            The PR was accepted, this issue can be resolved/closed.

            Show
            lordmatanza Melvyn de Kort added a comment - The PR was accepted, this issue can be resolved/closed.

              People

              • Assignee:
                amuniz Antonio Muñiz
                Reporter:
                nossnevs Mikael Svensson
              • Votes:
                7 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: