Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-31103

How to troubleshoot maximum number of statuses errors?

    Details

    • Similar Issues:

      Description

      We are using Jenkins in conjunction with GitHub. Sometimes the tests are failing with the following exception:

      Sep 10, 2015 7:19:35 AM hudson.model.AbstractBuild$AbstractBuildExecution reportError
      WARNING: Publisher 'Set build status on GitHub commit' aborted due to exception: 
      java.io.IOException: {"message":"Validation Failed","errors":[{"resource":"Status","code":"custom","message":"This SHA and context has reached the maximum number of statuses."}],"documentation_url":"https://developer.github.com/v3/repos/statuses/#create-a-status"}
              at org.kohsuke.github.Requester.handleApiError(Requester.java:493)
              at org.kohsuke.github.Requester._to(Requester.java:245)
              at org.kohsuke.github.Requester.to(Requester.java:191)
              at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:774)
              at com.cloudbees.jenkins.GitHubCommitNotifier.updateCommitStatus(GitHubCommitNotifier.java:127)
              at com.cloudbees.jenkins.GitHubCommitNotifier.perform(GitHubCommitNotifier.java:84)
              at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
      

      The error message seems to report the error details sent back by the GitHub RPC endpoint, stating that Jenkins tried to update more than 1000 times the statuses associated to a given SHA or “context”. Since we have a hand of different tests, it is strange to hit that 1000 limit.

      How to more precisely diagnose this (non-fatal) error and figure out what is causing that limit to be hit?

        Attachments

          Activity

          Hide
          lanwen Kirill Merkushev added a comment -

          How many server-configs do you have in your global setup?

          I see a cycle in a code:

           for (GitHubRepositoryName name : GitHubRepositoryNameContributor.parseAssociatedNames(build.getProject())) 
          

          which means that status will be updated with each config. (Originally it was done to try to find creds with repo:status scope).

          So maybe its a time to change this behaviour

          Show
          lanwen Kirill Merkushev added a comment - How many server-configs do you have in your global setup? I see a cycle in a code: for (GitHubRepositoryName name : GitHubRepositoryNameContributor.parseAssociatedNames(build.getProject())) which means that status will be updated with each config. (Originally it was done to try to find creds with repo:status scope). So maybe its a time to change this behaviour
          Hide
          integer Kanstantsin Shautsou added a comment -

          I wish some performance tests. Traffic can be dumped and measured somehow.

          Show
          integer Kanstantsin Shautsou added a comment - I wish some performance tests. Traffic can be dumped and measured somehow.
          Hide
          michipili Michael Grünewald added a comment - - edited

          Kirill Merkushev Thank you for your quick answer! Our Jenkins setup has just one *server-config* (I guess you mean the setting titled Servers configs with credentials to manage GitHub integrations) so reaching the limit is very surprising.

          Kanstantsin Shautsou Which kind of performance test would like to have? If you think of a precise measurement which could be useful to troubleshoot that issue, I can probably arrange this.

          BTW We just updated Jenkins and its plugins to the latest versions, let's see if this improves the situation.

          Show
          michipili Michael Grünewald added a comment - - edited Kirill Merkushev Thank you for your quick answer! Our Jenkins setup has just one * server-config * (I guess you mean the setting titled Servers configs with credentials to manage GitHub integrations ) so reaching the limit is very surprising. Kanstantsin Shautsou Which kind of performance test would like to have? If you think of a precise measurement which could be useful to troubleshoot that issue, I can probably arrange this. BTW We just updated Jenkins and its plugins to the latest versions, let's see if this improves the situation.
          Hide
          lanwen Kirill Merkushev added a comment - - edited

          another question - does this sha really have 1k statuses?
          Which context/messages is used for them (You can view it using curl + github api + token)? Maybe something else updating statuses in background?

          Show
          lanwen Kirill Merkushev added a comment - - edited another question - does this sha really have 1k statuses? Which context/messages is used for them (You can view it using curl + github api + token)? Maybe something else updating statuses in background?
          Hide
          lanwen Kirill Merkushev added a comment -

          Closed due to no response from user

          Show
          lanwen Kirill Merkushev added a comment - Closed due to no response from user

            People

            • Assignee:
              lanwen Kirill Merkushev
              Reporter:
              michipili Michael Grünewald
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: