Details

    • Type: Improvement
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: scm-api-plugin
    • Labels:
      None
    • Environment:
      Jenkins 2.73.3 and whatever the latest GitHub plugin is. :)
    • Similar Issues:

      Description

      We've got several repositories set up with GitHub webhooks to let the Jenkins server know when new PRs or commits happen. Generally, this works great.

      But sometimes, Jenkins fails to handle the web hook properly due to, we guess, temporary network or service outages. In those cases, Jenkins just forgets about the webhook notification. Later, we Jenkins admins get complaints from our users (our developers) asking why Jenkins didn't build such-and-such push/PR.

      It would be nice if Jenkins (and/or the GitHub plugin?) would keep a queue of webhook notifications and retry them a few (configurable #?) times before giving up.

      Here's a stack trace from an instance of this, which seemed to be due to a random socket timeout. Webhooks resumed working correctly after this w/o intervention. Though, of course, the PR in question wasn't processed until we manually intervened.

       

      [Tue Oct 10 11:02:59 CDT 2017] Push event to branch lower in repository wpengine/[cut: REPO_NAME] CREATED event from 192.30.252.41 ⇒ https://jenkins.wpengine.io:8080/github-webhook/ with timestamp Tue Oct 10 11:02:53 CDT 2017 processed in 1.1 sec
      [Tue Oct 10 11:04:06 CDT 2017] Received Pull request #16 opened in repository wpengine/[cut: REPO_NAME] CREATED event from 192.30.252.41 ⇒ https://jenkins.wpengine.io:8080/github-webhook/ with timestamp Tue Oct 10 11:04:01 CDT 2017
      11:04:07 Connecting to https://api.github.com using [cut: username]/****** (personal access token for [cut: username] github user, is used to scan our repos for jenkinsfiles)
      Looking up wpengine/[cut: REPO_NAME]
      Getting remote pull requests...
      Checking pull request #16
      (not from a trusted source)
      ERROR: Server returned HTTP response code: -1, message: 'null' for URL: https://api.github.com/users/[cut: username2]
      org.kohsuke.github.HttpException: Server returned HTTP response code: -1, message: 'null' for URL: https://api.github.com/users/[cut: username2]
      at org.kohsuke.github.Requester.parse(Requester.java:612)
      at org.kohsuke.github.Requester._to(Requester.java:262)
      at org.kohsuke.github.Requester.to(Requester.java:231)
      at org.kohsuke.github.GHPerson.populate(GHPerson.java:47)
      at org.kohsuke.github.GHPerson.getName(GHPerson.java:184)
      at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.doRetrieve(GitHubSCMSource.java:672)
      at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:462)
      at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300)
      at jenkins.scm.api.SCMSource.fetch(SCMSource.java:254)
      at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.processHeadCreate(MultiBranchProject.java:1283)
      at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.onSCMHeadEvent(MultiBranchProject.java:1125)
      at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:247)
      at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:230)
      at jenkins.scm.api.SCMEvent$Dispatcher.run(SCMEvent.java:481)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.net.SocketTimeoutException: timeout
      at okio.Okio$3.newTimeoutException(Okio.java:207)
      at okio.AsyncTimeout.exit(AsyncTimeout.java:261)
      at okio.AsyncTimeout$2.read(AsyncTimeout.java:215)
      at okio.RealBufferedSource.indexOf(RealBufferedSource.java:306)
      at okio.RealBufferedSource.indexOf(RealBufferedSource.java:300)
      at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:196)
      at com.squareup.okhttp.internal.http.Http1xStream.readResponse(Http1xStream.java:186)
      at com.squareup.okhttp.internal.http.Http1xStream.readResponseHeaders(Http1xStream.java:127)
      at com.squareup.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:737)
      at com.squareup.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:609)
      at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:460)
      at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:399)
      at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:527)
      at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
      at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:25)
      at org.kohsuke.github.Requester.parse(Requester.java:586)
      ... 20 more
      Caused by: java.net.SocketException: Socket closed
      at java.net.SocketInputStream.read(SocketInputStream.java:191)
      at java.net.SocketInputStream.read(SocketInputStream.java:122)
      at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
      at sun.security.ssl.InputRecord.read(InputRecord.java:480)
      at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:944)
      at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:901)
      at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
      at okio.Okio$2.read(Okio.java:139)
      at okio.AsyncTimeout$2.read(AsyncTimeout.java:211)
      ... 33 more

        Attachments

          Issue Links

            Activity

            There are no comments yet on this issue.

              People

              • Assignee:
                Unassigned
                Reporter:
                fcodyc Cody Casterline
              • Votes:
                1 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: