-
Improvement
-
Resolution: Unresolved
-
Minor
-
None
-
Jenkins 2.73.3 and whatever the latest GitHub plugin is. :)
We've got several repositories set up with GitHub webhooks to let the Jenkins server know when new PRs or commits happen. Generally, this works great.
But sometimes, Jenkins fails to handle the web hook properly due to, we guess, temporary network or service outages. In those cases, Jenkins just forgets about the webhook notification. Later, we Jenkins admins get complaints from our users (our developers) asking why Jenkins didn't build such-and-such push/PR.
It would be nice if Jenkins (and/or the GitHub plugin?) would keep a queue of webhook notifications and retry them a few (configurable #?) times before giving up.
Here's a stack trace from an instance of this, which seemed to be due to a random socket timeout. Webhooks resumed working correctly after this w/o intervention. Though, of course, the PR in question wasn't processed until we manually intervened.
[Tue Oct 10 11:02:59 CDT 2017] Push event to branch lower in repository wpengine/[cut: REPO_NAME] CREATED event from 192.30.252.41 ⇒ https://jenkins.wpengine.io:8080/github-webhook/ with timestamp Tue Oct 10 11:02:53 CDT 2017 processed in 1.1 sec [Tue Oct 10 11:04:06 CDT 2017] Received Pull request #16 opened in repository wpengine/[cut: REPO_NAME] CREATED event from 192.30.252.41 ⇒ https://jenkins.wpengine.io:8080/github-webhook/ with timestamp Tue Oct 10 11:04:01 CDT 2017 11:04:07 Connecting to https://api.github.com using [cut: username]/****** (personal access token for [cut: username] github user, is used to scan our repos for jenkinsfiles) Looking up wpengine/[cut: REPO_NAME] Getting remote pull requests... Checking pull request #16 (not from a trusted source) ERROR: Server returned HTTP response code: -1, message: 'null' for URL: https://api.github.com/users/[cut: username2] org.kohsuke.github.HttpException: Server returned HTTP response code: -1, message: 'null' for URL: https://api.github.com/users/[cut: username2] at org.kohsuke.github.Requester.parse(Requester.java:612) at org.kohsuke.github.Requester._to(Requester.java:262) at org.kohsuke.github.Requester.to(Requester.java:231) at org.kohsuke.github.GHPerson.populate(GHPerson.java:47) at org.kohsuke.github.GHPerson.getName(GHPerson.java:184) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.doRetrieve(GitHubSCMSource.java:672) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:462) at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:254) at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.processHeadCreate(MultiBranchProject.java:1283) at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.onSCMHeadEvent(MultiBranchProject.java:1125) at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:247) at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:230) at jenkins.scm.api.SCMEvent$Dispatcher.run(SCMEvent.java:481) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.SocketTimeoutException: timeout at okio.Okio$3.newTimeoutException(Okio.java:207) at okio.AsyncTimeout.exit(AsyncTimeout.java:261) at okio.AsyncTimeout$2.read(AsyncTimeout.java:215) at okio.RealBufferedSource.indexOf(RealBufferedSource.java:306) at okio.RealBufferedSource.indexOf(RealBufferedSource.java:300) at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:196) at com.squareup.okhttp.internal.http.Http1xStream.readResponse(Http1xStream.java:186) at com.squareup.okhttp.internal.http.Http1xStream.readResponseHeaders(Http1xStream.java:127) at com.squareup.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:737) at com.squareup.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:609) at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:460) at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:399) at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:527) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:586) ... 20 more Caused by: java.net.SocketException: Socket closed at java.net.SocketInputStream.read(SocketInputStream.java:191) at java.net.SocketInputStream.read(SocketInputStream.java:122) at sun.security.ssl.InputRecord.readFully(InputRecord.java:442) at sun.security.ssl.InputRecord.read(InputRecord.java:480) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:944) at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:901) at sun.security.ssl.AppInputStream.read(AppInputStream.java:102) at okio.Okio$2.read(Okio.java:139) at okio.AsyncTimeout$2.read(AsyncTimeout.java:211) ... 33 more
- is related to
-
JENKINS-45142 Appears a timeout isn't being handled properly: "Server returned HTTP response code: -1, message: 'null' for URL"
-
- In Review
-
