Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61823

"--httpKeepAliveTimeout" no longer has any effect after upgrading to 2.222.1

    Details

    • Similar Issues:
    • Released As:
      Jenkins 2.243

      Description

      After upgrading from Jenkins 2.204.2 to 2.222.1, our builds started failing non-deterministically when trying to download artifacts.
      It seems that Jenkins no longer honors the "--httpKeepAliveTimeout=60000" switch.
      I confirmed that Jenkins is running with this switch.
      Yet a download with a slow consumer will fail with a partial download. (for testing: "curl -f -u $jenkins_auth $artifact_url | (sleep 6; wc -c)" 

      On the server side, the log indicates that the command-line switch was ineffective and the default 5s timeout is still in use:

      java.util.concurrent.TimeoutException: Idle timeout expired: 5000/5000 ms
       at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:171)
       at org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113)
       at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
       at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
       at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
       at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
       at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      Caused: java.io.IOException
       at org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:234)
       at org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:268)
       at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:825)
       at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:612)
       at org.kohsuke.stapler.ResponseImpl.serveFile(ResponseImpl.java:216)
       at hudson.model.DirectoryBrowserSupport.serveFile(DirectoryBrowserSupport.java:370)
       at hudson.model.DirectoryBrowserSupport.generateResponse(DirectoryBrowserSupport.java:155)
       at org.kohsuke.stapler.HttpResponseRenderer$Default.handleHttpResponse(HttpResponseRenderer.java:124)
       at org.kohsuke.stapler.HttpResponseRenderer$Default.generateResponse(HttpResponseRenderer.java:69)
       at org.kohsuke.stapler.Function.renderResponse(Function.java:164)
       at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:147)
       at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:535)
      

      The default 5s timeout is short enough that a simple "curl | tar xf" sometimes trips it when unpacking many small files on an overloaded VM. Please consider changing the default back to jetty's 30s.

        Attachments

          Issue Links

            Activity

            Hide
            grunwald Daniel Grunwald added a comment -

            I downgraded to 2.204.6, and now my downloads are complete again.

            Show
            grunwald Daniel Grunwald added a comment - I downgraded to 2.204.6, and now my downloads are complete again.
            Hide
            danielbeck Daniel Beck added a comment -

            Olivier Lamy Another victim of our Jetty upgrades?

            Show
            danielbeck Daniel Beck added a comment - Olivier Lamy Another victim of our Jetty upgrades?
            Show
            olamy Olivier Lamy added a comment - nope nothing related to Jetty but to this  https://github.com/jenkinsci/winstone/commit/5e1a8982f1412cacbe3bc4d0feb29436918bf8be#r38330848
            Hide
            olamy Olivier Lamy added a comment -

            temporary workaround until is fixed with new version is to use  --KeepAliveTimeout

            but it will be fixed with next version of winstone and this workaround will not work anymore  

            Show
            olamy Olivier Lamy added a comment - temporary workaround until is fixed with new version is to use  --KeepAliveTimeout but it will be fixed with next version of winstone and this workaround will not work anymore  
            Show
            olamy Olivier Lamy added a comment - pr  https://github.com/jenkinsci/winstone/pull/100  
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            BACKPORTING NOTES:

            • Winstone master branch includes a patch for Winstone redirects by Alex Earlhttps://github.com/jenkinsci/winstone/pull/98 
            • The fix is quite trivial, but we may not want to backport it to 2.222.x due to our experience in the past months
            • I can prepare a release of Winstone 5.9.1 with a backport -> DONE

            https://github.com/jenkinsci/winstone/releases/tag/winstone-5.9.1

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited BACKPORTING NOTES: Winstone master branch includes a patch for Winstone redirects by Alex Earl :  https://github.com/jenkinsci/winstone/pull/98   The fix is quite trivial, but we may not want to backport it to 2.222.x due to our experience in the past months I can prepare a release of Winstone 5.9.1 with a backport -> DONE https://github.com/jenkinsci/winstone/releases/tag/winstone-5.9.1
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            I submitted https://github.com/jenkinsci/jenkins/pull/4811 with an update for the Jenkins weekly.

             

             

            Show
            oleg_nenashev Oleg Nenashev added a comment - I submitted  https://github.com/jenkinsci/jenkins/pull/4811  with an update for the Jenkins weekly.    
            Hide
            oleg_nenashev Oleg Nenashev added a comment -
            Show
            oleg_nenashev Oleg Nenashev added a comment - https://github.com/jenkinsci/jenkins/pull/4812  for the LTS backport

              People

              • Assignee:
                olamy Olivier Lamy
                Reporter:
                grunwald Daniel Grunwald
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: