Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-5119

Winstone: Memory leak due to default session timeout of -1

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Component/s: core
    • Labels:
      None
    • Environment:
      Hudson 1.337 (Winstone), Solaris 10 SPARC, Sun JDK 1.6.0_16

      Description

      We are running Hudson with its embedded Winstone servlet engine. User authentication within Hudson is done via Hudsons LDAP security realm using our corporate ActiveDirectory via LDAP.

      After running Hudson a few weeks we're running into OutOfMemory exceptions. Here the culprit is a single winstone.WebAppConfiguration instance holding a lot of winstone.WinStoneSession instances each holding an Acegi RememberMeAuthenticationToken instance with 36k data from the authenticated users AD record.

      It seems that Hudson isn't setting Winstones session timeout. Without specifying a session timeout winstone.WebAppConfiguration.makeNewSession() is using a session timeout of -1.

        Issue Links

          Activity

          Hide
          tjuerge tjuerge added a comment -

          This can be fixed by specifying a session timeout in hudson/main/war/resources/WEB-INF/web.xml:

          Index: resources/WEB-INF/web.xml
          ===================================================================
          --- resources/WEB-INF/web.xml	(revision 24689)
          +++ resources/WEB-INF/web.xml	(working copy)
          @@ -30,6 +30,10 @@
             <display-name>Hudson</display-name>
             <description>Build management system</description>
           
          +  <session-config>
          +    <session-timeout>60</session-timeout>
          +  </session-config>
          +
             <servlet>
               <servlet-name>Stapler</servlet-name>
               <servlet-class>org.kohsuke.stapler.Stapler</servlet-class>
          
          Show
          tjuerge tjuerge added a comment - This can be fixed by specifying a session timeout in hudson/main/war/resources/WEB-INF/web.xml : Index: resources/WEB-INF/web.xml =================================================================== --- resources/WEB-INF/web.xml (revision 24689) +++ resources/WEB-INF/web.xml (working copy) @@ -30,6 +30,10 @@ <display-name>Hudson</display-name> <description>Build management system</description> + <session-config> + <session-timeout>60</session-timeout> + </session-config> + <servlet> <servlet-name>Stapler</servlet-name> <servlet-class>org.kohsuke.stapler.Stapler</servlet-class>
          Hide
          tjuerge tjuerge added a comment -

          Answered my question: winstone.HostConfiguration.run() is responsible for calling WebAppCOnfiguration.invalidateExpiredSessions()

          Show
          tjuerge tjuerge added a comment - Answered my question: winstone.HostConfiguration.run() is responsible for calling WebAppCOnfiguration.invalidateExpiredSessions()
          Hide
          slurdge slurdge added a comment -

          We have also the same issue.
          I tried tjuerge patch to web.xml but it did not change anything (by the way I had to remove a sha1 - computed file to do this).
          We have a ActiveDirectory that is configured via LDAP. Every connection leaks a little bit. The problem is worsened by the fact that we use a tray applet that checks the build status. This tray applet authenticates itself with Hudson. I believe that is creating a WinStoneSession each time, leaking memory slowly.
          I've also looked at the memory dump and my findings are the same as tjuerge.
          In one week, Hudson takes the maximum of 512Mb allocated and starts to fail builds.

          Any solution is welcomed, I would be happy to try a build with a fix for this.
          Also, if there is a workaround in forcing the flush of expired sessions, this would allow us to periodically do it and not interrupt operation by restarting Hudson (the only available option so far, but not very good for us).

          Show
          slurdge slurdge added a comment - We have also the same issue. I tried tjuerge patch to web.xml but it did not change anything (by the way I had to remove a sha1 - computed file to do this). We have a ActiveDirectory that is configured via LDAP. Every connection leaks a little bit. The problem is worsened by the fact that we use a tray applet that checks the build status. This tray applet authenticates itself with Hudson. I believe that is creating a WinStoneSession each time, leaking memory slowly. I've also looked at the memory dump and my findings are the same as tjuerge. In one week, Hudson takes the maximum of 512Mb allocated and starts to fail builds. Any solution is welcomed, I would be happy to try a build with a fix for this. Also, if there is a workaround in forcing the flush of expired sessions, this would allow us to periodically do it and not interrupt operation by restarting Hudson (the only available option so far, but not very good for us).
          Hide
          steventheamusing StevenTheAmusing added a comment -

          We're also seeing a large number of objects 6,143 sessions with 17.9Mb. Can this timeout be added to the standard distribution?

          We're going OOM as often as 3 times a day. When monit restarts Hudson we blow away the expanded WAR directory which messing with the web.xml a bit much pain.

          We are also using the Hudson tray applet. Our primary OOM problem seems to be related to the Cobertura reports, by the way, but this is not a small leak either.

          Hudson ver. 1.353 on Linux, Sun JDK 1.6.0_18

          Show
          steventheamusing StevenTheAmusing added a comment - We're also seeing a large number of objects 6,143 sessions with 17.9Mb. Can this timeout be added to the standard distribution? We're going OOM as often as 3 times a day. When monit restarts Hudson we blow away the expanded WAR directory which messing with the web.xml a bit much pain. We are also using the Hudson tray applet. Our primary OOM problem seems to be related to the Cobertura reports, by the way, but this is not a small leak either. Hudson ver. 1.353 on Linux, Sun JDK 1.6.0_18
          Hide
          steventheamusing StevenTheAmusing added a comment -

          VisualVM showing a large number of sessions.

          Show
          steventheamusing StevenTheAmusing added a comment - VisualVM showing a large number of sessions.
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          Fixed in Winstone 0.9.10-hudson-17 to have the finite default session timeout value. This change still needs to be integrated into Hudson.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - Fixed in Winstone 0.9.10-hudson-17 to have the finite default session timeout value. This change still needs to be integrated into Hudson.
          Hide
          amrangan amrangan added a comment -

          Will this change be integrated into the next release of Hudson (i.e. 1.356) ?

          Show
          amrangan amrangan added a comment - Will this change be integrated into the next release of Hudson (i.e. 1.356) ?
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/war/pom.xml
          trunk/www/changelog.html
          http://jenkins-ci.org/commit/30797
          Log:
          [FIXED JENKINS-5119] integrated a newer version of Winstone with memory leak fix and SSL certificate handling improvement.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/war/pom.xml trunk/www/changelog.html http://jenkins-ci.org/commit/30797 Log: [FIXED JENKINS-5119] integrated a newer version of Winstone with memory leak fix and SSL certificate handling improvement.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/war/pom.xml
          trunk/www/changelog.html
          http://jenkins-ci.org/commit/33533
          Log:
          [FIXED JENKINS-5119] integrated another Winstone that fixes session object leak.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/war/pom.xml trunk/www/changelog.html http://jenkins-ci.org/commit/33533 Log: [FIXED JENKINS-5119] integrated another Winstone that fixes session object leak.
          Hide
          dogfood dogfood added a comment -

          Integrated in hudson_main_trunk #154
          [FIXED JENKINS-5119] integrated another Winstone that fixes session object leak.

          kohsuke :
          Files :

          • /trunk/hudson/main/war/pom.xml
          • /trunk/www/changelog.html
          Show
          dogfood dogfood added a comment - Integrated in hudson_main_trunk #154 [FIXED JENKINS-5119] integrated another Winstone that fixes session object leak. kohsuke : Files : /trunk/hudson/main/war/pom.xml /trunk/www/changelog.html

            People

            • Assignee:
              Unassigned
              Reporter:
              tjuerge tjuerge
            • Votes:
              5 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: