Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13617

64-bit java.lang.OutOfMemoryError: PermGen space

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: core
    • Environment:
    • Similar Issues:

      Description

      Even with -XX:PermSize=512M I still get java.lang.OutOfMemoryError: PermGen space about once a day with light load. Our 32-bit Jenkins has never had this problem and no special settings. Memory leak?

      Apr 26, 2012 9:56:34 AM winstone.Logger logInternal
      WARNING: Untrapped Error in Servlet
      java.lang.OutOfMemoryError: PermGen space
      at java.lang.Throwable.getStackTraceElement(Native Method)
      at java.lang.Throwable.getOurStackTrace(Throwable.java:591)
      at java.lang.Throwable.printStackTraceAsCause(Throwable.java:529)
      at java.lang.Throwable.printStackTraceAsCause(Throwable.java:545)
      at java.lang.Throwable.printStackTraceAsCause(Throwable.java:545)
      at java.lang.Throwable.printStackTrace(Throwable.java:516)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:224)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:171)
      at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
      at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:74)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:331)
      at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:215)
      at winstone.RequestHandlerThread.run(RequestHandlerThread.java:138)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      Apr 26, 2012 9:56:37 AM winstone.Logger logInternal
      WARNING: Untrapped Error in Servlet
      java.lang.OutOfMemoryError: PermGen space
      Apr 26, 2012 9:56:50 AM hudson.triggers.SafeTimerTask run
      SEVERE: Timer task hudson.model.LoadStatistics$LoadStatisticsUpdater@2b1c2043 failed
      java.lang.OutOfMemoryError: PermGen space

        Attachments

        1. memory.dump.bz2
          8.53 MB
        2. memory.dump.bz2
          8.54 MB
        3. periodicbackup.hpi
          1.67 MB
        4. periodicbackup.hpi
          1.67 MB

          Issue Links

            Activity

            Hide
            niklaus_giger Niklaus Giger added a comment -

            Restarted jenkins with the arguments suggested by you. Will let it run for about two days.
            If I get no memory dumps, I will move the jenkins to run under tomcat6.
            Thanks a lot for your work!

            Show
            niklaus_giger Niklaus Giger added a comment - Restarted jenkins with the arguments suggested by you. Will let it run for about two days. If I get no memory dumps, I will move the jenkins to run under tomcat6. Thanks a lot for your work!
            Hide
            johno Johno Crawford added a comment - - edited

            Niklaus, if there are no memory dumps in that time I think we can rule PeriodicBackup plugin as the culprit and there is no need to move Jenkins to run under Tomcat.

            I have merged the fixes to the PeriodicBackup plugin repo JENKINS-16223 (1.2+ contains the fixes in my snapshot) which should now be available from your plugin manager (you may need to click "Check now" in the advanced tab).

            The exception java.lang.OutOfMemoryError: PermGen space is generic, there is no way to avoid it if there is a class loader memory leak, increasing the max permsize, enabling other switches which modify the JVM GC behaviour will (most of the time) prolong the ticking time bomb.

            For those interested in knowing more about PermGen check out this article: http://frankkieviet.blogspot.ca/2006/10/classloader-leaks-dreaded-permgen-space.html

            If there are others receiving this error after increasing both -XX:MaxPermSize and -Xmx to reasonable values the OOM might be caused by plugin(s) you have installed or the way you use Jenkins (there might be undiscovered memory leaks in core), however without a heap dump to analyse it is practically impossible to debug due to the size of the code base and amount of plugins.

            Adam, instead of using the switch -XX:PermSize you should be using -XX:MaxPermSize, in addition, 64bit JVM will use almost double the amount of memory for object pointers ( https://wikis.oracle.com/display/HotSpotInternals/CompressedOops ). You can try adding -XX:+UseCompressedOops to your JVM args to help, however you may have ran into a memory leak like Niklaus (in which case a heap dump is required).

            Show
            johno Johno Crawford added a comment - - edited Niklaus, if there are no memory dumps in that time I think we can rule PeriodicBackup plugin as the culprit and there is no need to move Jenkins to run under Tomcat. I have merged the fixes to the PeriodicBackup plugin repo JENKINS-16223 (1.2+ contains the fixes in my snapshot) which should now be available from your plugin manager (you may need to click "Check now" in the advanced tab). The exception java.lang.OutOfMemoryError: PermGen space is generic, there is no way to avoid it if there is a class loader memory leak, increasing the max permsize, enabling other switches which modify the JVM GC behaviour will (most of the time) prolong the ticking time bomb. For those interested in knowing more about PermGen check out this article: http://frankkieviet.blogspot.ca/2006/10/classloader-leaks-dreaded-permgen-space.html If there are others receiving this error after increasing both -XX:MaxPermSize and -Xmx to reasonable values the OOM might be caused by plugin(s) you have installed or the way you use Jenkins (there might be undiscovered memory leaks in core), however without a heap dump to analyse it is practically impossible to debug due to the size of the code base and amount of plugins. Adam, instead of using the switch -XX:PermSize you should be using -XX:MaxPermSize, in addition, 64bit JVM will use almost double the amount of memory for object pointers ( https://wikis.oracle.com/display/HotSpotInternals/CompressedOops ). You can try adding -XX:+UseCompressedOops to your JVM args to help, however you may have ran into a memory leak like Niklaus (in which case a heap dump is required).
            Hide
            niklaus_giger Niklaus Giger added a comment -

            Johno, I can confirm that there were no memory dumps in the last three days. I have therefore upgraded jenkins to 1.496 and periodicbackup to 1.3. I will report back in a few days to tell you whether everything is okay with this combination.

            Show
            niklaus_giger Niklaus Giger added a comment - Johno, I can confirm that there were no memory dumps in the last three days. I have therefore upgraded jenkins to 1.496 and periodicbackup to 1.3. I will report back in a few days to tell you whether everything is okay with this combination.
            Hide
            johno Johno Crawford added a comment -

            How's it looking Niklaus?

            Show
            johno Johno Crawford added a comment - How's it looking Niklaus?
            Hide
            niklaus_giger Niklaus Giger added a comment -

            It is looking good. Running since January 06, 00:37 AM without any PermGen errors.
            It seems that this nasty problem has been fixed for me! I think you are entitled now to collect your 100$ on freedomsponsors.

            Thanks a lot for your help!

            Show
            niklaus_giger Niklaus Giger added a comment - It is looking good. Running since January 06, 00:37 AM without any PermGen errors. It seems that this nasty problem has been fixed for me! I think you are entitled now to collect your 100$ on freedomsponsors. Thanks a lot for your help!

              People

              • Assignee:
                johno Johno Crawford
                Reporter:
                asloan7 Adam Sloan
              • Votes:
                4 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: