Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33358

Groovy and PermGen memory leak

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: core
    • Environment:
    • Similar Issues:

      Description

      We have recently experienced OutOfMemory errors on some large Jenkins instances that predominantly have frequently executed jobs that contain either System Groovy scripts or Build Flow project types. In one case after disabling a groovy script that was being executed via the Global Post Script Plugin after every build, the PermGen profile improved dramatically.

      We tested further with GroovyShell to run a groovy script, not depending on Jenkins.
      Java7: java.lang.OutOfMemoryError: PermGen space
      Java8: java.lang.OutOfMemoryError: Java heap space

      The tests use 2.4.6 version of groovy which is the latest version. However, the issue is still there.
      <dependency>
      <groupId>org.codehaus.groovy</groupId>
      <artifactId>groovy-all</artifactId>
      <version>2.4.6</version>
      </dependency>

      ---------------------------------------
      Configure 10 jobs which build per minute
      Each build triggers a groovy script by Global Post Script Plugin.

      class Main {
          static void main(String... args) {
              println 'Groovy world!'
              def p = new Person()
              p.write()
          }
      }
      
      class Person {
          def write(){
              println 'i am writing ...!'
          }
      }
      

      After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.

      I checked the source code of the plugin:
      https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

      My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
      1. if this is a potential memory leak, and
      2. if anyone has met similar issue.

        Attachments

        1. after.png
          after.png
          43 kB
        2. before.png
          before.png
          70 kB
        3. permgenIssue.jpg
          permgenIssue.jpg
          35 kB
        4. Screenshot from 2016-03-07 09.43.29.png
          Screenshot from 2016-03-07 09.43.29.png
          113 kB
        5. Screenshot from 2016-04-01 11.04.43.png
          Screenshot from 2016-04-01 11.04.43.png
          182 kB

          Issue Links

            Activity

            Hide
            danielbeck Daniel Beck added a comment -

            2.73.1 is scheduled for release this week and will contain Groovy 2.4.11.

            Show
            danielbeck Daniel Beck added a comment - 2.73.1 is scheduled for release this week and will contain Groovy 2.4.11.
            Hide
            jglick Jesse Glick added a comment -

            In 2.61 apparently.

            Show
            jglick Jesse Glick added a comment - In 2.61 apparently.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            Jenkinsfile
            http://jenkins-ci.org/commit/workflow-cps-plugin/9c5667d59d47d4bff4e3dfd5f68ea496448029e1
            Log:
            JENKINS-33358 Test against 2.73.x since that includes Groovy 2.4.11.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: Jenkinsfile http://jenkins-ci.org/commit/workflow-cps-plugin/9c5667d59d47d4bff4e3dfd5f68ea496448029e1 Log: JENKINS-33358 Test against 2.73.x since that includes Groovy 2.4.11.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Sam Van Oort
            Path:
            Jenkinsfile
            pom.xml
            http://jenkins-ci.org/commit/workflow-cps-plugin/a5dea90a589f094caf58411c271f139322f37f2f
            Log:
            Merge pull request #175 from jglick/new-Groovy-JENKINS-33358

            JENKINS-33358 Test against 2.73.x since that includes Groovy 2.4.11

            Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/3396c9715894...a5dea90a589f

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: Jenkinsfile pom.xml http://jenkins-ci.org/commit/workflow-cps-plugin/a5dea90a589f094caf58411c271f139322f37f2f Log: Merge pull request #175 from jglick/new-Groovy- JENKINS-33358 JENKINS-33358 Test against 2.73.x since that includes Groovy 2.4.11 Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/3396c9715894...a5dea90a589f
            Hide
            svanoort Sam Van Oort added a comment -

            https://issues.jenkins-ci.org/browse/JENKINS-47758 might offer us a way to guard against some of the memory leaks. 

            Show
            svanoort Sam Van Oort added a comment - https://issues.jenkins-ci.org/browse/JENKINS-47758  might offer us a way to guard against some of the memory leaks. 

              People

              • Assignee:
                ympaul2 paul young
                Reporter:
                hongkailiu Hongkai Liu
              • Votes:
                23 Vote for this issue
                Watchers:
                54 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: