Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46687

Job-DSL Leaks Memory Like A Sieve Thanks To Groovy

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Exactly what it says on the tin - unfortunately it's a stability problem for more intensive uses.  There was a previous fix done but more recent versions of Groovy from later cores no longer handle classes int he same way and the hacks to purge its memory-leaky ways need to change. 

      Now that we have gotten the "dangit groovy!" bits out of the way, there's an easy fix!  I would suggest incorporating the latest version of the fixes Pipeline is using to work around the issues from here: https://github.com/jenkinsci/workflow-cps-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java#L1052

      It should be a pretty much direct copy+paste, thankfully, and the above code is already pretty heavily tested (you can port over the tests too if'n y'like). 

        Attachments

          Activity

          Hide
          sag47 Sam Gleske added a comment -

          Sam Van Oort in your example you recommend source code lines from the master branch which is a moving target; subject to change. I recommend picking a commit and then reference the source file and line from that commit so it stays the same.

          Show
          sag47 Sam Gleske added a comment - Sam Van Oort in your example you recommend source code lines from the master branch which is a moving target; subject to change. I recommend picking a commit and then reference the source file and line from that commit so it stays the same.
          Hide
          sag47 Sam Gleske added a comment -

          Do you think this could be run from a Job DSL script? i.e. run a "clean up classloader" step at the end of running job dsl scripts?

          https://github.com/jenkinsci/workflow-cps-plugin/blob/d348018be09df2a742e6a4bdcb2efd2cf7a5ed41/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java#L1038-L1082

          Show
          sag47 Sam Gleske added a comment - Do you think this could be run from a Job DSL script? i.e. run a "clean up classloader" step at the end of running job dsl scripts? https://github.com/jenkinsci/workflow-cps-plugin/blob/d348018be09df2a742e6a4bdcb2efd2cf7a5ed41/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java#L1038-L1082
          Hide
          jglick Jesse Glick added a comment -

          I feel like this is a duplicate. There was some popular bug (in core?) about Groovy leaks but it did not apply to Pipeline. I cannot find it now.

          See JENKINS-23762 and JENKINS-29068 however.

          Show
          jglick Jesse Glick added a comment - I feel like this is a duplicate. There was some popular bug (in core ?) about Groovy leaks but it did not apply to Pipeline. I cannot find it now. See  JENKINS-23762 and  JENKINS-29068 however.
          Hide
          svanoort Sam Van Oort added a comment -

          Jesse Glick If I can get https://issues.jenkins-ci.org/browse/JENKINS-47758 working it might protect against this

          Show
          svanoort Sam Van Oort added a comment - Jesse Glick If I can get https://issues.jenkins-ci.org/browse/JENKINS-47758  working it might protect against this
          Hide
          svanoort Sam Van Oort added a comment -

          https://issues.jenkins-ci.org/browse/JENKINS-47758 won't protect job-dsl because it has a very custom approach to parsing/classloading/executing the groovy – I might be able to add some hooks in script-security that job-dsl can consume to get the cleanup benefits though.  

          Show
          svanoort Sam Van Oort added a comment - https://issues.jenkins-ci.org/browse/JENKINS-47758  won't protect job-dsl because it has a very custom approach to parsing/classloading/executing the groovy – I might be able to add some hooks in script-security that job-dsl can consume to get the cleanup benefits though.  

            People

            • Assignee:
              daspilker Daniel Spilker
              Reporter:
              svanoort Sam Van Oort
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: