Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56500

Declarative pipeline restricted in code size

    Details

    • Similar Issues:

      Description

      There is partial fix for this in pipeline-model-definition-plugin v1.4.0 and later.  Due to the extent to which it change how pipelines are executed it is turned off by default.  It can be turned on by setting a JVM property (either on the command-line or in Jenkins script console):

      org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true 

      As noted, this still works best with a Jenkinsfile with pipeline directive as the only root item in the file.
      This workaround also works to some extent for pipelines using `def` variables before the pipeline directive, but not nearly as well. 
      This workaround generally does NOT work if the pipeline directive inside a shared library method. If this is a scenario you want, please come join the pipeline authoring SIG and we can discuss.

      Please give it a try and provide feedback.  

      When working with a declarative pipeline script, we run into an error "Method Code too large".

      This seems to happen when the code between 'pipeline{}' is more than a specific size. 

       

      I'm creating this issue for this component following the suggestion of Jesse Glick at the issue 37984

      I've attached a declarative pipeline script that reproduces the issue. 

       

       

        Attachments

          Issue Links

            Activity

            Hide
            mbrunton27 Matthew Brunton added a comment -

            Liam Newman 

            We have also been running into the method code too large. I was able to get a workaround by switching to a hybrid scripted/declarative, but have been trying various things to get back to declarative.

            I've tried various methods, including having everything under vars, a mixture of vars and src, and playing around with the various classes we have. Eventually I commented out chunks of code until the pipeline stopped giving "method code too large" and then started adding stuff back.

            I think I've narrowed it down to the "when {}" stage conditionals. I'm currently using 25 parallel stages (each one has 2 stages, and they are skeletal because of my commented out code) and I've changed all the conditionals to "when { expression

            {true}

            }". Some of our conditionals use anyOf {} and not {}, but I set everything to true in order to rule out the expressions themselves.

            Without the conditionals, there is no "method code too large" error. When I add back the conditionals, the error returns.

            Show
            mbrunton27 Matthew Brunton added a comment - Liam Newman   We have also been running into the method code too large. I was able to get a workaround by switching to a hybrid scripted/declarative, but have been trying various things to get back to declarative. I've tried various methods, including having everything under vars, a mixture of vars and src, and playing around with the various classes we have. Eventually I commented out chunks of code until the pipeline stopped giving "method code too large" and then started adding stuff back. I think I've narrowed it down to the "when {}" stage conditionals. I'm currently using 25 parallel stages (each one has 2 stages, and they are skeletal because of my commented out code) and I've changed all the conditionals to "when { expression {true} }". Some of our conditionals use anyOf {} and not {}, but I set everything to true in order to rule out the expressions themselves. Without the conditionals, there is no "method code too large" error. When I add back the conditionals, the error returns.
            Hide
            bitwiseman Liam Newman added a comment -

            Matthew Brunton

            Have you tried the feature flag? (See description for note)

            Do you have any def variables before (or after) your pipeline directive in your Jenkinsfile?
            Is your pipeline directive inside a shared library method?

            Show
            bitwiseman Liam Newman added a comment - Matthew Brunton Have you tried the feature flag? (See description for note) Do you have any def variables before (or after) your pipeline directive in your Jenkinsfile? Is your pipeline directive inside a shared library method?
            Hide
            mbrunton27 Matthew Brunton added a comment - - edited

            Liam Newman

            Yes, I have the feature flag turned on. I also got rid of all def variables outside of the pipeline directive, and I even got rid of all script blocks inside just in case.

            We do run inside a shared library method, however. Our pipeline is version controlled separately. I see now that this could be the reason why the feature flag workaround isn't working.

            I tried copying the pipeline directive into the Jenkinsfile so the only thing outside of the pipeline is the " @ Library ( ) _ " . I still get the error.

            Show
            mbrunton27 Matthew Brunton added a comment - - edited Liam Newman Yes, I have the feature flag turned on. I also got rid of all def variables outside of the pipeline directive, and I even got rid of all script blocks inside just in case. We do run inside a shared library method, however. Our pipeline is version controlled separately. I see now that this could be the reason why the feature flag workaround isn't working. I tried copying the pipeline directive into the Jenkinsfile so the only thing outside of the pipeline is the " @ Library ( ) _ " . I still get the error.
            Hide
            mbrunton27 Matthew Brunton added a comment - - edited

            I ran some tests with a fresh bare-bones script and a fresh Jenkins, both with and without the feature flag. With the flag on, I do indeed get a few more stages before I hit the error.

            This is a typical stage, and I'm able to get 27 in parallel without the flag. With the flag, I can get somewhere between 30 and 35 stages.

            def call()
            { 
              pipeline 
              { 
                agent any 
                stages
                { 
                  stage('Build') 
                  { 
                    parallel 
                    {
            
            ...
            
            stage('Parallel Stage 35')
            { 
                when {expression {true}}
                stages { 
                stage('35-A')
                {
                    when 
                    { 
                        anyOf
                        { 
                            expression { true } 
                            not { expression { true } }
                        } 
                    } 
                    steps
                    { 
                        testMethod('TEST') 
                    } 
                } 
                stage('35-B') 
                { 
                    options { timeout(time: 1, unit: 'HOURS') }
                    when 
                    { 
                        anyOf 
                        { 
                            expression { true } 
                            not { expression { true } } 
                        } 
                    } 
                    steps 
                    { 
                        testMethod('TEST') 
                    } 
                } 
                } 
            }

            The call to testMethod is just another file under vars:

            // vars/testMethod.groovy
            
            def call(String testInput)
            {    
                def testVariable
            }
            

             

            I get a similar result when I move the pipeline into the Jenkinsfile 

             

            Show
            mbrunton27 Matthew Brunton added a comment - - edited I ran some tests with a fresh bare-bones script and a fresh Jenkins, both with and without the feature flag. With the flag on, I do indeed get a few more stages before I hit the error. This is a typical stage, and I'm able to get 27 in parallel without the flag. With the flag, I can get somewhere between 30 and 35 stages. def call() { pipeline { agent any stages { stage( 'Build' ) { parallel { ... stage( 'Parallel Stage 35' ) { when {expression { true }} stages { stage( '35-A' ) { when { anyOf { expression { true } not { expression { true } } } } steps { testMethod( 'TEST' ) } } stage( '35-B' ) { options { timeout(time: 1, unit: 'HOURS' ) } when { anyOf { expression { true } not { expression { true } } } } steps { testMethod( 'TEST' ) } } } } The call to testMethod is just another file under vars: // vars/testMethod.groovy def call( String  testInput) {     def testVariable }   I get a similar result when I move the pipeline into the Jenkinsfile   
            Hide
            bitwiseman Liam Newman added a comment - - edited

            Matthew Brunton
            Thanks for trying that out. So at least we know the feature flag is doing something.

            I just found out the shared library method limitation this week.

            Show
            bitwiseman Liam Newman added a comment - - edited Matthew Brunton Thanks for trying that out. So at least we know the feature flag is doing something. I just found out the shared library method limitation this week.

              People

              • Assignee:
                bitwiseman Liam Newman
                Reporter:
                wim Wim Gaethofs
              • Votes:
                8 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated: