Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50317

Pipeline job is failed with java.lang.StackOverflowError

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: pipeline
    • Environment:
      Jenkins Master WIndows Server 2012, Jenkins
      version 2.112, 2 slaves, Jenkins Slave Windows 10 and Jenkins Slave Debian 9.3(Stretch)
    • Similar Issues:

      Description

      We use Multi branch job. Sometimes pipeline job is failed with java.lang.StackOverflowError.I send you our Jenkinsfile and pipeline.log. Maybe is job name too long?

      Also sometimes parallel stages hang when one stage is failed(failFast true) via this exception but I think it's other bug. Please write if you need some additional logs or system information.

      Jenkinsfile

      log.txt

        Attachments

        1. 111.zip
          228 kB
        2. autoloader.zip
          31 kB
        3. Jenkinsfile
          6 kB
        4. log.txt
          263 kB
        5. utils.groovy
          5 kB

          Activity

          olecsys Aleksandr Gamoskin created issue -
          Hide
          olecsys Aleksandr Gamoskin added a comment -

          I'am using Jenkins Pipeline Shared Library but there is still this exception. It breaks our build flow. Please tell me what to do?

          Show
          olecsys Aleksandr Gamoskin added a comment - I'am using Jenkins Pipeline Shared Library but there is still this exception. It breaks our build flow. Please tell me what to do?
          Hide
          abayer Andrew Bayer added a comment -

          Can you attach your utils.groovy, please? It looks like something is going wrong specifically in serialization of the program state, and given that all your work seems to actually be in that utils.groovy script, I can't tell what's going on. Also, it'd be great if you could put together a minimal reproduction case or zip up the build directory for a build that's failing like this (specifically the workflow subdirectory) so that we can see where exactly the failure is happening.

          Show
          abayer Andrew Bayer added a comment - Can you attach your utils.groovy , please? It looks like something is going wrong specifically in serialization of the program state, and given that all your work seems to actually be in that utils.groovy script, I can't tell what's going on. Also, it'd be great if you could put together a minimal reproduction case or zip up the build directory for a build that's failing like this (specifically the workflow subdirectory) so that we can see where exactly the failure is happening.
          olecsys Aleksandr Gamoskin made changes -
          Field Original Value New Value
          Attachment utils.groovy [ 41984 ]
          olecsys Aleksandr Gamoskin made changes -
          Attachment autoloader.zip [ 41985 ]
          Hide
          olecsys Aleksandr Gamoskin added a comment -

          Hello Andrew, yes, I'am sending you utils.groovy. Also I could send you workspace subdirectory but we use conan io server, cmake etc. I look forward for your response.

           

          autoloader.zip

          utils.groovy

           ^^ 

          Show
          olecsys Aleksandr Gamoskin added a comment - Hello Andrew, yes, I'am sending you utils.groovy. Also I could send you workspace subdirectory but we use conan io server, cmake etc. I look forward for your response.   autoloader.zip utils.groovy  ^^ 
          Hide
          abayer Andrew Bayer added a comment -

          Aleksandr Gamoskin - I'm specifically looking for the build directory on the master - it'd be something like JENKINS_HOME/jobs/JOB_NAME/builds/BUILD_NUMBER, though if this is a multibranch job, it would be in JENKINS_HOME/jobs/MULTIBRANCH_NAME/jobs/BRANCH/builds/BUILD_NUMBER. In that directory, what we really care about is the workflow subdirectory, if the whole build directory is too large. Thanks!

          Show
          abayer Andrew Bayer added a comment - Aleksandr Gamoskin - I'm specifically looking for the build directory on the master - it'd be something like JENKINS_HOME/jobs/JOB_NAME/builds/BUILD_NUMBER , though if this is a multibranch job, it would be in JENKINS_HOME/jobs/MULTIBRANCH_NAME/jobs/BRANCH/builds/BUILD_NUMBER . In that directory, what we really care about is the workflow subdirectory, if the whole build directory is too large. Thanks!
          olecsys Aleksandr Gamoskin made changes -
          Attachment 111.zip [ 42015 ]
          Hide
          olecsys Aleksandr Gamoskin added a comment -

          Andrew Bayer I see. So I'am sending one of the crashed builds.

          111.zip

          Show
          olecsys Aleksandr Gamoskin added a comment - Andrew Bayer I see. So I'am sending one of the crashed builds. 111.zip
          Hide
          olecsys Aleksandr Gamoskin added a comment -

          Andrew Bayer hello. Maybe do you have a temporary workaround for this problem?All our slaves hang in this situation and then I abort these jobs manualy.

          Show
          olecsys Aleksandr Gamoskin added a comment - Andrew Bayer hello. Maybe do you have a temporary workaround for this problem?All our slaves hang in this situation and then I abort these jobs manualy.
          Hide
          abayer Andrew Bayer added a comment -

          It appears that the error is happening when your debian-6 x64 debug stage's success post action is running. That said, I can't tell what was actually being serialized - anything that was in the state of your Pipeline at that point would be getting serialized. All I can be sure of is that the script step started and then everything blew up. I'm actually wondering if the withEnv without any actual body content in conan.groovy's success method might be the culprit somehow? Not sure. I'd suggest trying moving to non-parallel build stages to see if the same error shows up then.

          Show
          abayer Andrew Bayer added a comment - It appears that the error is happening when your debian-6 x64 debug stage's success post action is running. That said, I can't tell what was actually being serialized - anything that was in the state of your Pipeline at that point would be getting serialized. All I can be sure of is that the script step started and then everything blew up. I'm actually wondering if the withEnv without any actual body content in conan.groovy 's success method might be the culprit somehow? Not sure. I'd suggest trying moving to non-parallel build stages to see if the same error shows up then.
          Hide
          olecsys Aleksandr Gamoskin added a comment -

          So I've changed utils.groovy to Jenkins Pipeline Shared Library(git repository). But problem occurs any way. Maybe do you need more detailed information? Switch trace log level? Is parallel mechanism is not so stable for now? it's just very useful thing. I'll try to moving build stages to non-parallel build stages.

          Show
          olecsys Aleksandr Gamoskin added a comment - So I've changed utils.groovy to Jenkins Pipeline Shared Library(git repository). But problem occurs any way. Maybe do you need more detailed information? Switch trace log level? Is parallel mechanism is not so stable for now? it's just very useful thing. I'll try to moving build stages to non-parallel build stages.
          Hide
          olecsys Aleksandr Gamoskin added a comment -

          We use Gitea plugin. Could it be it's problem? It uses some notification mechanism.

          Show
          olecsys Aleksandr Gamoskin added a comment - We use Gitea plugin. Could it be it's problem? It uses some notification mechanism.
          Hide
          olecsys Aleksandr Gamoskin added a comment -

          Andrew Bayer I've remove parallel stages and problem does not occur for now. If you need I could send you new groovy files and pipeline.

          Show
          olecsys Aleksandr Gamoskin added a comment - Andrew Bayer I've remove parallel stages and problem does not occur for now. If you need I could send you new groovy files and pipeline.
          Hide
          abayer Andrew Bayer added a comment -

          Ok, if it's not happening when you're not in parallel, then it's got to be something that's temporarily/briefly exposed in one of the parallel branches that Pipeline attempts to serialize when it reaches a safepoint on one of the other branches. Could be Gitea, could be something else - I really can't tell. The best I can advise is that you try to reproduce this as minimally as possible - starting with empty parallel branches and gradually adding bits and pieces until you can reproduce the error.

          Show
          abayer Andrew Bayer added a comment - Ok, if it's not happening when you're not in parallel, then it's got to be something that's temporarily/briefly exposed in one of the parallel branches that Pipeline attempts to serialize when it reaches a safepoint on one of the other branches. Could be Gitea, could be something else - I really can't tell. The best I can advise is that you try to reproduce this as minimally as possible - starting with empty parallel branches and gradually adding bits and pieces until you can reproduce the error.
          abayer Andrew Bayer made changes -
          Priority Critical [ 2 ] Major [ 3 ]
          Hide
          abayer Andrew Bayer added a comment -

          Any update from your side, Aleksandr Gamoskin?

          Show
          abayer Andrew Bayer added a comment - Any update from your side, Aleksandr Gamoskin ?
          vivek Vivek Pandey made changes -
          Labels pipeline windows pipeline triaged-2018-11 windows
          Hide
          erikpaulmiller Erik Miller added a comment -

          Happening for me as well. Update?

          Show
          erikpaulmiller Erik Miller added a comment - Happening for me as well. Update?
          thomashutchins Thomas Hutchins made changes -
          Rank Ranked higher
          Hide
          based3 Basile Chandesris added a comment - - edited

          I also got this, while playing with environment variables (bindings) and pipeline/workflow environment section.

          java.lang.StackOverflowError
          at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:498)
          at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503)
          at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503)
          at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503)
          at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503)
          at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503)
          at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503)

          Jenkins 2.190.1, lastest plugin updates,  multibranch, GNU/LInux.

           

           

          Show
          based3 Basile Chandesris added a comment - - edited I also got this, while playing with environment variables (bindings) and pipeline/workflow environment section. java.lang.StackOverflowError at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:498) at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503) at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503) at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503) at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503) at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503) at org.jenkinsci.plugins.workflow.cps.DSL.flattenGString(DSL.java:503) Jenkins 2.190.1, lastest plugin updates,  multibranch, GNU/LInux.    

            People

            • Assignee:
              Unassigned
              Reporter:
              olecsys Aleksandr Gamoskin
            • Votes:
              4 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated: