Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38383

StackOverflow when loading a large pipeline job

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I've been using infinite Pipeline for a demo. The job was running for a couple of hours, and then Jenkins master declined to the build due to the StackOverflow.

      node {
          int i = 0;
          while(true) {
              sh "echo 'Hello, world ${i}!'"
              sh "sleep 5"
              i = i + 1;
          }
      }
      

      Log: See attachment

        Attachments

        1. InfinitePipeline.zip
          1.45 MB
        2. log.txt
          7.50 MB
        3. workflow-job.hpi
          109 kB

          Issue Links

            Activity

            Hide
            svanoort Sam Van Oort added a comment -

            Thomas Johansen I'm not sure exactly what's triggering an NPE there (looks like it somehow got caught in an invalid state) – tried to reproduce but I can't.

            But I've got a fix that's verified to remove the StackOverflowException and allow a build to cleanly resume in https://github.com/jenkinsci/workflow-job-plugin/pull/91 – please try the attached snapshot built off of it which should remove your issue.

            In general I'd advise to find alternate courses in Pipeline than explicit polling loops – in this case an enhancement to JIRA plugin or simply accept the use of an agent to do polling while needed (perhaps with exponential backoff). workflow-job.hpi

            Show
            svanoort Sam Van Oort added a comment - Thomas Johansen I'm not sure exactly what's triggering an NPE there (looks like it somehow got caught in an invalid state) – tried to reproduce but I can't. But I've got a fix that's verified to remove the StackOverflowException and allow a build to cleanly resume in https://github.com/jenkinsci/workflow-job-plugin/pull/91 – please try the attached snapshot built off of it which should remove your issue. In general I'd advise to find alternate courses in Pipeline than explicit polling loops – in this case an enhancement to JIRA plugin or simply accept the use of an agent to do polling while needed (perhaps with exponential backoff). workflow-job.hpi
            Hide
            thxmasj Thomas Johansen added a comment -

            Sam Van Oort: I tried the snapshot. No StackOverflowException in the log this time, but the build did not resume either (waited almost half an hour). I'm considering to use a polling agent in combination with the input construct to completely avoid these damned loops.

            [Pipeline] echo
            19:41:58 Iteration 6143
            [Pipeline] echo
            Resuming build at Tue Feb 27 19:42:36 GMT 2018 after Jenkins restart
            Waiting to resume part of jenkins-pipeline » work/PBLEID-15078 #1: jenkins-slave-2 is offline
            19:41:58 Iteration 6143
            Waiting to resume part of jenkins-pipeline » work/PBLEID-15078 #1: jenkins-slave-2 is offline
            Waiting to resume part of jenkins-pipeline » work/PBLEID-15078 #1: jenkins-slave-2 is offline
            Waiting to resume part of jenkins-pipeline » work/PBLEID-15078 #1: Jenkins doesn’t have label jenkins-slave-2
            jenkins-slave-1 doesn’t have label jenkins-slave-2
            jenkins-slave-2 is offline
            Ready to run at Tue Feb 27 19:43:10 GMT 2018
            Show
            thxmasj Thomas Johansen added a comment - Sam Van Oort : I tried the snapshot. No StackOverflowException in the log this time, but the build did not resume either (waited almost half an hour). I'm considering to use a polling agent in combination with the input construct to completely avoid these damned loops. [Pipeline] echo 19:41:58 Iteration 6143 [Pipeline] echo Resuming build at Tue Feb 27 19:42:36 GMT 2018 after Jenkins restart Waiting to resume part of jenkins-pipeline » work/PBLEID-15078 #1: jenkins-slave-2 is offline 19:41:58 Iteration 6143 Waiting to resume part of jenkins-pipeline » work/PBLEID-15078 #1: jenkins-slave-2 is offline Waiting to resume part of jenkins-pipeline » work/PBLEID-15078 #1: jenkins-slave-2 is offline Waiting to resume part of jenkins-pipeline » work/PBLEID-15078 #1: Jenkins doesn’t have label jenkins-slave-2 jenkins-slave-1 doesn’t have label jenkins-slave-2 jenkins-slave-2 is offline Ready to run at Tue Feb 27 19:43:10 GMT 2018
            Hide
            svanoort Sam Van Oort added a comment -

            Thomas Johansen That sounds like an issue I'm tracking that sometimes happens with performance-optimized builds with specific combinations of build agents used (noted in a couple quirks by https://issues.jenkins-ci.org/browse/JENKINS-47173) – do you have a sample of your pipeline here to reproduce with?

            Show
            svanoort Sam Van Oort added a comment - Thomas Johansen That sounds like an issue I'm tracking that sometimes happens with performance-optimized builds with specific combinations of build agents used (noted in a couple quirks by https://issues.jenkins-ci.org/browse/JENKINS-47173 ) – do you have a sample of your pipeline here to reproduce with?
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Sam Van Oort
            Path:
            src/main/java/org/jenkinsci/plugins/workflow/job/WorkflowRun.java
            http://jenkins-ci.org/commit/workflow-job-plugin/896703ee33d7c7678272a1cecb7cba1a4c3e1f63
            Log:
            Merge pull request #91 from svanoort/fix-prefix-bugs

            JENKINS-38383 Avoid Stackoverflow and full-flowgraph scan when trying to get branch names for log prefixes

            Compare: https://github.com/jenkinsci/workflow-job-plugin/compare/d23e989dc6d5...896703ee33d7

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: src/main/java/org/jenkinsci/plugins/workflow/job/WorkflowRun.java http://jenkins-ci.org/commit/workflow-job-plugin/896703ee33d7c7678272a1cecb7cba1a4c3e1f63 Log: Merge pull request #91 from svanoort/fix-prefix-bugs JENKINS-38383 Avoid Stackoverflow and full-flowgraph scan when trying to get branch names for log prefixes Compare: https://github.com/jenkinsci/workflow-job-plugin/compare/d23e989dc6d5...896703ee33d7
            Hide
            svanoort Sam Van Oort added a comment -

            Released with v2.19

            Show
            svanoort Sam Van Oort added a comment - Released with v2.19

              People

              • Assignee:
                svanoort Sam Van Oort
                Reporter:
                oleg_nenashev Oleg Nenashev
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: