Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34201

Pipeline plugin can't handle large numbers of parallel build jobs

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Duplicate
    • Component/s: pipeline
    • Labels:
      None
    • Environment:
      Jenkins 1.656 on RHEL7 with Pipeline plugin 2.0
    • Similar Issues:

      Description

      Consider following snippet:

      stage name: 'foo', concurrency: 10
      foo = [:]
      foo['failFast'] = true
      
      for (int i = 0; i < 25; i++) {
          for (int j = 0; j < 4; j++) {
              foo["branch${i}-${j}"] = {
                  node {
                      build job: 'job1', parameters: [
                          [ $class: 'StringParameterValue', name: 'foo', value: 'f' ],
                          [ $class: 'StringParameterValue', name: 'bar', value: 'b' ],
                          [ $class: 'StringParameterValue', name: 'baz', value: 'z' ]
                      ], quietPeriod: 0
                  }
      
                  node {
                      build job: 'job2', parameters: [
                          [ $class: 'StringParameterValue', name: 'foo', value: 'f' ],
                          [ $class: 'StringParameterValue', name: 'bar', value: 'b' ],
                          [ $class: 'StringParameterValue', name: 'baz', value: 'z' ]
                      ], quietPeriod: 0
                  }
              }
          }
      
          parallel foo
      }
      

      It starts to build the requested jobs just fine.

      INFO: job1 #125 main build action completed: SUCCES
      INFO: job1 #127 main build action completed: SUCCES
      INFO: job1 #126 main build action completed: SUCCES
      INFO: job2 #124 main build action completed: SUCCES
      ...

      However when all of the jobs are done it seems Pipeline can't seem to merge all those results and is just stuck. After 24h it's still hanging, seemingly waiting for a parallel job to finish.

      Now when I remove the outermost for loop things run just fine.

      for (int j = 0; j < 4; j++) {
              foo["branch${j}"] = {
                      ...
              }
      }
      

      Removing the inner for loop and increasing the outer loop to 100 results in foo[] being too big for Jenkins to handle. Same happens without said for loops obviously, which lead me to start using them.

      for (int i = 0; i < 100; j++) {
              foo["branch${i}"] = {
                      ...
              }
      }
      

      There's probably a better way to handle this.
      Any pointers how to get there?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                tomdevylder Tom De Vylder
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: