Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28107

Matrix parent should take an executor slot until the point where children are triggered

    Details

    • Similar Issues:

      Description

      When lots of matrix jobs are queued because a slave has been unavailable, all of these are started at the same time because these do not take up executor slots.

      As the parent job still accesses the VCS, all of these accesses happen in parallel then and may starve each other for resources.

      It would be great if the job would take an executor slot, and relinquish it when it starts triggering the child jobs.

        Attachments

          Activity

          Hide
          danielbeck Daniel Beck added a comment -

          Possible workaround: Don't restrict the parent build to a slave?

          Show
          danielbeck Daniel Beck added a comment - Possible workaround: Don't restrict the parent build to a slave?
          Hide
          simonrichter Simon Richter added a comment -

          That helps distribute the load a bit, but is still suboptimal (rather than 30 concurrent parent jobs I now have 10).

          Show
          simonrichter Simon Richter added a comment - That helps distribute the load a bit, but is still suboptimal (rather than 30 concurrent parent jobs I now have 10).
          Hide
          danielbeck Daniel Beck added a comment -

          That helps distribute the load a bit, but is still suboptimal (rather than 30 concurrent parent jobs I now have 10).

          Isn't that just due to you currently already experiencing this load? If this would have been the config to begin with, they'd have started at different times (unless all have the same trigger, and then the quiet period may be a workaround).

          Show
          danielbeck Daniel Beck added a comment - That helps distribute the load a bit, but is still suboptimal (rather than 30 concurrent parent jobs I now have 10). Isn't that just due to you currently already experiencing this load? If this would have been the config to begin with, they'd have started at different times (unless all have the same trigger, and then the quiet period may be a workaround).
          Hide
          simonrichter Simon Richter added a comment -

          Indeed, that's why I prioritized it as "Minor" – it is possible to work around the problem, but I'd rather see that the system avoids getting into a situation where more processes run on a single system than I have executors configured for it.

          Show
          simonrichter Simon Richter added a comment - Indeed, that's why I prioritized it as "Minor" – it is possible to work around the problem, but I'd rather see that the system avoids getting into a situation where more processes run on a single system than I have executors configured for it.
          Hide
          danielbeck Daniel Beck added a comment -

          I cannot think of a good solution to this issue. Moving between executors, or changing a job from "heavy-weight" to fly-weight isn't really possible AFAIK.

          The best approach I can think of right now is to limit the number of matrix jobs allowed to run in parallel using something like Throttle Concurrent Builds, but that's of course a different limitation (you don't care about how many matrix parents are running after they've finished SCM operations).

          Show
          danielbeck Daniel Beck added a comment - I cannot think of a good solution to this issue. Moving between executors, or changing a job from "heavy-weight" to fly-weight isn't really possible AFAIK. The best approach I can think of right now is to limit the number of matrix jobs allowed to run in parallel using something like Throttle Concurrent Builds, but that's of course a different limitation (you don't care about how many matrix parents are running after they've finished SCM operations).

            People

            • Assignee:
              kohsuke Kohsuke Kawaguchi
              Reporter:
              simonrichter Simon Richter
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: