Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18323

Least Load doesn't appear to work for multi-configuration projects

    Details

    • Similar Issues:

      Description

      Least Load doesn't seem to work for multi-configuration projects. I noticed this by creating a job with 10 configurations (5x2) while I had 8 nodes with 2 executors each in my system. All 8 nodes were part of a label called "Main" and the job was assigned to build on the "Main" label.

      When I triggered the job, 2 of my nodes were completely empty while 4 other nodes were double-loaded. Given my setup, I should have only had 2 double-loaded nodes and no empty nodes.

      I've attached the XML configuration for my job. If you create a "Main" label with 8 nodes, you should be able to reproduce this behavior. Heck, you should be able to reproduce it like so if you don't want to use my exact setup:

      Nodes = N
      Executors per node = E > 1
      Number of configurations in project = C, where (N) < C < (E * N)

      Given Jenkins' normal scheduling algorithm, it's possible you'll need to make C = (E * N) to fill your nodes, then add a node (or subtract configurations) to fit the above formula.

        Attachments

          Activity

          Hide
          bstick12 Brendan Nolan added a comment -

          Version 1.0.2 - Gives best effort fix

          Show
          bstick12 Brendan Nolan added a comment - Version 1.0.2 - Gives best effort fix
          Hide
          bstick12 Brendan Nolan added a comment -

          I've added an alternative computer.isIdle() check to check the actual executors to avoid having a node as not being idle when if it running the control multi-configuration project.

          I added some randomness when retrieving the list of applicable executors in an attempt to better balance the projects launched by the multi-configuration project.

          It is not possible to guarantee that in the case of multi-configuration projects that the least loaded node will be selected as the mapping is batched and the state of the nodes isn't updated during the batching.

          Show
          bstick12 Brendan Nolan added a comment - I've added an alternative computer.isIdle() check to check the actual executors to avoid having a node as not being idle when if it running the control multi-configuration project. I added some randomness when retrieving the list of applicable executors in an attempt to better balance the projects launched by the multi-configuration project. It is not possible to guarantee that in the case of multi-configuration projects that the least loaded node will be selected as the mapping is batched and the state of the nodes isn't updated during the batching.
          Hide
          bstick12 Brendan Nolan added a comment -

          Looks to be two separate issues here.

          1. When a multi-configuration project is launched it is assigned to a slave node but not an executor. Even thought the project isn't using an executor is marks the node as not being idle.
          2. The mapping of tasks from a multi-configuration project occurs in a batch. So the status and number of executors available to a node isn't updated as each task is mapped.

          Will continue investigation.

          Show
          bstick12 Brendan Nolan added a comment - Looks to be two separate issues here. 1. When a multi-configuration project is launched it is assigned to a slave node but not an executor. Even thought the project isn't using an executor is marks the node as not being idle. 2. The mapping of tasks from a multi-configuration project occurs in a batch. So the status and number of executors available to a node isn't updated as each task is mapped. Will continue investigation.

            People

            • Assignee:
              bstick12 Brendan Nolan
              Reporter:
              peachj Jerrod Peach
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: