Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32073

Scheduling of workflow "node" allocation needs some better controls.

XMLWordPrintable

      I have a jenkins workflow job that has the option enabled to "Execute concurrent builds if necessary".

      In each of the workflows I allocate a number of nodes in serial as the workflow goes through different stages, e.g.

      {{// Single setup.
      node() {
      setup()
      }

      // Massive parallel tests.
      def tests = [:]
      for( def i=0; i < Globals.TESTS.size; i++ ) {
      def platform = Globals.TESTS[i][0]
      def target = Globals.TESTS[i][1]

      tests_parallel["${platform}_${target}"] = {
      node()

      { build_make_test(platform, target) }

      }
      }
      parallel tests

      // Single epilogue
      node()

      { notifications_and_stuff() } }}

      The problem here is that I really want to have multiple of these builds in flight at the same time and share the resources,
      but I'd like the builds that got scheduled earlier to finish first. Essentially it should be a FIFO on a build level.

      What in actuality happens is that if we have the following:

      Trigger:
      Build #1
      Build #2
      Build #3
      Build #4

      Finish:
      Build #2
      Build #1
      Build #4
      Build #3

      This seems to happen because once the jobs come to the parallel section, they pretty much all fight over the nodes and there is not real alogrithm apart from chaos?

      Or is there? I would really just like a checkbox that allows the workflow to allocate nodes based on a priority that weights the build number into consideration.

      That is, lower build number should have higher priority – so that a lower build number job never starves for a slot if there is one avaliable.

      Another thing to note is that the build I have in workflow steps through each change as they come in and I really want to have each one finish, no skipping builds or changes. And they need to come in order.

      Cheers,
      Jim

            jglick Jesse Glick
            jtilander Jim Tilander
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: