Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32073

Scheduling of workflow "node" allocation needs some better controls.

    Details

    • Similar Issues:

      Description

      I have a jenkins workflow job that has the option enabled to "Execute concurrent builds if necessary".

      In each of the workflows I allocate a number of nodes in serial as the workflow goes through different stages, e.g.

      {{// Single setup.
      node()

      { setup() }

      // Massive parallel tests.
      def tests = [:]
      for( def i=0; i < Globals.TESTS.size; i++ ) {
      def platform = Globals.TESTS[i][0]
      def target = Globals.TESTS[i][1]

      tests_parallel["${platform}_${target}"] = {
      node()

      { build_make_test(platform, target) }

      }
      }
      parallel tests

      // Single epilogue
      node()

      { notifications_and_stuff() }

      }}

      The problem here is that I really want to have multiple of these builds in flight at the same time and share the resources,
      but I'd like the builds that got scheduled earlier to finish first. Essentially it should be a FIFO on a build level.

      What in actuality happens is that if we have the following:

      Trigger:
      Build #1
      Build #2
      Build #3
      Build #4

      Finish:
      Build #2
      Build #1
      Build #4
      Build #3

      This seems to happen because once the jobs come to the parallel section, they pretty much all fight over the nodes and there is not real alogrithm apart from chaos?

      Or is there? I would really just like a checkbox that allows the workflow to allocate nodes based on a priority that weights the build number into consideration.

      That is, lower build number should have higher priority – so that a lower build number job never starves for a slot if there is one avaliable.

      Another thing to note is that the build I have in workflow steps through each change as they come in and I really want to have each one finish, no skipping builds or changes. And they need to come in order.

      Cheers,
      Jim

        Attachments

          Activity

          Hide
          jglick Jesse Glick added a comment -

          Probably solvable with the stage step, though I would need to see a self-contained demo to really evaluate.

          Show
          jglick Jesse Glick added a comment - Probably solvable with the stage step, though I would need to see a self-contained demo to really evaluate.
          Hide
          jtilander Jim Tilander added a comment - - edited

          I have the following script:

          jobs = [:]
          for( def i=0; i < 100; i++ ) {
          	jobs["my_job_${i}"] = {
          		node('small') {
          			println "Now executing small job ${i}"
          			bat 'ping -n 60 8.8.8.8'
          		}
          	}
          }
          
          stage 'Setup'
          node('small') {
          	println "Setup wide jobs..."
          	bat 'ping -n 10 8.8.8.8'
          }
          
          stage 'Wide'
          parallel jobs
          
          stage 'Join'
          node('small') {
          	println "Joining results"
          	bat 'ping -n 10 8.8.8.8'
          }
          

          This produces the pipeline view I attached as a .png above. I would want to have the older batches finish first, before the more recent ones start. But now it seems pretty random from which batch the smaller jobs are picked up from.

          Show
          jtilander Jim Tilander added a comment - - edited I have the following script: jobs = [:] for ( def i=0; i < 100; i++ ) { jobs[ "my_job_${i}" ] = { node( 'small' ) { println "Now executing small job ${i}" bat 'ping -n 60 8.8.8.8' } } } stage 'Setup' node( 'small' ) { println "Setup wide jobs..." bat 'ping -n 10 8.8.8.8' } stage 'Wide' parallel jobs stage 'Join' node( 'small' ) { println "Joining results" bat 'ping -n 10 8.8.8.8' } This produces the pipeline view I attached as a .png above. I would want to have the older batches finish first, before the more recent ones start. But now it seems pretty random from which batch the smaller jobs are picked up from.
          Hide
          jglick Jesse Glick added a comment -

          Offhand, probably a task for the proposed milestone step.

          Show
          jglick Jesse Glick added a comment - Offhand, probably a task for the proposed milestone step.

            People

            • Assignee:
              jglick Jesse Glick
              Reporter:
              jtilander Jim Tilander
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: