Details

    • Similar Issues:

      Description

      Would be useful to have a step which lets you throttle how many ExecutorStepExecution instances from this flow run (or job?) are permitted to run concurrently, even if there are enough free executors in the system. For example:

      def branches = [:];
      for (int i = 0; i < 100; i++) {
        branches["part${i}"] = {
          node {
            if (i % 5 == 0) {
              runSomethingQuiteSlow();
            } else {
              runSomethingQuick();
            }
          }
        };
      }
      throttleNodes(10) {
          parallel branches;
      }
      

      This would ensure that at most ten slaves are running stuff for your job at any given time. This is better than just creating ten branches, because if a slave finishes one of the quick branches it can proceed to pick up and run another branch while some other slaves remain busy with slow branches.

      The step need do nothing more than register a QueueTaskDispatcher active during its body which counts running Queue.Task instances associated with this Run (or Job) and rejects attempts to schedule more than a specified limit.

        Attachments

          Issue Links

            Activity

            Hide
            abayer Andrew Bayer added a comment -

            These are related - when I get a chance, I'm going to try to extend TCB to work within Workflow steps...

            Show
            abayer Andrew Bayer added a comment - These are related - when I get a chance, I'm going to try to extend TCB to work within Workflow steps...
            Hide
            abayer Andrew Bayer added a comment -

            So to integrate with TCB (i.e., for JENKINS-31801), what I'd need to be able to do is check a property or action or something of the like that could be attached to the Run/Job that represents the individual step execution to see if it's got a reference to a known category, and if so, check how many other Run are going on at the moment with that same property/flag/whatever. In AbstractProject-derived land, we use a JobProperty for that, but I'm not sure how that would work in Workflow-land.

            Show
            abayer Andrew Bayer added a comment - So to integrate with TCB (i.e., for JENKINS-31801 ), what I'd need to be able to do is check a property or action or something of the like that could be attached to the Run / Job that represents the individual step execution to see if it's got a reference to a known category, and if so, check how many other Run are going on at the moment with that same property/flag/whatever. In AbstractProject -derived land, we use a JobProperty for that, but I'm not sure how that would work in Workflow-land.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -
            Show
            oleg_nenashev Oleg Nenashev added a comment - TCB 2.0 has been released: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/blob/master/CHANGELOG.md#20 Jesse Glick probably this issue can be closed
            Hide
            jglick Jesse Glick added a comment -

            Oleg Nenashev according to documentation

            Throttling within a single job is not supported

            so how is this a replacement for the requested feature?

            Show
            jglick Jesse Glick added a comment - Oleg Nenashev according to documentation Throttling within a single job is not supported so how is this a replacement for the requested feature?
            Hide
            abayer Andrew Bayer added a comment -

            That's getting fixed in the docs, this does support that use case.

            Show
            abayer Andrew Bayer added a comment - That's getting fixed in the docs, this does support that use case.
            Hide
            jglick Jesse Glick added a comment -

            OK, just close when the docs reflect that.

            Show
            jglick Jesse Glick added a comment - OK, just close when the docs reflect that.
            Hide
            abayer Andrew Bayer added a comment -

            Documentation has long since been updated, so closing.

            Show
            abayer Andrew Bayer added a comment - Documentation has long since been updated, so closing.

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                jglick Jesse Glick
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: