Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7296

Throttle a matrix job build based upon user criteria

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • matrix-project-plugin
    • None
    • Any OS

      Some multi-configuration/matrix job can have a large number of jobs that get spawned. Often, getting through all the configurations as quickly as possible is desirable. But some of the configurations are blocking against each other. Perhaps some of the configurations use the same remote database, or use the same TCP/IP port for a test or build, or perhaps part of the build process relies on a tool that cannot have multiple instances running. There is already a "Run each configuration sequentially" option, but this makes the job run at the slowest possible rate.

      Some possible ways of throttling the job:
      1) Allow only one job to execute per axis value on selected axis. For example, given axis Alpha with A and B values and axis Numeric with 1 and 2 values; axis Alpha is marked to run one job per value. So when the run kicks off, config A1 could run with B1 or B2, but A2 would have to wait until A1 finished; likewise, B2 and B1 could not run at the same time.

      2) Allow the user to specify multiple Groovy rules like the Combination Filter that evaluate to True/False. When the matrix job is determining if a particular configuration can be run, have it evaluate the rules. If one rule (or maybe all rules) evaluates to true, then the configuration can be run at that time. The Groovy rules would probably have to be extended to allow the rule access to the list of running jobs. For example, a rule like "(Alpha == "A").Count < 1" might be desirable; the .Count would be a count of currently running configurations that match that criteria.

      3) Allow each configuration to construct a alphanumeric "lock" based on it's axis values. Every lock could only have one running configuration associated with it. Using the example from 1): If the lock value was defined as "$ALPHA-$NUMERIC", then the actual locks would be "A-1", "A-2", etc. This particular example would be the same as sequential processing. But if the matrix used a lock like "$ALPHA", the result would be similar to allowing only one configuration to run per Alpha axis value. NOTE: This approach might be implementable via the "Locks and Latches" plug-in if the plugin allowed for dynamically specifying the name of a lock via parameters.

            Unassigned Unassigned
            jswager jswager
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: