Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40284

Fair Queuing strategy levels out to an identical weight for all jobs

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: prioritysorter-plugin
    • Labels:
      None
    • Environment:
      Jenkins 1.609.
      Jenkins Enterprise extensions (Folders).
      Priority Sorter Plugin 3.4.1.
    • Similar Issues:

      Description

      It seems the Fair Queuing strategy does not work, at least for us.

      In the global configuration, we specified the Fair Queuing stragegy, with 5 priorities, the default being 1.

      We created two folders, priority1 and priority2.

      De defined the global Job Priorities so that priority 1 is assigned to (jobs in) folder priority1, and priority 2 is assigned to (jobs in) folder priority2.

      In each folder, we created 4 jobs which simply wait for 30 seconds before terminating in success ("sleep 30" in shell script).

      We specified only one executor in order to have jobs queued.

      We manually start the 4 jobs in folder priority1 and immediately after, we start the 4 jobs in folder priority2, before the first jobs in folder priority1 terminates.

      We expect to have in the queue the jobs from the 2 folders intermixed, which we do.

      But if we repeat the experiment a few times, we end up in a state where jobs get queued in the order in which they are started, meaning that the Priority Sorter Plugin does not do its jos properly.

      I have attached part of a log produced by the plugin, in which we see that the weight assigned to jobs levels out at 0.00007 and does not increase anymore. I believe this is the reason for the problem.

      By looking at the code here https://github.com/rvesse/priority-sorter-plugin/blob/master/src/main/java/jenkins/advancedqueue/sorter/strategy/FQBaseStrategy.java I conclude there the imprecise floating point arithmetic is the culprit.

      I wrote this simple Java test class:

      public class TestFloat {
      	public static void main(String[] args) {
      		Float step = 0.00001F;
      		Float max = 0F;
      		double weight;
      
      		for (int i = 0; i < 10; i++) {
      			System.out.println("Max: " + max);
      			weight = Math.ceil(max / step) * step;
      			System.out.println("Weight: " + weight);
      			if (weight <= max) {
      				weight += step;
      			}
      			System.out.println("Weight: " + weight);
      			max = (float)weight;
      
      			System.out.println();
      		}
      	}
      }
      

      It mimmics what I understand the code in the plugin does, and it reproduces the same behavior.

        Attachments

          Activity

          Hide
          allan_burdajewicz Allan BURDAJEWICZ added a comment -

          The weight cap to 6.999999823165126E-5 after building 7 items.

          The weight with double precision is equal to the minimumWeightToAssign: https://github.com/jenkinsci/priority-sorter-plugin/blob/master/src/main/java/jenkins/advancedqueue/sorter/strategy/FQBaseStrategy.java#L78

          We should probably compare the (float)weight to minimumWeightToAssign as (float)weight is the value we return.

          Show
          allan_burdajewicz Allan BURDAJEWICZ added a comment - The weight cap to 6.999999823165126E-5 after building 7 items. The weight with double precision is equal to the minimumWeightToAssign : https://github.com/jenkinsci/priority-sorter-plugin/blob/master/src/main/java/jenkins/advancedqueue/sorter/strategy/FQBaseStrategy.java#L78 We should probably compare the (float) weight to minimumWeightToAssign as (float)weight is the value we return.
          Hide
          allan_burdajewicz Allan BURDAJEWICZ added a comment -

          There is an open PR that fixes that problem: https://github.com/jenkinsci/priority-sorter-plugin/pull/32
          Magnus Sandberg Could this be reviewed/merged ?

          Show
          allan_burdajewicz Allan BURDAJEWICZ added a comment - There is an open PR that fixes that problem: https://github.com/jenkinsci/priority-sorter-plugin/pull/32 Magnus Sandberg Could this be reviewed/merged ?
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          src/main/java/jenkins/advancedqueue/sorter/strategy/FQBaseStrategy.java
          src/test/java/jenkins/advancedqueue/sorter/strategy/FQStrategyTest.java
          src/test/java/jenkins/advancedqueue/sorter/strategy/WFQStrategyTest.java
          http://jenkins-ci.org/commit/priority-sorter-plugin/5e5657d7d246d12a2d121739f42be45749042b7b
          Log:
          JENKINS-40284 - Fix blocked weights with FQ strategies (#32)

          JENKINS-40284 - Fix blocked weights with FQ strategies

          Compare: https://github.com/jenkinsci/priority-sorter-plugin/compare/5d333f6bd1fc...5e5657d7d246

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/jenkins/advancedqueue/sorter/strategy/FQBaseStrategy.java src/test/java/jenkins/advancedqueue/sorter/strategy/FQStrategyTest.java src/test/java/jenkins/advancedqueue/sorter/strategy/WFQStrategyTest.java http://jenkins-ci.org/commit/priority-sorter-plugin/5e5657d7d246d12a2d121739f42be45749042b7b Log: JENKINS-40284 - Fix blocked weights with FQ strategies (#32) JENKINS-40284 - Fix blocked weights with FQ strategies Compare: https://github.com/jenkinsci/priority-sorter-plugin/compare/5d333f6bd1fc...5e5657d7d246
          Hide
          allan_burdajewicz Allan BURDAJEWICZ added a comment -

          Released in 3.5.0

          Show
          allan_burdajewicz Allan BURDAJEWICZ added a comment - Released in 3.5.0

            People

            • Assignee:
              emsa23 Magnus Sandberg
              Reporter:
              draymond David Raymond
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: