Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27968

Throttle concurrent builds and lockable resource do not work with block on upstream

    Details

    • Similar Issues:

      Description

      Setup

      Set Jenkins to have one node (master), and two executors.
      Create a throttle category my-category with global max 1.

      Create freestyle project A.
      Build step `sleep 10`.

      Create freestyle project B.
      Throttle on my-category.
      Build step `sleep 10`.

      Create freestyle project C.
      Throttle on my-category.
      Build step `sleep 10`.
      Post-build, trigger downstream project D.

      Create freestyle project D.
      Block on upstream jobs.
      Build step `sleep 10`.

      Trigger A, B, C, and D, in that order.

      Expected

      A starts.
      B starts.
      A finishes.
      B finishes.
      C starts.
      C finishes.
      D starts.
      D finishes.

      Actual

      A starts.
      B starts.
      A finishes.
      D starts.
      B finishes.
      C starts.
      D finishes.
      C finishes.

      D does not wait on C.

      I also tried lockable resource plugin with the same setup, but it didn't work either.

      I do not know of any workaround.

        Attachments

          Issue Links

            Activity

            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            I would say there's no bug:
            1) You do not throttle D, hence Jenkins is eligible to run it when it has a free executor
            2) "Block on upstream jobs" only prevents conflicts of RUNNING jobs, a task in the queue does not prevent D from starting
            3) C is blocked by throttling, hence Jenkins takes D

            Paul, do you agree?

            Show
            oleg_nenashev Oleg Nenashev added a comment - I would say there's no bug: 1) You do not throttle D, hence Jenkins is eligible to run it when it has a free executor 2) "Block on upstream jobs" only prevents conflicts of RUNNING jobs, a task in the queue does not prevent D from starting 3) C is blocked by throttling, hence Jenkins takes D Paul, do you agree?
            Hide
            danielbeck Daniel Beck added a comment -

            There is no bug here. Jenkins schedules a build of D because nothing says it cannot. (It would probably prefer C as its queue item is older, but that is prevented.)

            Show
            danielbeck Daniel Beck added a comment - There is no bug here. Jenkins schedules a build of D because nothing says it cannot. (It would probably prefer C as its queue item is older, but that is prevented.)
            Hide
            draperp Paul Draper added a comment - - edited

            > "Block on upstream jobs" only prevents conflicts of RUNNING jobs, a task in the queue does not prevent D from starting.

            I'm not quite sure I know what you mean. "Block on upstream jobs" will certainly prevent a job from running if its upstream is in the queue.

            Create jobs A, B, C, where C is upstream to B. Have a single executor, and start A, B, and C. Even though B was queued before C, it will run after C, because the fact that C is queued prevents B from running.

            > Jenkins schedules a build of D because nothing says it cannot.

            "Block on upstream builds" says it cannot, as D's upstream C is still in the queue.

            "When this option is checked, Jenkins will prevent the project from building when a dependency of this project is in the queue, or building. The dependencies include the direct as well as the transitive dependencies."

            Show
            draperp Paul Draper added a comment - - edited > "Block on upstream jobs" only prevents conflicts of RUNNING jobs, a task in the queue does not prevent D from starting. I'm not quite sure I know what you mean. "Block on upstream jobs" will certainly prevent a job from running if its upstream is in the queue. Create jobs A, B, C, where C is upstream to B. Have a single executor, and start A, B, and C. Even though B was queued before C, it will run after C, because the fact that C is queued prevents B from running. > Jenkins schedules a build of D because nothing says it cannot. "Block on upstream builds" says it cannot, as D's upstream C is still in the queue. "When this option is checked, Jenkins will prevent the project from building when a dependency of this project is in the queue, or building. The dependencies include the direct as well as the transitive dependencies."
            Hide
            danielbeck Daniel Beck added a comment -

            Would be interesting to know whether this has recently behaved as documented. I have a feeling this has been broken for a long time.

            Show
            danielbeck Daniel Beck added a comment - Would be interesting to know whether this has recently behaved as documented. I have a feeling this has been broken for a long time.
            Hide
            draperp Paul Draper added a comment -

            @Daniel, I suspect you are correct. It's quite possible these have never worked together.

            It is present in latest (1.609), and in the latest stable (1.596.2).

            Show
            draperp Paul Draper added a comment - @Daniel, I suspect you are correct. It's quite possible these have never worked together. It is present in latest (1.609), and in the latest stable (1.596.2).
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Javadoc documentation:
            > "True to keep builds of this project in queue when upstream projects are building. False by default to keep from breaking existing behavior."

            Jobs in the queue and pending jobs (assigned to executor, but not started) have Run.State.NOT_STARTED status (there's also Run.State.BUILDING status). Build::isBuilding() returns try for both statuses, so there's a risk of confusion.

            Paul, my bad. Your case is valid.

            The handling of the restrictions is based on Queue::getUnblockedItems(), so Jenkins ignores dependencies on blocked tasks like throttled runs.

            We just need to decide if it is a functional or a documentation bug. I'd vote fore the first option.

            Show
            oleg_nenashev Oleg Nenashev added a comment - Javadoc documentation: > "True to keep builds of this project in queue when upstream projects are building. False by default to keep from breaking existing behavior." Jobs in the queue and pending jobs (assigned to executor, but not started) have Run.State.NOT_STARTED status (there's also Run.State.BUILDING status). Build::isBuilding() returns try for both statuses, so there's a risk of confusion. Paul, my bad. Your case is valid. The handling of the restrictions is based on Queue::getUnblockedItems(), so Jenkins ignores dependencies on blocked tasks like throttled runs. We just need to decide if it is a functional or a documentation bug. I'd vote fore the first option.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Unfortunately I was unable to find time to work on plugin over last year. I decided to step down as maintainer and to mark the plugin for adoption.

            If anybody is interested, let me know

            Show
            oleg_nenashev Oleg Nenashev added a comment - Unfortunately I was unable to find time to work on plugin over last year. I decided to step down as maintainer and to mark the plugin for adoption. If anybody is interested, let me know

              People

              • Assignee:
                Unassigned
                Reporter:
                draperp Paul Draper
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: