Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-25326

Throttle Concurrent Builds doesn't work when builds are in cloudbees folders

    Details

    • Similar Issues:

      Description

      We recently organized our builds into folders, and we noticed that our concurrent build throttling wasn't working. It doesn't work between folders or for builds within the same folder. It only works when it's in the main workspace.

        Attachments

          Activity

          Hide
          danielbeck Daniel Beck added a comment - - edited

          I confirm the reported behavior on 1.586, 1.8.4, and Folders 4.6.1.

          Set up a category 'foo' with 1 maximum concurrent build.
          Create two freestyle jobs JENKINS-25326-one and JENKINS-25326-two in the same folder 'experiments'. Enable throttling and assign the category 'foo'. Make both of them 'sleep 30' in a shell build step.
          Trigger both of them.
          They will run in parallel.

          Show
          danielbeck Daniel Beck added a comment - - edited I confirm the reported behavior on 1.586, 1.8.4, and Folders 4.6.1. Set up a category 'foo' with 1 maximum concurrent build. Create two freestyle jobs JENKINS-25326 -one and JENKINS-25326 -two in the same folder 'experiments'. Enable throttling and assign the category 'foo'. Make both of them 'sleep 30' in a shell build step. Trigger both of them. They will run in parallel.
          Hide
          danielbeck Daniel Beck added a comment - - edited

          There's something weird going on. It seems to only occur with newly created jobs. Jobs that existed when Jenkins was last started are not affected.

          Show
          danielbeck Daniel Beck added a comment - - edited There's something weird going on. It seems to only occur with newly created jobs. Jobs that existed when Jenkins was last started are not affected.
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          No response...
          AFAIK, it has been fixed in 1.8.2

          Show
          oleg_nenashev Oleg Nenashev added a comment - No response... AFAIK, it has been fixed in 1.8.2
          Hide
          danielbeck Daniel Beck added a comment -

          Oleg: I confirmed the report and added additional information. There's nothing to respond to.

          Show
          danielbeck Daniel Beck added a comment - Oleg: I confirmed the report and added additional information. There's nothing to respond to.
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          Yes, my bad. I was going to close JENKINS-22049, but selected a wrong tab in the browser.
          I'm investigating the issue now

          Show
          oleg_nenashev Oleg Nenashev added a comment - Yes, my bad. I was going to close JENKINS-22049 , but selected a wrong tab in the browser. I'm investigating the issue now
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          Confirmed. It also appears for moved projects.

          Show
          oleg_nenashev Oleg Nenashev added a comment - Confirmed. It also appears for moved projects.
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          This is a regression in the core. The case works well on 1.554.1

          Show
          oleg_nenashev Oleg Nenashev added a comment - This is a regression in the core. The case works well on 1.554.1
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          I cannot reproduce the behavior on 1.585 with installed Matrix Project Plugin.
          I tried Daniel Beck's reproduction steps with additional states (like new category), but nothing helps to reproduce the issue.

          Are you sure that Matrix Project plugin was installed?

          Show
          oleg_nenashev Oleg Nenashev added a comment - I cannot reproduce the behavior on 1.585 with installed Matrix Project Plugin. I tried Daniel Beck 's reproduction steps with additional states (like new category), but nothing helps to reproduce the issue. Are you sure that Matrix Project plugin was installed?
          Hide
          danielbeck Daniel Beck added a comment -

          Oleg: Matrix Project was installed for me.

          That said I just set up another pristine instance and can no longer repro. I tried twice from scratch and the plugin works.

          Wouldn't resolve this yet though.

          Show
          danielbeck Daniel Beck added a comment - Oleg: Matrix Project was installed for me. That said I just set up another pristine instance and can no longer repro. I tried twice from scratch and the plugin works. Wouldn't resolve this yet though.
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          Yes, let's leave the issue open.
          Something strange happens in any case, so we just need a reproduction scenario. Hopefully, somebody provides an additional info

          Show
          oleg_nenashev Oleg Nenashev added a comment - Yes, let's leave the issue open. Something strange happens in any case, so we just need a reproduction scenario. Hopefully, somebody provides an additional info
          Hide
          ulich Timo Ulich added a comment -

          I am experiencing the same problem with jenkins 1.596.1 LTS, cloudbees folder plugin 4.7 and Throttle Concurrent Builds Plug-in 1.8.4 (Matrix Project Plugin is installed).

          This happens with newly created jobs and jobs that existed before restarting jenkins.

          Can I provide any other information?

          Show
          ulich Timo Ulich added a comment - I am experiencing the same problem with jenkins 1.596.1 LTS, cloudbees folder plugin 4.7 and Throttle Concurrent Builds Plug-in 1.8.4 (Matrix Project Plugin is installed). This happens with newly created jobs and jobs that existed before restarting jenkins. Can I provide any other information?
          Hide
          scorpio vijay konenki added a comment - - edited

          I am experiencing the same issue too with jenkins 1.611, cloudbees folder plugin from 4.6.1 to 4.8 and throttle concurrent build 1.8.4.

          I created two free style projects in the top level and tested for concurrency, one job was waiting for the other to complete.

          I moved both the jobs into a folder and started both at the same time, now the concurrency doesnot work. Both the jobs started in parallel

          Please let me know if you need more details.

          Thanks

          Show
          scorpio vijay konenki added a comment - - edited I am experiencing the same issue too with jenkins 1.611, cloudbees folder plugin from 4.6.1 to 4.8 and throttle concurrent build 1.8.4. I created two free style projects in the top level and tested for concurrency, one job was waiting for the other to complete. I moved both the jobs into a folder and started both at the same time, now the concurrency doesnot work. Both the jobs started in parallel Please let me know if you need more details. Thanks
          Hide
          ifwui Austin Ward added a comment -

          I am also seeing this with Jenkins 1.609.1 with Throttle concurrent builds 1.8.4 and CloudBess Folder Plugin 4.9

          Show
          ifwui Austin Ward added a comment - I am also seeing this with Jenkins 1.609.1 with Throttle concurrent builds 1.8.4 and CloudBess Folder Plugin 4.9
          Hide
          ifwui Austin Ward added a comment - - edited

          This seems to be occurring in Line 188 of ThrottleJobProperty.java:

                              if (/* not deleted */getItem(p.getParent(), p.getName()) == p && /* has not since been reconfigured */ p.getProperty(ThrottleJobProperty.class) == t) {
                                  categoryProjects.add(p);
                                  if (p instanceof MatrixProject && t.isThrottleMatrixConfigurations()) {
                                      for (MatrixConfiguration mc : ((MatrixProject)p).getActiveConfigurations()) {
                                          categoryProjects.add((AbstractProject<?,?>)mc);
                                      }
                                  }
                              }
          
          getItem(p.getParent(), p.getName())
          

          Is returning Null when the AbstractProject's parent is a Folder.

          Show
          ifwui Austin Ward added a comment - - edited This seems to be occurring in Line 188 of ThrottleJobProperty.java: if ( /* not deleted */ getItem(p.getParent(), p.getName()) == p && /* has not since been reconfigured */ p.getProperty(ThrottleJobProperty.class) == t) { categoryProjects.add(p); if (p instanceof MatrixProject && t.isThrottleMatrixConfigurations()) { for (MatrixConfiguration mc : ((MatrixProject)p).getActiveConfigurations()) { categoryProjects.add((AbstractProject<?,?>)mc); } } } getItem(p.getParent(), p.getName()) Is returning Null when the AbstractProject's parent is a Folder.
          Hide
          ikedam ikedam added a comment -

          This looks a permission issue.
          Steps to reproduce:

          1. Launch Jenkins 1.642.1.
          2. Install Cloudbees Folder 5.1
          3. Install Throttle Concurrent Builds 1.8.4
          4. Go to Manage Jenkins > Configure System, and add a new Multi-Project Throttle Category:
            • Category Name: category
            • Maximum Total Concurrent Builds: 0
            • Maximum Concurrent Builds Per Node: 1
          5. Create a folder "folder"
          6. Create a Freestyle project "test"
            • Check "Throttle Concurent Builds":
              • Check "Throttle this project as part of one or more categories"
              • Maximum Total Concurrent Builds: 0
              • Maximum Concurrent Builds Per Node: 1
              • Multi-Project Throttle Category: category
            • Check "Execute concurrent builds if necessary"
            • Add "Execute shell":
              sleep 10
              
          7. Start a new build of "folder/test"
          8. After the build started, start a new build of test again.
          9. Result: test #2 doesn't start till #1 finish.
          10. Go to Manage Jenkins > Configure Global Security and enable security.
            • Check "Enable security"
            • Check "Jenkins’ own user database"
              • Check "Allow users to sign up"
            • Check "Matrix-based security"
              • No permissions for Anonymous
              • All permissions for "admin"
          11. Login as admin (signing up)
          12. Start a new build of "folder/test"
          13. After the build started, start a new build of test again.
          14. Result: test #3 and test #4 runs at the same time (throttle-concurrent-builds doesn't work)

          This is caused for throttle-cuncurrent-builds works as anonymous when it manages build queues,
          and they can't access projects inside folders.
          For projects outside folders, throttle-concurrent-builds can access them with a special method
          provided by Jenkins core that doesn't perform permission checks.

          Workaround: Add Overall/Read and Job/Read permissions to anonymous.

          This is critical for me, and I'll work on this issue.

          Show
          ikedam ikedam added a comment - This looks a permission issue. Steps to reproduce: Launch Jenkins 1.642.1. Install Cloudbees Folder 5.1 Install Throttle Concurrent Builds 1.8.4 Go to Manage Jenkins > Configure System, and add a new Multi-Project Throttle Category: Category Name: category Maximum Total Concurrent Builds: 0 Maximum Concurrent Builds Per Node: 1 Create a folder "folder" Create a Freestyle project "test" Check "Throttle Concurent Builds": Check "Throttle this project as part of one or more categories" Maximum Total Concurrent Builds: 0 Maximum Concurrent Builds Per Node: 1 Multi-Project Throttle Category: category Check "Execute concurrent builds if necessary" Add "Execute shell": sleep 10 Start a new build of "folder/test" After the build started, start a new build of test again. Result: test #2 doesn't start till #1 finish. Go to Manage Jenkins > Configure Global Security and enable security. Check "Enable security" Check "Jenkins’ own user database" Check "Allow users to sign up" Check "Matrix-based security" No permissions for Anonymous All permissions for "admin" Login as admin (signing up) Start a new build of "folder/test" After the build started, start a new build of test again. Result: test #3 and test #4 runs at the same time (throttle-concurrent-builds doesn't work) This is caused for throttle-cuncurrent-builds works as anonymous when it manages build queues, and they can't access projects inside folders. For projects outside folders, throttle-concurrent-builds can access them with a special method provided by Jenkins core that doesn't perform permission checks. Workaround: Add Overall/Read and Job/Read permissions to anonymous. This is critical for me, and I'll work on this issue.
          Hide
          ikedam ikedam added a comment -
          Show
          ikedam ikedam added a comment - Created a pull request: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/35
          Hide
          ikedam ikedam added a comment -
          Show
          ikedam ikedam added a comment - Created a new pull request: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/37
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: ikedam
          Path:
          pom.xml
          src/test/java/hudson/plugins/throttleconcurrents/ThrottleIntegrationTest.java
          src/test/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcherTest.java
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/d2f6ca409155c32dcb79d32c835614b95d9afc38
          Log:
          JENKINS-25326 Added a test to reproduce JENKINS-25326

          Throttling doesn't work for projects in cloudbees-folder with Jenkins >= 1.536.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: pom.xml src/test/java/hudson/plugins/throttleconcurrents/ThrottleIntegrationTest.java src/test/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcherTest.java http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/d2f6ca409155c32dcb79d32c835614b95d9afc38 Log: JENKINS-25326 Added a test to reproduce JENKINS-25326 Throttling doesn't work for projects in cloudbees-folder with Jenkins >= 1.536.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: ikedam
          Path:
          src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/09234147a9bf23de0cd40cfa420cb8d4e97a4a6e
          Log:
          [FIXED JENKINS-25326] Elevates to SYSTEM when handling throttling in executor threads.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/09234147a9bf23de0cd40cfa420cb8d4e97a4a6e Log: [FIXED JENKINS-25326] Elevates to SYSTEM when handling throttling in executor threads.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: ikedam
          Path:
          pom.xml
          src/test/java/hudson/plugins/throttleconcurrents/ThrottleIntegrationTest.java
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/5b5728a069df14df1ce48377682b74ed338605df
          Log:
          JENKINS-25326 Reverted changes required for the test to reproduce JENKINS-25326.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: pom.xml src/test/java/hudson/plugins/throttleconcurrents/ThrottleIntegrationTest.java http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/5b5728a069df14df1ce48377682b74ed338605df Log: JENKINS-25326 Reverted changes required for the test to reproduce JENKINS-25326 .
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          pom.xml
          src/test/java/hudson/plugins/throttleconcurrents/ThrottleIntegrationTest.java
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/dfb3d2f13c139c430648556ece1f6b67d016dedb
          Log:
          JENKINS-25326 - Enable folder tests created by @ikedam

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: pom.xml src/test/java/hudson/plugins/throttleconcurrents/ThrottleIntegrationTest.java http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/dfb3d2f13c139c430648556ece1f6b67d016dedb Log: JENKINS-25326 - Enable folder tests created by @ikedam
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          pom.xml
          src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java
          src/test/java/hudson/plugins/throttleconcurrents/ThrottleIntegrationTest.java
          src/test/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcherTest.java
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/c823d8d9284aa6d0e645b7c2a7702a4d1d354e2c
          Log:
          Merge pull request #41 from oleg-nenashev/feature/JENKINS-25326_ExecutorPermission

          JENKINS-25326 - Elevate user to system during build throttling

          Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/e1aeb176b649...c823d8d9284a

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: pom.xml src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java src/test/java/hudson/plugins/throttleconcurrents/ThrottleIntegrationTest.java src/test/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcherTest.java http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/c823d8d9284aa6d0e645b7c2a7702a4d1d354e2c Log: Merge pull request #41 from oleg-nenashev/feature/ JENKINS-25326 _ExecutorPermission JENKINS-25326 - Elevate user to system during build throttling Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/e1aeb176b649...c823d8d9284a
          Hide
          suzannehamilton Suzanne Hamilton added a comment -

          Really sorry for accidentally updating this ticket! I hit a keyboard shortcut and must have pressed the wrong key because JIRA assigned the ticket to me. I've unassigned it again.

          Show
          suzannehamilton Suzanne Hamilton added a comment - Really sorry for accidentally updating this ticket! I hit a keyboard shortcut and must have pressed the wrong key because JIRA assigned the ticket to me. I've unassigned it again.

            People

            • Assignee:
              ikedam ikedam
              Reporter:
              gailterman Gail Terman
            • Votes:
              3 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: