Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44756

throttle(...) step can be ineffective at restart time

    Details

    • Similar Issues:

      Description

      See discussion at JENKINS-44747 and https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/50 - I'm legitimately not sure whether anyone will ever actually encounter it, but it's a theoretical possibility that throttled Pipeline threads could fail to actually throttle upon master restart due to the fix for JENKINS-44747.

        Attachments

          Issue Links

            Activity

            Hide
            casz Joseph Petersen added a comment -

            Your correct Jesse, my bad

            Show
            casz Joseph Petersen added a comment - Your correct Jesse, my bad
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            No plan to work on it anytime soon. Pull requests will be appreciated, reviewed and integrated if possible.

            Show
            oleg_nenashev Oleg Nenashev added a comment - No plan to work on it anytime soon. Pull requests will be appreciated, reviewed and integrated if possible.
            Hide
            basil Basil Crow added a comment -

            I observed this issue recently after restarting one of my Jenkins masters that had Throttle Concurrents in use. Throttle Concurrents was supposed to keep the # of builds that use a specific external resource down to 30. After the restart, I verified that at least 40 builds were using that external resource, indicating that the throttle was not taking effect. I only observed this after the restart; before the restart, things were fine. The end result for me as a user was that the external resource was overwhelmed and all my jobs failed. It's disappointing to me that bugs like this have existed for over a year without resolution. It effectively means the Throttle Concurrents plugin is unusable, at least in my use case.

            Show
            basil Basil Crow added a comment - I observed this issue recently after restarting one of my Jenkins masters that had Throttle Concurrents in use. Throttle Concurrents was supposed to keep the # of builds that use a specific external resource down to 30. After the restart, I verified that at least 40 builds were using that external resource, indicating that the throttle was not taking effect. I only observed this after the restart; before the restart, things were fine. The end result for me as a user was that the external resource was overwhelmed and all my jobs failed. It's disappointing to me that bugs like this have existed for over a year without resolution. It effectively means the Throttle Concurrents plugin is unusable, at least in my use case.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            > It's disappointing to me that bugs like this have existed for over a year without resolution

            Agreed, it is a disappointment. The plugin is open for adoption, and anybody is more than welcome to take an ownership or to propose a fix. Contributors move on sometimes, so plugins may stay in stale until somebody has interest to maintain them

             

            Show
            oleg_nenashev Oleg Nenashev added a comment - > It's disappointing to me that bugs like this have existed for over a year without resolution Agreed, it is a disappointment. The plugin is open for adoption, and anybody is more than welcome to take an ownership or to propose a fix. Contributors move on sometimes, so plugins may stay in stale until somebody has interest to maintain them  
            Hide
            vizzard Aron Smyk added a comment -

            It's not only during the master restart. It affects every build started before restart.

            Due to some other issues I have to restart Jenkins every week (number of open files is rising to the point where Jenkins is not usable). As a result I'm observing this issue every week, and have to cancel builds started before reboot.

            My observations are a little different than the issue description. For me, throttle for the running pipelines is forgotten during the reboot. All new Jobs after the reboot are throttling correctly but they are not respecting throttling that happened before reboot. This leads to builds deadlocks and is very frustrating because builds takes a really long time in my case.

            I did not observed throttling problem during or after the restart but I haven't done much testing either. For me, it affects every single build started before reboot, making the Pipeline durability totally useless (meaning it can resume after master restart). I have to abort such build anyway because new builds will not see throttling of that build.

            Show
            vizzard Aron Smyk added a comment - It's not only during the master restart. It affects every build started before restart. Due to some other issues I have to restart Jenkins every week (number of open files is rising to the point where Jenkins is not usable). As a result I'm observing this issue every week, and have to cancel builds started before reboot. My observations are a little different than the issue description. For me, throttle for the running pipelines is forgotten during the reboot. All new Jobs after the reboot are throttling correctly but they are not respecting throttling that happened before reboot. This leads to builds deadlocks and is very frustrating because builds takes a really long time in my case. I did not observed throttling problem during or after the restart but I haven't done much testing either. For me, it affects every single build started before reboot, making the Pipeline durability totally useless (meaning it can resume after master restart). I have to abort such build anyway because new builds will not see throttling of that build.

              People

              • Assignee:
                Unassigned
                Reporter:
                abayer Andrew Bayer
              • Votes:
                6 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated: