Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19622

PeriodicWork can block timer thread; use ScheduledExecutorService instead

    Details

    • Similar Issues:

      Description

      A buggy plugin can block the Jenkins timer thread. This will prevent slave provisioning (NodeProvisionerInvoker) from running, among many other problems.

      AsyncPeriodicWork is designed for this purpose and should be used if fast execution cannot be guaranteed. Nonetheless, Jenkins core should defend against plugins with slow tasks (disk I/O, network) which fail to use it and use PeriodicWork instead; or which have an extension which is called from within a core PeriodicWork, such as a Trigger, and fail to spawn a thread for their body.

      It would be better to deprecate Trigger.timer (which unfortunately still leaves a useless TimerThread lying around) and switch to a ScheduledExecutorService, which allows concurrency in case one task has not finished by the time the next is ready to run.

      It would also be useful to have an administrative monitor which reports non-async PeriodicWork implementations that are taking "too long", defined either as still being running when the next task is ready to go, or which are taking more than e.g. 10% of their defined recurrence period, etc.

        Attachments

          Issue Links

            Activity

            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Ryan Campbell
            Path:
            core/src/main/groovy/hudson/util/LoadMonitor.groovy
            core/src/main/java/hudson/model/AperiodicWork.java
            core/src/main/java/hudson/model/ComputerSet.java
            core/src/main/java/hudson/model/PeriodicWork.java
            core/src/main/java/hudson/model/Queue.java
            core/src/main/java/hudson/node_monitors/AbstractNodeMonitorDescriptor.java
            core/src/main/java/hudson/triggers/Trigger.java
            core/src/main/java/hudson/util/DoubleLaunchChecker.java
            core/src/main/java/jenkins/model/Jenkins.java
            core/src/main/java/jenkins/util/Timer.java
            core/src/test/java/jenkins/util/TimerTest.java
            http://jenkins-ci.org/commit/jenkins/565ced0dba75c3f198c5cf9b9df247941df6ad46
            Log:
            [FIXES JENKINS-19622] Replace hudson.model.Trigger#timer with jenkins.util.Timer, which exposes a ScheduledExecutorService. Up to 10 additional threads will be created to run pending tasks even if running tasks are blocked.

            Also, move static initialization of timers to the classes which require them.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ryan Campbell Path: core/src/main/groovy/hudson/util/LoadMonitor.groovy core/src/main/java/hudson/model/AperiodicWork.java core/src/main/java/hudson/model/ComputerSet.java core/src/main/java/hudson/model/PeriodicWork.java core/src/main/java/hudson/model/Queue.java core/src/main/java/hudson/node_monitors/AbstractNodeMonitorDescriptor.java core/src/main/java/hudson/triggers/Trigger.java core/src/main/java/hudson/util/DoubleLaunchChecker.java core/src/main/java/jenkins/model/Jenkins.java core/src/main/java/jenkins/util/Timer.java core/src/test/java/jenkins/util/TimerTest.java http://jenkins-ci.org/commit/jenkins/565ced0dba75c3f198c5cf9b9df247941df6ad46 Log: [FIXES JENKINS-19622] Replace hudson.model.Trigger#timer with jenkins.util.Timer, which exposes a ScheduledExecutorService. Up to 10 additional threads will be created to run pending tasks even if running tasks are blocked. Also, move static initialization of timers to the classes which require them.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/jenkins/util/Timer.java
            http://jenkins-ci.org/commit/jenkins/53317a8045ec797e0fd718ec1599be95c18a2ba5
            Log:
            JENKINS-19622 Merged #989: allow background tasks to run simultaneously, preventing task blockage.

            Compare: https://github.com/jenkinsci/jenkins/compare/48efa739c7d6...53317a8045ec

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/jenkins/util/Timer.java http://jenkins-ci.org/commit/jenkins/53317a8045ec797e0fd718ec1599be95c18a2ba5 Log: JENKINS-19622 Merged #989: allow background tasks to run simultaneously, preventing task blockage. Compare: https://github.com/jenkinsci/jenkins/compare/48efa739c7d6...53317a8045ec
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #3051
            [FIXES JENKINS-19622] Replace hudson.model.Trigger#timer with jenkins.util.Timer, which exposes a ScheduledExecutorService. Up to 10 additional threads will be created to run pending tasks even if running tasks are blocked. (Revision 565ced0dba75c3f198c5cf9b9df247941df6ad46)
            JENKINS-19622 Merged #989: allow background tasks to run simultaneously, preventing task blockage. (Revision 53317a8045ec797e0fd718ec1599be95c18a2ba5)

            Result = SUCCESS
            Ryan Campbell : 565ced0dba75c3f198c5cf9b9df247941df6ad46
            Files :

            • core/src/main/java/hudson/triggers/Trigger.java
            • core/src/test/java/jenkins/util/TimerTest.java
            • core/src/main/java/hudson/model/ComputerSet.java
            • core/src/main/groovy/hudson/util/LoadMonitor.groovy
            • core/src/main/java/hudson/util/DoubleLaunchChecker.java
            • core/src/main/java/hudson/node_monitors/AbstractNodeMonitorDescriptor.java
            • core/src/main/java/hudson/model/PeriodicWork.java
            • core/src/main/java/jenkins/util/Timer.java
            • core/src/main/java/jenkins/model/Jenkins.java
            • core/src/main/java/hudson/model/AperiodicWork.java
            • core/src/main/java/hudson/model/Queue.java

            Jesse Glick : 53317a8045ec797e0fd718ec1599be95c18a2ba5
            Files :

            • core/src/main/java/jenkins/util/Timer.java
            • changelog.html
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #3051 [FIXES JENKINS-19622] Replace hudson.model.Trigger#timer with jenkins.util.Timer, which exposes a ScheduledExecutorService. Up to 10 additional threads will be created to run pending tasks even if running tasks are blocked. (Revision 565ced0dba75c3f198c5cf9b9df247941df6ad46) JENKINS-19622 Merged #989: allow background tasks to run simultaneously, preventing task blockage. (Revision 53317a8045ec797e0fd718ec1599be95c18a2ba5) Result = SUCCESS Ryan Campbell : 565ced0dba75c3f198c5cf9b9df247941df6ad46 Files : core/src/main/java/hudson/triggers/Trigger.java core/src/test/java/jenkins/util/TimerTest.java core/src/main/java/hudson/model/ComputerSet.java core/src/main/groovy/hudson/util/LoadMonitor.groovy core/src/main/java/hudson/util/DoubleLaunchChecker.java core/src/main/java/hudson/node_monitors/AbstractNodeMonitorDescriptor.java core/src/main/java/hudson/model/PeriodicWork.java core/src/main/java/jenkins/util/Timer.java core/src/main/java/jenkins/model/Jenkins.java core/src/main/java/hudson/model/AperiodicWork.java core/src/main/java/hudson/model/Queue.java Jesse Glick : 53317a8045ec797e0fd718ec1599be95c18a2ba5 Files : core/src/main/java/jenkins/util/Timer.java changelog.html
            Hide
            jglick Jesse Glick added a comment -

            The full pull request in case this is accepted for backport: https://github.com/jenkinsci/jenkins/compare/48efa739c7d6...53317a8045ec

            Show
            jglick Jesse Glick added a comment - The full pull request in case this is accepted for backport: https://github.com/jenkinsci/jenkins/compare/48efa739c7d6...53317a8045ec
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Ryan Campbell
            Path:
            core/src/main/groovy/hudson/util/LoadMonitor.groovy
            core/src/main/java/hudson/model/AperiodicWork.java
            core/src/main/java/hudson/model/ComputerSet.java
            core/src/main/java/hudson/model/PeriodicWork.java
            core/src/main/java/hudson/model/Queue.java
            core/src/main/java/hudson/node_monitors/AbstractNodeMonitorDescriptor.java
            core/src/main/java/hudson/triggers/Trigger.java
            core/src/main/java/hudson/util/DoubleLaunchChecker.java
            core/src/main/java/jenkins/model/Jenkins.java
            core/src/main/java/jenkins/util/Timer.java
            core/src/test/java/jenkins/util/TimerTest.java
            http://jenkins-ci.org/commit/jenkins/44ee0bb39a134fd12745ccb5e922af93266ad4e2
            Log:
            [FIXES JENKINS-19622] Replace hudson.model.Trigger#timer with jenkins.util.Timer, which exposes a ScheduledExecutorService. Up to 10 additional threads will be created to run pending tasks even if running tasks are blocked.

            Also, move static initialization of timers to the classes which require them.

            (cherry picked from commit 565ced0dba75c3f198c5cf9b9df247941df6ad46)

            Conflicts:
            core/src/main/java/hudson/model/Queue.java

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ryan Campbell Path: core/src/main/groovy/hudson/util/LoadMonitor.groovy core/src/main/java/hudson/model/AperiodicWork.java core/src/main/java/hudson/model/ComputerSet.java core/src/main/java/hudson/model/PeriodicWork.java core/src/main/java/hudson/model/Queue.java core/src/main/java/hudson/node_monitors/AbstractNodeMonitorDescriptor.java core/src/main/java/hudson/triggers/Trigger.java core/src/main/java/hudson/util/DoubleLaunchChecker.java core/src/main/java/jenkins/model/Jenkins.java core/src/main/java/jenkins/util/Timer.java core/src/test/java/jenkins/util/TimerTest.java http://jenkins-ci.org/commit/jenkins/44ee0bb39a134fd12745ccb5e922af93266ad4e2 Log: [FIXES JENKINS-19622] Replace hudson.model.Trigger#timer with jenkins.util.Timer, which exposes a ScheduledExecutorService. Up to 10 additional threads will be created to run pending tasks even if running tasks are blocked. Also, move static initialization of timers to the classes which require them. (cherry picked from commit 565ced0dba75c3f198c5cf9b9df247941df6ad46) Conflicts: core/src/main/java/hudson/model/Queue.java
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/jenkins/util/Timer.java
            http://jenkins-ci.org/commit/jenkins/56a424073c886d86cd700168bbd30740a5d1713f
            Log:
            JENKINS-19622 Merged #989: allow background tasks to run simultaneously, preventing task blockage.

            (cherry picked from commit 53317a8045ec797e0fd718ec1599be95c18a2ba5)

            Conflicts:
            changelog.html

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/util/Timer.java http://jenkins-ci.org/commit/jenkins/56a424073c886d86cd700168bbd30740a5d1713f Log: JENKINS-19622 Merged #989: allow background tasks to run simultaneously, preventing task blockage. (cherry picked from commit 53317a8045ec797e0fd718ec1599be95c18a2ba5) Conflicts: changelog.html

              People

              • Assignee:
                recampbell Ryan Campbell
                Reporter:
                jglick Jesse Glick
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: