Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22044

Denial of service by browsing node build history

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core
    • None

      I have a setup with a node running a lot of short jobs for which we want to keep the history for a year or so. That is few hundred of thousands build kept.

      Whenever one browse GET /ci/computer/MyNode/builds, Jenkins lazy load all the jobs that happened on that node and apparently parse most of the builds. That makes a RequestHandlerThread eating 100% CPU for quite a long time.

      The front end web proxy / web browser eventually timeout and a user would usually refresh the page several time, creating more RequestHandlerThread trying to lazy load the whole build history.

      End results: the pool of RequestHandlerThread is filled with long running queries. The web interface is no more accessible. All core are at 100% usage making the box unusable.

      Attached is a stacktrace of a RequestHandlerThread.

      Possible suggestions:

      • implement a lock mechanism to avoid several threads to do the exact same long running task. If the exact same query is done it should wait for the first one to complete and give the same result
      • limit the number of builds shown on the node/build page
      • have a way to easily set a timeout for RequestHandlerThread so it dies after X minutes.

            Unassigned Unassigned
            hashar Antoine Musso
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: