The init script stored the PID of the java process start in /var/run/jenkins.pid. This is used when the init script stops or restarts the jenkins proces. It is also used when rotating log files via the logrotate script.

However, if I install a plugin, or update a plugin and restart jenkins from the web interface the PID is incorrect (it's the PID when jenkins first started, NOT after it was restarted via the web interface). This causes the log rotation script to not work and the init scripts to not shut down/restart jenkins.

The logrotation is what really brought this to my attention as the logs were rotated, but since the PID was incorrect jenkins was not sent the SIGALARM so that it would reopen the log files.. And as such kept logging to the now deleted log file and consequently filled up that partition.

Ideally jenkins should manage the PID itself, and just have an extra argument passed on startup.. i.e. --pidfile /var/run/jenkins.pid