I'm attaching some screenshots from MAT for heap analysis. Not %100 positive but it appears the Metrics plugin holds data and references for Build Slaves that have been destroyed and no longer exist.
We spin up nodes ephemerally and then destory them. The Docker Plugin creates these as DockerComputer objects. Over time, we end up with a reference leak. I took a heap dump and traced the references being held and it looks like the jenkins.metrics.impl.JenkinsMetricProviderImpl is holding references to a large chunk of hashmaps that maintain data on all the build nodes ever created since the start of Jenkins service.
This is also confirmed as the Graphite reporting plugin happily transmits the data for all of these nodes (right now I cull it graphite server side).
I'm not sure if there's a way I can force the plugin to flush this data without a Jenkins server restart.
I can't upload the Hprof as even compressed it's too big for the limits established here. Happy to send it over to whomever wants to look