We've been seeing a problem on builds.apache.org with some Maven jobs hanging after their first module's install goal runs, hanging all other attempts to do remoting through that slave - this hangs other jobs and ends up locking up SCM polling as well. After kohsuke looked at the thread dumps from the master (https://gist.github.com/abayer/ccd1fdafdddff9917ba0) and the maven-agent process on the slave (https://gist.github.com/abayer/2e1a777bb496584955d4), he found that there was a hung thread at https://gist.github.com/abayer/ccd1fdafdddff9917ba0#file-gistfile1-txt-L805 - I checked the code in SplittableBuildListener, checked the job config again and then realized it had "-T3" in the goals. I guessed that that could be connected, since parallelization is problematic in the first place - I changed the job to not have -T in its goals, reran it, and it kept going past the initial hang point.
builds.apache.org is still having issues right now, but I think that's just remnant confused threads from the two other similarly configured jobs that managed to start while I was debugging this and before I got rid of -T from all MavenModuleSet jobs. I'll restart shortly and confirm that this did the trick, but it looks pretty likely.