The background I can give is vague as its specific to how we use hudson but I
hope it helps:

1) a Job is configured to be built by ANT and run in a different JVM version
than what hudson is running on

2) ant starts up a weblogic JVM instance and a separate JUNIT JVM instance

3) The junit tests talk to remote service running on the J2EE app server

4) Because of an underlying database issue, the JUNIT instance eventually has a
java.lang.OutOfMemoryError exception

5) This hangs the job execution

6) Using hudson GUI we manually kill the job

Expected=The separate JVM instances to be correctly killed off
Actual= They seem to be still running and have to be manually killed off

As I said a scenario very specific to how we use hudson and may very well be
difficult for you to repro.