Just posted a fix here: https://github.com/jenkinsci/jenkins/pull/408
The problem is, that swarm clients from the Jenkins.slaves array are not stored into the config.xml (being an EphemeralNode). But when Reloading from Disk, the array is overwritten with only the entries from the file. The swarm clients are then lost from the slaves array, but still exist in the list of computers (Jenkins.computers). This leads to the inconsistent behaviour described above.
The fix simply remembers all existing swarm clients, and re-adds them to the slaves array after the reload finished.