We have enabled LDAP authentication for Jenkins, but do not have authorization enabled ('Anyone can do anything').
We do not pass a username or password for the swarm clients. This results in a 401 during the jnlp connection phase.
I've attached a very simple code change to hudson.plugins.swarm.Client.java that works in our environment. Under the condition that a username and password aren't available, do not add the '-credential' switch to the Main args.