By default, when the Kubernetes plugin checks for the number of agents in the namespace, it get the number of ALL pods in the namespace, not just the ones created by the plugin.
This is because when the current list of agents is being fetched, it filters all pods in the namespace by the pod labels applied to all agents created by the Jenkins cloud, which defaults to no labels. See: https://github.com/jenkinsci/kubernetes-plugin/blob/e026c003c90cec99328c1311c62c4b9f431e5f26/src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesCloud.java#L451
This has some critical consequences. In our case, we had 10 pods running in the namespace already, and a containerCap of 10. Because of this, it became impossible to create a new agent from the Kubernetes cloud integration.
I have a couple suggestions to fix this.
1) Set the default Kubernetes cloud label list to be non-empty. Perhaps default to a label like `kubernetes-cloud: <cloud-name>`.
2) Make the list of pod labels editable from the Jenkins management console. Right now, the only way to set this value (which is the needed workaround for the problem) is through a groovy script.