-
Bug
-
Resolution: Not A Defect
-
Critical
-
None
-
Jenkins v2.121.3
kubernetes-plugin v1.12.3
kubernetes 1.7.10
If the pod template is misconfigured, and the actual pod container exits with a non-0 code, then kubernetes tries to re-create it, in an endless loop. Mind you that there is no restart count on the pod, it's actually recreated, with a different random suffix.
Manually deleting the pod doesn't help, because it was never created successfully, so kubernetes re-creates it again.
In our case, the misconfiguration was a wrong JENKINS_URL.
Also, when the kubernetes-plugin creates a pod, there isn't any associated type in kubernetes itself (i.e. deployment, replicaset, job, etc...). So there wasn't anything to delete on a higher level.
Wouldn't a kubernetes "job" be suitable for this plugin ? (Instead of creating a standalone/orphan pod). This way if something goes wrong then the situation can be rectified by deleting the "job" in kubernetes.
I hope I haven't made any wrong assumptions here because I'm quite new to the kubernetes world
EDIT: Forgot to mention that this was happening both with Jenkins alive without any jobs running, AND with Jenkins itself shut down (the whole VM).