Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-69196

[Kubernetes plugin] Problem when create kubernetes agent

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Major Major
    • kubernetes-plugin
    • None
    • PROD

      Hello,

      We have configured a kubernetes cloud in our jenkins with our kubernetes aws server. And we have a sharedlib to create dynamic agents to build, deploy, quality audit, run integration tests and tnr tests, deploy on ec2, deploy on kubernetes, etc.

      This agent is formed with differents container according to the type of projects and the goal of the pipeline. But the agent contains always a jnlp container using the official image inbound-agent-4.7-1.

      The problem is that sometimes (not systematic) the agent fail to be created. In the namespace I have a pod created but not already running and in my jenkins pipeline I have this error:

      [Pipeline] Start of Pipeline[Pipeline] libraryResource[Pipeline] libraryResource[Pipeline] libraryResource[Pipeline] libraryResource[Pipeline] libraryResource[Pipeline] libraryResource[Pipeline] echo[WARNING] label option is deprecated. To use a static pod template, use the 'inheritFrom' option.[Pipeline] podTemplate[Pipeline] {[Pipeline] nodeCreated Pod: kubernetes-npr fr-jenkins-npr/phoxy-74-kd5b5-58gg1Still waiting to schedule task
      'phoxy-74-kd5b5-58gg1' is offlinefr-jenkins-npr/phoxy-74-kd5b5-58gg1 Container jnlp was terminated (Exit Code: 255, Reason: Error)

      • ansible – running
        ---{}Logs{}-----------
      • builder – running
        ---{}Logs{}-----------
      • docker-engine – running
        ---{}Logs{}-----------
        time="2022-08-01T12:12:30.344538274Z" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
        time="2022-08-01T12:12:30.344546495Z" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
        time="2022-08-01T12:12:30.344557644Z" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
        time="2022-08-01T12:12:30.344565085Z" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
        time="2022-08-01T12:12:30.344680457Z" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
        time="2022-08-01T12:12:30.344696034Z" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
        time="2022-08-01T12:12:30.344706214Z" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
        time="2022-08-01T12:12:30.344716883Z" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
        time="2022-08-01T12:12:30.344868007Z" level=info msg=serving... address=/var/run/docker/containerd/containerd-debug.sock
        time="2022-08-01T12:12:30.344905365Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
        time="2022-08-01T12:12:30.344932077Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
        time="2022-08-01T12:12:30.344944199Z" level=info msg="containerd successfully booted in 0.527532s"
        time="2022-08-01T12:12:30.349395796Z" level=info msg="Setting the storage driver from the $DOCKER_DRIVER environment variable (overlay2)"
        time="2022-08-01T12:12:30.349522634Z" level=info msg="parsed scheme: \"unix\"" module=grpc
        time="2022-08-01T12:12:30.349537625Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
        time="2022-08-01T12:12:30.349550401Z" level=info msg="ccResolverWrapper: sending update to cc: {[ {unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}

        ] <nil> <nil>}" module=grpc
        time="2022-08-01T12:12:30.349562681Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
        time="2022-08-01T12:12:30.350035560Z" level=info msg="parsed scheme: \"unix\"" module=grpc
        time="2022-08-01T12:12:30.350051957Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
        time="2022-08-01T12:12:30.350062553Z" level=info msg="ccResolverWrapper: sending update to cc:

        Unknown macro: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>}

        " module=grpc
        time="2022-08-01T12:12:30.350069464Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
        time="2022-08-01T12:12:36.234134503Z" level=warning msg="Your kernel does not support cgroup blkio weight"
        time="2022-08-01T12:12:36.234164728Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
        time="2022-08-01T12:12:36.234345297Z" level=info msg="Loading containers: start."
        time="2022-08-01T12:12:37.059043276Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
        time="2022-08-01T12:12:37.138903088Z" level=info msg="Loading containers: done."
        time="2022-08-01T12:12:38.111797300Z" level=info msg="Docker daemon" commit=87a90dc graphdriver(s)=overlay2 version=20.10.14
        time="2022-08-01T12:12:38.111893797Z" level=info msg="Daemon has completed initialization"
        time="2022-08-01T12:12:38.192371582Z" level=info msg="API listen on /var/run/docker.sock"
        time="2022-08-01T12:12:38.197138181Z" level=info msg="API listen on [::]:2375"

      • email-sender – running
        ---{}Logs{}-----------
      • image-tools – running
        ---{}Logs{}-----------
      • jnlp – terminated (255)
        ---{}Logs{}-----------
        INFO: Using Remoting version: 4.7
        Aug 01, 2022 12:11:58 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
        INFO: Using /home/jenkins/agent/remoting as a remoting work directory
        Aug 01, 2022 12:11:58 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
        INFO: Both error and output logs will be printed to /home/jenkins/agent/remoting
        Aug 01, 2022 12:11:58 PM hudson.remoting.jnlp.Main$CuiListener status
        INFO: Locating server among [http://lx2173.res.jcdecaux.org/]
        Aug 01, 2022 12:12:18 PM hudson.remoting.jnlp.Main$CuiListener error
        SEVERE: Failed to connect to http://lx2173.res.jcdecaux.org/tcpSlaveAgentListener/: lx2173.res.jcdecaux.org
        java.io.IOException: Failed to connect to http://lx2173.res.jcdecaux.org/tcpSlaveAgentListener/: lx2173.res.jcdecaux.org
        at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:214)
        at hudson.remoting.Engine.innerRun(Engine.java:689)
        at hudson.remoting.Engine.run(Engine.java:514)
        Caused by: java.net.UnknownHostException: lx2173.res.jcdecaux.org
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:607)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
        at sun.net.www.http.HttpClient.New(HttpClient.java:339)
        at sun.net.www.http.HttpClient.New(HttpClient.java:357)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
        at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990)
        at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:211)
        ... 2 more

      lx2173.res.jcdecaux.org is the host of my jenkins server. This server is installed in ec2 vm and has also other physical ec2 slaves.
      The dns alias exist and is used in my the reverse proxy configuration (httpd) in face of my jenkins process.
      Joined my kubernetes cloud configuration.

      The version of jenkins is 2.332.3

      The version of kubernetes plugin is 3670.v6ca_059233222
      Can you help us to solve this problem please?

            Unassigned Unassigned
            ramim Rami Ammar
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: