Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42959

Failed known_hosts verification for SSH agent

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • ssh-slaves-plugin
    • Versions:

      Jenkins: 2.51
      SSH Agent Plugin: 1.14
      SSH Credentials plugin: 1.13
      SSH Slaves Plugin : 1.15

      Ubuntu 14.04, 16.04

      SSH agent isn't launched after the latest update, complaining about missing records in the known hosts file.

      Nevertheless, the records do exist. I've tried to connect manually with ssh, everything was fine, ssh reports that it has found the host in known_hosts.

      Probable reason is the hashed host name in known_hosts.

      Here is the log from the Jenkins slave launch page:

      [03/21/17 11:40:34] [SSH] Opening SSH connection to xxx.xxx.xxx.xxx:22
      [03/21/17 11:40:34] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host.      Connections will be denied until this new host and its associated key is added to the Known Hosts file.
           Key exchange was not finished, connection is closed.
           java.io.IOException: There was a problem while connecting to xxx.xxx.xxx.xxx:22
      	at com.trilead.ssh2.Connection.connect(Connection.java:818)
      	at com.trilead.ssh2.Connection.connect(Connection.java:687)
      	at com.trilead.ssh2.Connection.connect(Connection.java:601)
      	at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265)
      	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790)
      	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
           Caused by: java.io.IOException: Key exchange was not finished, connection is closed.
      	at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:93)
      	at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:230)
      	at com.trilead.ssh2.Connection.connect(Connection.java:770)
      	... 9 more
           Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback
      	at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591)
      	at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777)
      	at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489)
      	... 1 more
           [03/21/17 11:40:34] Launch failed - cleaning up connection
           [03/21/17 11:40:34] [SSH] Connection closed.
      
      

      And this is the debug output from ssh, showing that it has found a record in the second line of known_hosts.

        $ sudo -u jenkins -g jenkins ssh -v jenkins@xxx.xxx.xxx.xxx
          OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
          debug1: Reading configuration data /etc/ssh/ssh_config
          debug1: /etc/ssh/ssh_config line 19: Applying options for *
          debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
          debug1: Connection established.
          debug1: identity file /var/lib/jenkins/.ssh/id_rsa type 1
      ...
          debug1: sending SSH2_MSG_KEX_ECDH_INIT
          debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
          debug1: Server host key: ECDSA xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
          debug1: Host 'xxx.xxx.xxx.xxx.xxx' is known and matches the ECDSA host key.
          debug1: Found key in /var/lib/jenkins/.ssh/known_hosts:2
          debug1: ssh_ecdsa_verify: signature correct
      ...
          Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-67-generic x86_64)
      

      Additional logs from Jenkins system log:

      Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
          kex_algo=diffie-hellman-group14-sha1
      Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
          server_host_key_algo=ssh-rsa
          Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
          enc_algo_client_to_server=aes256-ctr
          Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
          enc_algo_server_to_client=aes256-ctr
          Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
          mac_algo_client_to_server=hmac-sha1
          Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
          mac_algo_server_to_client=hmac-sha1
          Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
          comp_algo_client_to_server=none
          Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
          comp_algo_server_to_client=none
          Mar 21, 2017 12:06:08 PM FINE com.trilead.ssh2.transport.TransportManager
          Receive thread: error in receiveLoop
          java.io.IOException: The server hostkey was not accepted by the verifier callback
                  at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591)
                  at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777)
                  at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489)
                  at java.lang.Thread.run(Thread.java:745)
      
          Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.TransportManager
          Receive thread: back from receiveLoop
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
          kex_algo=diffie-hellman-group14-sha1
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
          server_host_key_algo=ssh-rsa
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
          enc_algo_client_to_server=aes256-ctr
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
          enc_algo_server_to_client=aes256-ctr
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
          mac_algo_client_to_server=hmac-sha1
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
          mac_algo_server_to_client=hmac-sha1
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
          comp_algo_client_to_server=none
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
          comp_algo_server_to_client=none
          Mar 21, 2017 12:06:10 PM FINE com.trilead.ssh2.transport.TransportManager
          Receive thread: error in receiveLoop
          java.io.IOException: The server hostkey was not accepted by the verifier callback
                  at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591)
                  at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777)
                  at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489)
                  at java.lang.Thread.run(Thread.java:745)
      
          Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.TransportManager
          Receive thread: back from receiveLoop
      

      File /var/lib/jenkins/.ssh/known_hosts contains strings, looking like being base64-encoded, delimited by '|'. Here is the sample.

      |1|DAg  ...   o... 1ll9wI=| ...  ....  tIrM= ecdsa-sha2-nistp256 xxxxxx..... bmlzdHAyNTYAAAAIbm................. .................xxxxxxxxxxxoKEHF3Vr0q685jI2+6vWjvAAG4lz5Ckujy9k=
      

      Github issue

            mc1arke Michael Clarke
            wl2776 Vladimir Eremeev
            Votes:
            55 Vote for this issue
            Watchers:
            75 Start watching this issue

              Created:
              Updated:
              Resolved: