Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57143

JNLP4 error: "Connection closed before acknowledgement sent"

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Blocker
    • Resolution: Cannot Reproduce
    • Component/s: github-oauth-plugin
    • Labels:
      None
    • Environment:
    • Similar Issues:

      Description

      Hello!

      We run JNLP-powered agents on 4 Mac machines, after Jenkins update to 2.164.2 (from 2.150.3), those machines are not able to connect using JNLP-4 (see exception below), although JNLP-3 connection works.

      Agent is run as:
      /usr/bin/java -Djava.awt.headless=true -jar /Applications/Jenkins/agent.jar -jnlpUrl https://<redacted-master-address>/computer/<redacted-agent>/slave-agent.jnlp -secret <redacted-secret> -workDir /Users/jenkins/jenkins_root

      Exception on agent:
      INFO: Protocol JNLP4-connect encountered an unexpected exception
      java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
      at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
      at hudson.remoting.Engine.innerRun(Engine.java:614)
      at hudson.remoting.Engine.run(Engine.java:474)
      Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
      at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.onRecvClosed(AckFilterLayer.java:280)
      at org.jenkinsci.remoting.protocol.FilterLayer.abort(FilterLayer.java:164)
      at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.access$000(AckFilterLayer.java:43)
      at org.jenkinsci.remoting.protocol.impl.AckFilterLayer$1.run(AckFilterLayer.java:176)
      at org.jenkinsci.remoting.protocol.IOHub$DelayedRunnable.run(IOHub.java:964)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
      at java.lang.Thread.run(Thread.java:748

      On jenkins master:

      Apr 22, 2019 3:42:42 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
      WARNING: Connection #1512 failed
      java.io.EOFException
      at java.io.DataInputStream.readFully(DataInputStream.java:197)
      at java.io.DataInputStream.readFully(DataInputStream.java:169)
      at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:244)

      Apr 22, 2019 3:42:42 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
      INFO: Accepted JNLP4-connect connection #1,513 from /<redacted-agent-ip>:58595
      Apr 22, 2019 3:42:54 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
      WARNING: Connection #1514 failed
      java.io.EOFException
      at java.io.DataInputStream.readFully(DataInputStream.java:197)
      at java.io.DataInputStream.readFully(DataInputStream.java:169)
      at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:244)

      Things that I checked:

      • agent.jar is up-to-date and matches the one on master
      • JNLP port on master is open and available from the agent

        Attachments

          Activity

          Hide
          nneul Nathan Neulinger added a comment -

          Yep, almost exact same failure, with addition of it indicating that all protocols were rejected. Re-enabling JNLP3 fixed it as a test. It was just dumb luck viewing an strace of the jenkins master that I saw the hangs/failures of reverse dns lookups on the master.

           

           

          Show
          nneul Nathan Neulinger added a comment - Yep, almost exact same failure, with addition of it indicating that all protocols were rejected. Re-enabling JNLP3 fixed it as a test. It was just dumb luck viewing an strace of the jenkins master that I saw the hangs/failures of reverse dns lookups on the master.    
          Hide
          jthompson Jeff Thompson added a comment -

          Any ideas what we could do to better handle this situation? Or provide better diagnostics?

          Show
          jthompson Jeff Thompson added a comment - Any ideas what we could do to better handle this situation? Or provide better diagnostics?
          Hide
          nneul Nathan Neulinger added a comment -

          There was near zero useful reporting server side - if the JNLP server side handler has "remote client has to resolve in DNS" at the very least reporting that explicitly in the failure logs. I can understand it not being able to necessarily make it back to the client side but at the least the server side should report it usefully in logs. (Just guessing there, but I can certainly see that possibility from JNLP4 protocol limitations.) 

           

          Would be great if server side could report it in the agent status on the node page, but at the very least getting it in the logs. 

          Show
          nneul Nathan Neulinger added a comment - There was near zero useful reporting server side - if the JNLP server side handler has "remote client has to resolve in DNS" at the very least reporting that explicitly in the failure logs. I can understand it not being able to necessarily make it back to the client side but at the least the server side should report it usefully in logs. (Just guessing there, but I can certainly see that possibility from JNLP4 protocol limitations.)    Would be great if server side could report it in the agent status on the node page, but at the very least getting it in the logs. 
          Hide
          nneul Nathan Neulinger added a comment -

          I'd also ask question of why it has to reverse resolve - i.e. why is that requirement in there in the first place. 

          Show
          nneul Nathan Neulinger added a comment - I'd also ask question of why it has to reverse resolve - i.e. why is that requirement in there in the first place. 
          Hide
          jthompson Jeff Thompson added a comment -

          It would be nice to get some time to see about getting some of that in. Better diagnosability would be a nice addition.

          Yeah, I wonder why it needs to reverse resolve. Would take some time to figure out what and why it's doing there.

          Thanks!

          Show
          jthompson Jeff Thompson added a comment - It would be nice to get some time to see about getting some of that in. Better diagnosability would be a nice addition. Yeah, I wonder why it needs to reverse resolve. Would take some time to figure out what and why it's doing there. Thanks!

            People

            • Assignee:
              sag47 Sam Gleske
              Reporter:
              conf Alexey Shein
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: