Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61006

AWS EC2 plugin failing to connect to Windows Slave

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: ec2-plugin
    • Labels:
    • Environment:
      Jenkins - 2.222.1
      ec2-plugin - 1.49.1
    • Similar Issues:

      Description

      Jenkins EC2 Plugin fails to launch Windows Slave after upgrading the plugin version from 1.46.1 to 1.49.1. While the Linux agents come up perfectly, the Windows agents fail to come up and fail with the below error. 

      I believe this issue is due to the update of the smbj version as mentioned in the release notes. In order to get my agents working, I had to revert to the previous version of the plugin (1.46.1).

       

      Please investigate and advise on the fix.

      WARNING h.plugins.ec2.win.WinConnection#ping: Failed to verify connectivity to Windows slave java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.readFully(DirectTcpPacketReader.java:70) at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.readTcpHeader(DirectTcpPacketReader.java:59) at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.doRead(DirectTcpPacketReader.java:48) Caused: com.hierynomus.protocol.transport.TransportException at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.doRead(DirectTcpPacketReader.java:53) at com.hierynomus.smbj.transport.PacketReader.readPacket(PacketReader.java:70) at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:48) Caused: com.hierynomus.smbj.common.SMBRuntimeException at com.hierynomus.smbj.common.SMBRuntimeException$1.wrap(SMBRuntimeException.java:27) at com.hierynomus.smbj.common.SMBRuntimeException$1.wrap(SMBRuntimeException.java:21) at com.hierynomus.protocol.commons.concurrent.Promise.deliverError(Promise.java:95) at com.hierynomus.smbj.connection.OutstandingRequests.handleError(OutstandingRequests.java:88) at com.hierynomus.smbj.connection.Connection.handleError(Connection.java:465) at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:54) Caused: java.util.concurrent.ExecutionException at com.hierynomus.protocol.commons.concurrent.PromiseBackedFuture.get(PromiseBackedFuture.java:60) at com.hierynomus.protocol.commons.concurrent.CancellableFuture.get(CancellableFuture.java:84) at com.hierynomus.protocol.commons.concurrent.Futures.get(Futures.java:38) Caused: com.hierynomus.protocol.transport.TransportException at com.hierynomus.protocol.transport.TransportException$1.wrap(TransportException.java:29) at com.hierynomus.protocol.transport.TransportException$1.wrap(TransportException.java:23) at com.hierynomus.protocol.commons.concurrent.Futures.get(Futures.java:43) at com.hierynomus.smbj.connection.Connection.sendAndReceive(Connection.java:305) at com.hierynomus.smbj.connection.Connection.initiateSessionSetup(Connection.java:244) at com.hierynomus.smbj.connection.Connection.authenticate(Connection.java:181) Caused: com.hierynomus.smbj.common.SMBRuntimeException at com.hierynomus.smbj.connection.Connection.authenticate(Connection.java:215) at hudson.plugins.ec2.win.WinConnection.ping(WinConnection.java:112) at hudson.plugins.ec2.win.EC2WindowsLauncher.connectToWinRM(EC2WindowsLauncher.java:181) at hudson.plugins.ec2.win.EC2WindowsLauncher.launchScript(EC2WindowsLauncher.java:39) at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:48) at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:291) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
      
      
      WARNING h.plugins.ec2.win.WinConnection#ping: Failed to verify connectivity to Windows slave java.net.SocketException: Broken pipe (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) at java.net.SocketOutputStream.write(SocketOutputStream.java:155) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at com.hierynomus.smbj.transport.tcp.direct.DirectTcpTransport.write(DirectTcpTransport.java:75) Caused: com.hierynomus.protocol.transport.TransportException at com.hierynomus.smbj.transport.tcp.direct.DirectTcpTransport.write(DirectTcpTransport.java:78) at com.hierynomus.smbj.connection.Connection.send(Connection.java:297) at com.hierynomus.smbj.connection.Connection.sendAndReceive(Connection.java:305) at com.hierynomus.smbj.connection.Connection.initiateSessionSetup(Connection.java:244) at com.hierynomus.smbj.connection.Connection.authenticate(Connection.java:181) Caused: com.hierynomus.smbj.common.SMBRuntimeException at com.hierynomus.smbj.connection.Connection.authenticate(Connection.java:215) at hudson.plugins.ec2.win.WinConnection.ping(WinConnection.java:112) at hudson.plugins.ec2.win.EC2WindowsLauncher.connectToWinRM(EC2WindowsLauncher.java:169) at hudson.plugins.ec2.win.EC2WindowsLauncher.launchScript(EC2WindowsLauncher.java:39) at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:48) at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:291) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      

        Attachments

        1. Ec2-linux-error.JPG
          Ec2-linux-error.JPG
          109 kB
        2. Ec2-win-1.46.1.jpg
          Ec2-win-1.46.1.jpg
          176 kB
        3. Ec2-win-error.jpg
          Ec2-win-error.jpg
          59 kB
        4. image-2020-07-24-13-33-11-072.png
          image-2020-07-24-13-33-11-072.png
          3 kB
        5. image-2020-07-24-13-35-10-259.png
          image-2020-07-24-13-35-10-259.png
          4 kB
        6. Jenkins-manage-error.JPG
          Jenkins-manage-error.JPG
          139 kB

          Activity

          Hide
          mramonleon Ramon Leon added a comment -

          Let me ask you some questions to try to reduce the factors implied:

          • Do you allow self-signed certificates for this windows instance?
          • You're not using https, right?

          Try to allow self-signed certificates to see if it helps. The latest release tries to avoid them for security reasons. But let's start from the beginning and iterate to solve your issue.

          Configuring a Windows instance to be totally secure is very cumbersome. Please check the documentation on the plugin GitHub repo: https://github.com/jenkinsci/ec2-plugin/#securing-the-connection-to-windows-amis and in the AMI Type field help, under the template configuration on your Jenkins instance.

          Let's first try without security in place to figure out what's going on

          Show
          mramonleon Ramon Leon added a comment - Let me ask you some questions to try to reduce the factors implied: Do you allow self-signed certificates for this windows instance? You're not using https, right? Try to allow self-signed certificates to see if it helps. The latest release tries to avoid them for security reasons. But let's start from the beginning and iterate to solve your issue. Configuring a Windows instance to be totally secure is very cumbersome. Please check the documentation on the plugin GitHub repo: https://github.com/jenkinsci/ec2-plugin/#securing-the-connection-to-windows-amis and in the AMI Type field help, under the template configuration on your Jenkins instance. Let's first try without security in place to figure out what's going on
          Hide
          sastac Steven Astac added a comment -

          I had the same issue, Windows 10 AMI, telnet 445 / 5985 / 5986 all succeeded, using Remote Desktop app to login using Administrator +password works.

          Direct HTTP, non-verify cert is ticked. Firewall is not an issue.

          I just see
          Connecting to (<ip>) with WinRM as Administrator, and hanging. Eventually it got terminated due to timeout.

          The plugin is what's on the master branch (1.51) right now.

          Show
          sastac Steven Astac added a comment - I had the same issue, Windows 10 AMI, telnet 445 / 5985 / 5986 all succeeded, using Remote Desktop app to login using Administrator +password works. Direct HTTP, non-verify cert is ticked. Firewall is not an issue. I just see Connecting to (<ip>) with WinRM as Administrator, and hanging. Eventually it got terminated due to timeout. The plugin is what's on the master branch (1.51) right now.
          Hide
          ajuc Andrei Juc added a comment - - edited

          I too am experiencing the same issue as Steven Astac.

          An AMI that used to previously connect without issue is now failing to connect.

          Jenkins immediately displays error "This agent is offline because Jenkins failed to launch the agent process on it"

          Checking the log does not provide any insight into the cause of the failure. Jenkins says the AMI booted and that WinRM is connecting, however that just hangs indefinitely:

          The AMI is configured to connect via HTTP and accept self-signed certificates.
          EC2 Plugin is latest available version

          Show
          ajuc Andrei Juc added a comment - - edited I too am experiencing the same issue as Steven Astac . An AMI that used to previously connect without issue is now failing to connect. Jenkins immediately displays error "This agent is offline because Jenkins failed to launch the agent process on it" Checking the log does not provide any insight into the cause of the failure. Jenkins says the AMI booted and that WinRM is connecting, however that just hangs indefinitely: The AMI is configured to connect via HTTP and accept self-signed certificates. EC2 Plugin is latest available version
          Hide
          rroques Remi Roques added a comment -

          I am seeing a similar behaviour. Nothing in the logs but the master hangs connecting. It used to work on 1.46.1.

          Show
          rroques Remi Roques added a comment - I am seeing a similar behaviour. Nothing in the logs but the master hangs connecting. It used to work on 1.46.1.
          Hide
          hungvotrung Hung Vo added a comment -

          Same issue from me. Downgrade to 1.46.1 work 

          Show
          hungvotrung Hung Vo added a comment - Same issue from me. Downgrade to 1.46.1 work 

            People

            • Assignee:
              thoulen FABRIZIO MANFREDI
              Reporter:
              jpara29 PJ
            • Votes:
              2 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated: