Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46601

error in log files when Unix Jenkins slaves are shutdown

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: ec2-plugin
    • Labels:
    • Environment:
      Jenkins Version: 2.60.3
      Amazon EC2 plugins: 1.36
      Idle termination time: -5
    • Similar Issues:

      Description

      We are seeing sporadic stack traces in the Jenkins master log file and Jenkins slave log file when Unix Jenkins slave are successfully shutdown at the end of the billing period. Could this be due to the AWS instance being shutdown before it was disconnected from Jenkins master?  In a local branch we changed the logic to disconnect the agent before stopping the instance.

      From Jenkins slave log file:

      Sep 01, 2017 12:18:35 PM null
      INFO: Launching slave agent (via Trilead SSH2 Connection): java -Xms128m -Xmx1g -jar /tmp/slave.jar
      <===[JENKINS REMOTING CAPACITY]===>Slave.jar version: 3.7
      This is a Unix agent
      Evacuated stdout
      Agent successfully connected and online
      ERROR: Connection terminated
      ha:////xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==java.io.EOFException
              at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638)
              at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113)
              at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
              at java.io.ObjectInputStream.<init>(ObjectInputStream.java:349)
              at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
              at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
              at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
      Caused: java.io.IOException: Unexpected termination of the channel
              at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
      

      From Jenkins master log file:

      Sep 01, 2017 1:13:34 PM hudson.plugins.ec2.EC2RetentionStrategy internalCheck
      INFO: Idle timeout of jenkins-slave-name (i-xxxxxxxxxxxxxxxx) after 23 idle minutes, with 4 minutes remaining in billing period
      Sep 01, 2017 1:13:34 PM hudson.plugins.ec2.EC2AbstractSlave idleTimeout
      INFO: EC2 instance idle time expired: i-xxxxxxxxxxxxxxxx
      Sep 01, 2017 1:13:34 PM hudson.remoting.SynchronousCommandTransport$ReaderThread run
      SEVERE: I/O error in channel jenkins-slave-name (i-xxxxxxxxxxxxxxxx)
      java.io.IOException: Unexpected termination of the channel
              at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
      Caused by: java.io.EOFException
              at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638)
              at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113)
              at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
              at java.io.ObjectInputStream.<init>(ObjectInputStream.java:349)
              at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
              at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
              at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
      Sep 01, 2017 1:13:35 PM hudson.plugins.ec2.EC2AbstractSlave stop
      INFO: EC2 instance stop request sent for i-xxxxxxxxxxxxxxxx
      

      From SSHD log file on Jenkins slave:

      Sep  1 13:13:34 ip-xx-xxx-xxx-xxx sshd[2220]: pam_unix(sshd:session): session closed for user centos
      Sep  1 13:13:34 ip-xx-xxx-xxx-xxx sshd[798]: Received signal 15; terminating.
      

      From messages log file on Jenkins slave:

      Sep  1 13:13:34 ip-xx-xxx-xxx-xxx systemd: Stopping OpenSSH server daemon...
      

        Attachments

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              francisu Francis Upton
              Reporter:
              mkozell Mike Kozell
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: