Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49312

AWS EC2 Plugin is not able to launch instances automatically on 1.37, 1.38

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We currently use 1.36 version of aws ec2 plugin for launching on demand slaves.
      Jenkins version LTS 2.89.2
      When we updated to 1.37 or 1.38 slaves never launch automatically. If you manually go and launch it launches.

      Logs below:

      Jan 02, 2018 5:12:57 PM hudson.plugins.ec2.EC2Cloud provision
      INFO: Attempting to provision slave from template hudson.plugins.ec2.SlaveTemplate@2dc32469 needed by excess workload of 1 units of label 'ba'
      Considering launching ami-***** for template ba-fedora26-slave
      Jan 02, 2018 5:12:59 PM hudson.plugins.ec2.SlaveTemplate logProvisionInfo
      INFO: Considering launching ami-**** for template ba-fedora26-slave
      Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate
      Jan 02, 2018 5:12:59 PM hudson.plugins.ec2.SlaveTemplate logProvisionInfo
      INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate
      Looking for existing instances with describe-instance: {Filters: [{Name: image-id,Values: [ami-****]}, {Name: availability-zone,Values: [us-east-1d]}, {Name: subnet-id,Values: [subnet-****]}, {Name: instance.group-id,Values: [sg-****, sg-**, **, sg-***]}, {Name: key-name,Values: [build_**_access]}, {Name: instance-type,Values: [t2.small]}, {Name: tag:app_group,Values: [***]}, {Name: tag:app_name,Values: [**]}, {Name: tag:Name,Values: [**]}],InstanceIds: [],}
      checkInstance: {AmiLaunchIndex: 0,ImageId: ami-***,InstanceId: i-0f165a71ba99e7540,InstanceType: t2.small,KeyName: build_****_access,LaunchTime: Tue Jan 02 16:41:27 MST 2018,Monitoring: {State: disabled},Placement: {AvailabilityZone: us-east-1d,GroupName: ,Tenancy: default,},PrivateDnsName: ip-172-20-78-118.ec2.internal,PrivateIpAddress: 172.20***,ProductCodes: [],PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIpAddress: 34.226.203.214,State: {Code: 16,Name: running},StateTransitionReason: ,SubnetId: subnet-**,VpcId: vpc-**,Architecture: x86_64,BlockDeviceMappings: [{DeviceName: /dev/sda1,Ebs: {AttachTime: Tue Jan 02 16:41:28 MST 2018,DeleteOnTermination: true,Status: attached,VolumeId: vol-0d94c185bd7d5754d}}],ClientToken: *****,EbsOptimized: false,Hypervisor: xen,IamInstanceProfile: {Arn: **,Id: **},ElasticGpuAssociations: [],NetworkInterfaces: [{Association: {IpOwnerId: amazon,PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIp: 34.226.203.214},Attachment: {AttachTime: Tue Jan 02 16:41:27 MST 2018,AttachmentId: eni-attach-dae44906,DeleteOnTermination: true,DeviceIndex: 0,Status: attached},Description: ,Groups: [{GroupName: jenkinsba_slave,GroupId: sg-***}, {GroupName: sonarqube-db,GroupId: sg-***}, {GroupName: sonarqube,GroupId: sg-**}, {GroupName: default,GroupId: sg-**}],Ipv6Addresses: [],MacAddress: 0e:f5:dd:94:eb:ce,NetworkInterfaceId: eni-**,OwnerId: **,PrivateDnsName: ip-172-20-78-118.ec2.internal,PrivateIpAddress: 172.20.78.118,PrivateIpAddresses: [{Association: {IpOwnerId: amazon,PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIp: 34.226.203.214},Primary: true,PrivateDnsName: ip-172-20-7***.ec2.internal,PrivateIpAddress: 172.**.**.**}],SourceDestCheck: true,Status: in-use,SubnetId: subnet-9***,VpcId: vpc-***}],RootDeviceName: /dev/sda1,RootDeviceType: ebs,SecurityGroups: [{GroupName: jenkinsba_slave,GroupId: sg-**}, {GroupName: sonarqube-db,GroupId: sg-**}, {GroupName: sonarqube,GroupId: sg-**}, {GroupName: default,GroupId: sg-**8}],SourceDestCheck: true,Tags: [{Key: app_group,Value: **}, {Key: app_name,Value: j**}, {Key: Name,Value: **}, {Key: jenkins_slave_type,Value: demand_ba-fedora26-slave}],VirtualizationType: hvm}
      true - Instance has no node, but can be used
      Found existing instance: {AmiLaunchIndex: 0,ImageId: ami-**,InstanceId: i-0f165a71ba99e7540,InstanceType: t2.small,KeyName: build_jenkinsba_access,LaunchTime: Tue Jan 02 16:41:27 MST 2018,Monitoring: {State: disabled},Placement: {AvailabilityZone: us-east-1d,GroupName: ,Tenancy: default,},PrivateDnsName: ip-172-20-78-118.ec2.internal,PrivateIpAddress: 172.20.78.118,ProductCodes: [],PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIpAddress: 34.226.203.214,State: {Code: 16,Name: running},StateTransitionReason: ,SubnetId: subnet-**,VpcId: vpc-**,Architecture: x86_64,BlockDeviceMappings: [{DeviceName: /dev/sda1,Ebs: {AttachTime: Tue Jan 02 16:41:28 MST 2018,DeleteOnTermination: true,Status: attached,VolumeId: vol-0d94c185bd7d5754d}}],ClientToken: ***,EbsOptimized: false,Hypervisor: xen,IamInstanceProfile: {Arn: *******,Id: **},ElasticGpuAssociations: [],NetworkInterfaces: [{Association: {IpOwnerId: amazon,PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIp: 34.226.203.214},Attachment: {AttachTime: Tue Jan 02 16:41:27 MST 2018,AttachmentId: eni-attach-dae44906,DeleteOnTermination: true,DeviceIndex: 0,Status: attached},Description: ,Groups: [{GroupName: jenkinsba_slave,GroupId: sg-****}, {GroupName: sonarqube-db,GroupId: sg-**}, {GroupName: sonarqube,GroupId: sg-**}, {GroupName: default,GroupId: sg-**}],Ipv6Addresses: [],MacAddress: 0e:f5:dd:94:eb:ce,NetworkInterfaceId: eni-437193c8,OwnerId: ****,PrivateDnsName: ip-172-*****.ec2.internal,PrivateIpAddress: 172.*****,PrivateIpAddresses: [{Association: {IpOwnerId: amazon,PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIp: 34.226.203.214},Primary: true,PrivateDnsName: ip-172-****.ec2.internal,PrivateIpAddress: 172.******}],SourceDestCheck: true,Status: in-use,SubnetId: subnet-**,VpcId: vpc-**}],RootDeviceName: /dev/sda1,RootDeviceType: ebs,SecurityGroups: [{GroupName: jenkinsba_slave,GroupId: sg-e665ee95}, {GroupName: sonarqube-db,GroupId: sg-**}, {GroupName: sonarqube,GroupId: sg-**}, {GroupName: default,GroupId: sg-**}],SourceDestCheck: true,Tags: [{Key: app_group,Value: **}, {Key: app_name,Value: **}, {Key: Name,Value: jenkins-ba-slave}, {Key: jenkins_slave_type,Value: demand_ba-fedora26-slave}],VirtualizationType: hvm}
      Found existing pending or running: running instance: {AmiLaunchIndex: 0,ImageId: ami-**,InstanceId: i-0f165a71ba99e7540,InstanceType: t2.small,KeyName: build_jenkinsba_access,LaunchTime: Tue Jan 02 16:41:27 MST 2018,Monitoring: {State: disabled},Placement: {AvailabilityZone: us-east-1d,GroupName: ,Tenancy: default,},PrivateDnsName: ip-172-20-78-118.ec2.internal,PrivateIpAddress: 172.20.78.118,ProductCodes: [],PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIpAddress: 34.226.203.214,State: {Code: 16,Name: running},StateTransitionReason: ,SubnetId: subnet-**,VpcId: vpc-**,Architecture: x86_64,BlockDeviceMappings: [{DeviceName: /dev/sda1,Ebs: {AttachTime: Tue Jan 02 16:41:28 MST 2018,DeleteOnTermination: true,Status: attached,VolumeId: vol-0d94c185bd7d5754d}}],ClientToken: **,EbsOptimized: false,Hypervisor: xen,IamInstanceProfile: {Arn: ****,Id: ***},ElasticGpuAssociations: [],NetworkInterfaces: [{Association: {IpOwnerId: amazon,PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIp: 34.226.203.214},Attachment: {AttachTime: Tue Jan 02 16:41:27 MST 2018,AttachmentId: eni-attach-dae44906,DeleteOnTermination: true,DeviceIndex: 0,Status: attached},Description: ,Groups: [{GroupName: jenkinsba_slave,GroupId: sg-**}, {GroupName: sonarqube-db,GroupId: sg-**}, {GroupName: sonarqube,GroupId: sg-ba8701c1}, {GroupName: default,GroupId: sg-fed32398}],Ipv6Addresses: [],MacAddress: 0e:f5:dd:94:eb:ce,NetworkInterfaceId: eni-437193c8,OwnerId: *******,PrivateDnsName: ip-172-20-78-118.ec2.internal,PrivateIpAddress: 172.20.78.118,PrivateIpAddresses: [{Association: {IpOwnerId: amazon,PublicDnsName: ec2-34-226-203-214.compute-1.amazonaws.com,PublicIp: 34.226.203.214},Primary: true,PrivateDnsName: ip-172-**.ec2.internal,PrivateIpAddress: 172.20**}],SourceDestCheck: true,Status: in-use,SubnetId: subnet-**,VpcId: vpc-**}],RootDeviceName: /dev/sda1,RootDeviceType: ebs,SecurityGroups: [{GroupName: jenkinsba_slave,GroupId: sg-**}, {GroupName: sonarqube-db,GroupId: sg-**}, {GroupName: sonarqube,GroupId: sg-**}, {GroupName: default,GroupId: sg-**}],SourceDestCheck: true,Tags: [{Key: app_group,Value: **}, {Key: app_name,Value: **}, {Key: Name,Value: jenkins-ba-slave}, {Key: jenkins_slave_type,Value: demand_ba-fedora26-slave}],VirtualizationType: hvm}

        Attachments

          Activity

          Hide
          bekzot89 Bek Azimov added a comment -

          I can see it is working, just much slower than before. Before it was detecting the need for slave within minute or two then launch new one. But now it is waiting for about 5-6 minutes.

          Show
          bekzot89 Bek Azimov added a comment - I can see it is working, just much slower than before. Before it was detecting the need for slave within minute or two then launch new one. But now it is waiting for about 5-6 minutes.
          Hide
          puneeth_n Puneeth Nanjundaswamy added a comment - - edited

          are facing the same issue too. We are now at Jenkins 2.89.4 and EC2 plugin 1.38. Bek Azimov Why is it not a defect?

          Show
          puneeth_n Puneeth Nanjundaswamy added a comment - - edited are facing the same issue too. We are now at Jenkins 2.89.4 and EC2 plugin 1.38. Bek Azimov Why is it not a defect?
          Hide
          bekzot89 Bek Azimov added a comment -

          It is a defect still. It worked on our 2 different Jenkins masters. And other 2 different Jenkins masters plugin is not working. All same core versions. I closed this when it worked on ofher but we are seeing the issue still.

          Show
          bekzot89 Bek Azimov added a comment - It is a defect still. It worked on our 2 different Jenkins masters. And other 2 different Jenkins masters plugin is not working. All same core versions. I closed this when it worked on ofher but we are seeing the issue still.
          Hide
          puneeth_n Puneeth Nanjundaswamy added a comment - - edited

          This is till an issue. so reopening it

          Show
          puneeth_n Puneeth Nanjundaswamy added a comment - - edited This is till an issue. so reopening it
          Hide
          francisu Francis Upton added a comment -

          Please reopen if you can reproduce this problem in 1.39 of the plugin.

          Show
          francisu Francis Upton added a comment - Please reopen if you can reproduce this problem in 1.39 of the plugin.

            People

            • Assignee:
              francisu Francis Upton
              Reporter:
              bekzot89 Bek Azimov
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: