-
Bug
-
Resolution: Unresolved
-
Minor
-
None
When launching instances, the EC2 plugin sometimes fails to launch the instance, though it doesn't throw any error.
The following was in the Jenkins logs. The final line seems to be printing out all reservations in the entire AWS account, not just those controlled by Jenkins. In our case, this is hundreds of pages in a single line, which I've trimmed.
INFO: Launching ami-a78ec5b0 for template jenkins-builder-12.04
Spot instance id in provision: sir-5cvg5szg
Oct 20, 2016 6:48:42 PM hudson.plugins.ec2.SlaveTemplate provisionSpot
INFO: Spot instance id in provision: sir-5cvg5szg
Oct 20, 2016 6:48:43 PM hudson.plugins.ec2.EC2Cloud provision
INFO: We have now 14 computers
Oct 20, 2016 6:48:43 PM hudson.plugins.ec2.EC2RetentionStrategy start
INFO: Start requested for jenkins-builder-12.04 (sir-5cvg5szg)
Oct 20, 2016 6:48:44 PM hudson.plugins.ec2.EC2Computer _describeInstanceOnce
INFO: Unexpected number of reservations reported by EC2 for instance id 'null', expected 1 result, found [{ReservationId: r-2e8ac590,OwnerId: 074332678689,Groups: [],Instances: [{InstanceId: i-28bbf919,ImageId: ami-2060e937,State: {Code: 16,Name: running},PrivateDnsName
: ip-10-60-5-254.ec2.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: vagrant-insecure,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m4.4xlarge,LaunchTime: Wed Oct 12 20:18:55 UTC 2016,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default,},M
onitoring: {State: disabled},SubnetId: subnet-2b874e01,VpcId: vpc-fc762198,PrivateIpAddress: 10.60.5.254,Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [{DeviceName: /dev/sda1,Ebs: {VolumeId: vol-5dce4288,Status: attached,AttachT
ime: Mon Sep 19 17:11:00 UTC 2016,DeleteOnTermination: true}}, {DeviceName: /dev/sdb,Ebs: {VolumeId: vol-9fce424a,Status: attached,AttachTime: Mon Sep 19 17:11:00 UTC 2016,DeleteOnTermination: true}}, {DeviceName: /dev/sdf,Ebs: {VolumeId: vol-d4ce4201,Status: attached,A
ttachTime: Mon Sep 19 17:11:00 UTC 2016,DeleteOnTermination: true}}, {DeviceName: /dev/xvdg,Ebs: {VolumeId: vol-90c1ce45,Status: attached,AttachTime: Wed Oct 12 20:10:15 UTC 2016,DeleteOnTermination: false}}],VirtualizationType: hvm,ClientToken: ,Tags: [{Key: Hostname,V
alue: app01}, {Key: Roles,Value: application}, {Key: Name,Value: vagrant-fds-release}, {Key: Owner,Value: jhuang}, {Key: Launched,Value: 2016-09-19T17:10:55Z}, {Key: Jira,Value: }],SecurityGroups: [{GroupName: fdsautomation_sg,GroupId: sg-62dc5619}, {GroupName: basic_ac
cess_sg,GroupId: sg-d39070ab}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-e99b7bfb,SubnetId: subnet-2b874e01,VpcId: vpc-fc762198,Description: ,OwnerId: 074332678689,Status: in-use,MacAddress: 12:a2:36:c3:64:95,PrivateIpAddress: 10
.60.5.254,PrivateDnsName: ip-10-60-5-254.ec2.internal,SourceDestCheck: true,Groups: [{GroupName: fdsautomation_sg,GroupId: sg-62dc5619}, {GroupName: basic_access_sg,GroupId: sg-d39070ab}],Attachment: {AttachmentId: eni-attach-1e77d9b5,DeviceIndex: 0,Status: attached,Att
achTime: Mon Sep 19 17:10:59 UTC 2016,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.60.5.254,PrivateDnsName:
It looks like it is querying AWS for instances matching a non-existant ID. No instance is created, and there are no errors in the log.
This went on for about an hour for us, with it retrying the launch every few seconds. It eventually was able to launch nodes, after all our existing nodes destroyed themselves (idle time).
