-
Bug
-
Resolution: Fixed
-
Critical
-
None
-
1.551 ubuntu native package, ec2 plugin v 1.20
I created a slave with a spot instance in ec2 config, however I shut it down over a weekend. Come monday the instance ID isn't recognized by aws, and it causes the ec2 plugin to crash and no longer function (no new slaves, etc).
Feb 24, 2014 11:55:24 PM hudson.plugins.ec2.EC2Cloud provision
WARNING: Failed to count the # of live instances on EC2
Status Code: 400, AWS Service: AmazonEC2, AWS Request ID: 48ef733c-28c1-423e-9900-62de12315457, AWS Error Code: InvalidSpotInstanceRequestID.NotFound, AWS Error Message: The spot instance request ID 'sir-f43fdc4f' does not exist
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:614)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:312)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:165)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:6047)
at com.amazonaws.services.ec2.AmazonEC2Client.describeSpotInstanceRequests(AmazonEC2Client.java:3068)
at hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:331)
at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281)
at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51)
at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:366)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
1)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
And now I get this over and over:
Feb 24, 2014 11:55:53 PM hudson.node_monitors.ResponseTimeMonitor$1 monitor
WARNING: Making 541693b6-21dc-4c5d-b98b-717d03349d20 offline because it’s not responding
however it will not launch any new slaves, even if I have the executor limit on the ec2 plugin set to 20.
I'm not sure how to clear the history of all executors, so I am stuck without any slaves now.