-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Jenkins: 1.609.3
ec2-plugin: 1.31 (probably an issue in 1.30 as well)
-
-
1.41
We have multiple Jenkins instances running, for testing new configuration before promoting to our primary jenkins. Our primary Jenkins is using ec2-plugin 1.29. On our test Jenkins, we have 1.31 and began experiencing an issue where our test Jenkins would be trying to create a new slave, it would describe-instances and find the slave attached to our primary Jenkins. Then, it would finish the job and eventually and delete the instance.
This can cause issues where jobs need to run in isolation as well as issues where slaves terminate when they should not. I tracked it down to these lines:
Describe Instances: https://github.com/jenkinsci/ec2-plugin/blob/f0168050822a461ba009caf1a986f5d27db65804/src/main/java/hudson/plugins/ec2/SlaveTemplate.java#L539-L545
Check if instance is a match: https://github.com/jenkinsci/ec2-plugin/blob/f0168050822a461ba009caf1a986f5d27db65804/src/main/java/hudson/plugins/ec2/SlaveTemplate.java#L380-L393
Logs below are from our jenkins logs. I have put "..." at the end of some long lines with stuff specific to our setup
Creation in our primary jenkins:
Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-86b49bec]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-c9c906a8]}, {Name: instance.group-id,Values: [sg-9ae805ff, sg-cce2a8a9, sg-73ae6417]}, {Name: key-name,Values: [rdkeys]}, {Name: instance-type,Values: [m3.large]}, {Name: tag:component,Values: [dxjenkins_cloud_slave-slave-ui]}, {Name: tag:Name,Values: [dxjenkins]}, {Name: tag:Environment,Values: [dev]}, {Name: tag:Project,Values: [jenkins]}, {Name: tag:Customer,Values: [dataxu]}, {Name: tag:Team,Values: [qa]}, {Name: instance-state-name,Values: [stopped, stopping]}],}
No existing instance found - created: {InstanceId: i-01809284,ImageId: ami-86b49bec, ...
Feb 17, 2016 8:55:33 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand
INFO: No existing instance found - created: {InstanceId: i-01809284,ImageId: ami-86b49bec, ...
Finding it in our jenkins-test
Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-86b49bec]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-c9c906a8]}, {Name: instance.group-id,Values: [sg-9ae805ff, sg-cce2a8a9, sg-73ae6417]}, {Name: key-name,Values: [rdkeys]}, {Name: instance-type,Values: [m3.large]}, {Name: tag:component,Values: [dxjenkins_cloud_slave-slave-ui]}, {Name: tag:Name,Values: [dxjenkins]}, {Name: tag:Environment,Values: [dev]}, {Name: tag:Project,Values: [jenkins]}, {Name: tag:Customer,Values: [dataxu]}, {Name: tag:Team,Values: [qa]}],}
checkInstance: {InstanceId: i-01809284,ImageId: ami-86b49bec, ...
true - Instance has no node, but can be used
