Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59216

LinkageError in ec2 spot fleet while performing jenkins UserRequest

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Major Major

      Hi everyone, apologize first for any information leak or mis-understanding, this is my first time to report issue in community. And I have toke merely all way I can to deal with it.

       

      [Environment]

      jenkins 2.176.3 with all plugins in latest version
      java version (both jenkins master and spot fleet) 1.8.0
      – plugins –
      ec2-fleet-plugin 1.10.2
      git-client 2.8.2
      junit 1.28
      --------------

      Jenkins master on ec2 r5.large, Amazon linux
      Spot fleet on r4.large / r5.large / r5.xlarge , Amazon linux

       

      [Scenario]

      I use jenkins with spot-fleet plugin for CI

      CI all pass in spot-fleet with initial ec2 mechine

      While ec2 scale out, CI fail in new instance with "remote call on  i-xxxxxx failed"

      Below is the log from instance of spot-fleet (retrieved from ${jenkinsHome/remoting/logs})

      remoting_log.txt 

       

      [Reproduce]

      3 CI job with identical codebase run at the same time
      a ec2-spot-fleet with init 2 instance

      In my case, 1 CI job take 2 ec2 instance, so one job will take the initial two instance, and the other jobs have to run in scaled out instances.

      • Result : CI pass in one job and fail in other two

      [More information]

      1. Below is the jenkinsfile with sensitive information replaced

      Jenkinsfile

      2. I face the problem in two scenario, because docker build and testing in different spot fleet

      scenario A:  

      building spot-fleet rotate ec2 every day, new ec2 instance fail to build 

      scenario B:

      is the reproduce experiment, happen in spot-fleet used in testing stage
      new ec2 (scaled out) fail in CI testing while initial instances success
      you can see pipeline with blueOcean in ec2_N_success/fail.png attachment

       

      3. Slave log viewed in jenkins
      ErrorInScaledOutInstance.pdf

        1. ec2_1_success.png
          156 kB
          TzuChin Hsu
        2. ec2_2_fail.png
          517 kB
          TzuChin Hsu
        3. ec2_3_fail.png
          872 kB
          TzuChin Hsu
        4. ec2_fail_build.png
          105 kB
          TzuChin Hsu
        5. ErrorInScaledOutInstance.pdf
          1.13 MB
          TzuChin Hsu
        6. Jenkinsfile
          14 kB
          TzuChin Hsu
        7. Jenkinsfile.txt
          14 kB
          TzuChin Hsu
        8. remoting_log.txt
          35 kB
          TzuChin Hsu

            terma Artem Stasiuk
            holis TzuChin Hsu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: