Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13990

Polling find old already considered baselines

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: config-rotator-plugin
    • Labels:
      None
    • Environment:
      All OS, Jenkins master-slave configuration
    • Similar Issues:

      Description

      Occassionally we see an error related to polling starting an job though there should be no new baselines on ClearCase components configured.

      In the following example we use two components. There are baselines model-1, model-2, model-3, client-1, client-2, client-3.

      We see jobs finding the baselines in correct order ending last configuration model-3 and client-3. After some random time, polling finds an old configuration and starts a new build.
      By the design the new build i failed, as the configuration is not new and the build get no new baselines when building.

      You will se failed jobs appearing and pollings logs like below. Build #11 is the first to fail, after finding no new baselines (though the polling log did).
      Build #10 was success.

      ---------- snip: polling log from build #11 ----------
      Polling Log
      View as plain text

      This page captures the polling log that triggered this build.

      Started on May 31, 2012 9:41:07 PM
      [ConfigRotator] Polling
      [ConfigRotator] Project configuration was not null
      [ConfigRotator] Configuration is not null and was not reconfigured
      Index: 0, Size: 0
      The configuration is:

      • component:Model@\ConfigRotatorProofOfConceptJob2126_PVOB, stream:ConfigRotatorProofOfConceptJob2126_one_int@\ConfigRotatorProofOfConceptJob2126_PVOB, model-3@\ConfigRotatorProofOfConceptJob2126_PVOB
      • component:Clientapp@\ConfigRotatorProofOfConceptJob2126_PVOB, stream:ConfigRotatorProofOfConceptJob2126_one_int@\ConfigRotatorProofOfConceptJob2126_PVOB, client-3@\ConfigRotatorProofOfConceptJob2126_PVOB

      Done. Took 2.2 sec
      Changes found

      ---------- snap: polling log from build #11 ----------

      ---------- snip: console output from build #11 ----------

      Started by an SCM change
      Building remotely on cc-nightcrawler in workspace C:\jslave\workspace\ConfigRotatorProofOfConceptJob2126
      Config-rotator version: 0.1.4
      [DEBUG] Getting last result
      [DEBUG] Action was NOT null
      [DEBUG] Obtaining new configuration based on old
      [DEBUG] Foreach configuration component
      [DEBUG] CONFIG: model-3@\ConfigRotatorProofOfConceptJob2126_PVOB@INITIAL(false/false)
      [DEBUG] Wasn't fixed: model-3@\ConfigRotatorProofOfConceptJob2126_PVOB
      [DEBUG] No baselines found: Index: 0, Size: 0
      Index: 0, Size: 0
      [DEBUG] CONFIG: client-3@\ConfigRotatorProofOfConceptJob2126_PVOB@INITIAL(false/false)
      [DEBUG] Wasn't fixed: client-3@\ConfigRotatorProofOfConceptJob2126_PVOB
      [DEBUG] No baselines found: Index: 0, Size: 0
      Index: 0, Size: 0
      [ConfigRotator] No new baselines
      [DEBUG] Getting last result
      ERROR: No new baselines found!
      Retrying after 10 seconds
      Config-rotator version: 0.1.4
      [DEBUG] Getting last result
      [DEBUG] Getting last result
      [DEBUG] Action was NOT null
      [DEBUG] Action was NOT null
      [DEBUG] Obtaining new configuration based on old
      [DEBUG] Obtaining new configuration based on old
      [DEBUG] Foreach configuration component
      [DEBUG] Foreach configuration component
      [DEBUG] CONFIG: model-3@\ConfigRotatorProofOfConceptJob2126_PVOB@INITIAL(false/false)
      [DEBUG] CONFIG: model-3@\ConfigRotatorProofOfConceptJob2126_PVOB@INITIAL(false/false)
      [DEBUG] Wasn't fixed: model-3@\ConfigRotatorProofOfConceptJob2126_PVOB
      [DEBUG] Wasn't fixed: model-3@\ConfigRotatorProofOfConceptJob2126_PVOB
      [DEBUG] No baselines found: Index: 0, Size: 0
      [DEBUG] No baselines found: Index: 0, Size: 0
      Index: 0, Size: 0
      [DEBUG] CONFIG: client-3@\ConfigRotatorProofOfConceptJob2126_PVOB@INITIAL(false/false)
      [DEBUG] CONFIG: client-3@\ConfigRotatorProofOfConceptJob2126_PVOB@INITIAL(false/false)
      [DEBUG] Wasn't fixed: client-3@\ConfigRotatorProofOfConceptJob2126_PVOB
      [DEBUG] Wasn't fixed: client-3@\ConfigRotatorProofOfConceptJob2126_PVOB
      [DEBUG] No baselines found: Index: 0, Size: 0
      [DEBUG] No baselines found: Index: 0, Size: 0
      Index: 0, Size: 0
      [ConfigRotator] No new baselines
      [DEBUG] Getting last result
      [DEBUG] Getting last result
      ERROR: No new baselines found!
      [DEBUG] SCM is part of ConfigRotator
      [DEBUG] SCM is part of ConfigRotator
      [DEBUG] Action object is: null
      [DEBUG] Action object is: null
      [ConfigRotator] Action was null, unable to set compatability of configuration
      Finished: FAILURE
      ---------- snap: console output from build #11 ----------

        Attachments

          Activity

          Hide
          bue Bue Petersen added a comment - - edited

          The issue is related to threading problems in our dependencies (praqmajutils and ClearCase Orienten Open Library). We are working on fixing these issues.

          WORK-AROUND:
          There is a workaround for this polling issue: make sure to configure your Config Rotator jobs, so that only one job polls and start a build at the time. Eg. if job1 polls 10 minutes paste every hour, and a job1 build takes up to 10 minutes, than make sure job2 start polling 22 minutes past every hour.

          Show
          bue Bue Petersen added a comment - - edited The issue is related to threading problems in our dependencies (praqmajutils and ClearCase Orienten Open Library). We are working on fixing these issues. WORK-AROUND: There is a workaround for this polling issue: make sure to configure your Config Rotator jobs, so that only one job polls and start a build at the time. Eg. if job1 polls 10 minutes paste every hour, and a job1 build takes up to 10 minutes, than make sure job2 start polling 22 minutes past every hour.
          Hide
          bue Bue Petersen added a comment -

          We fixed a threading issues in our libraries, and we made sure polling does not start a new build while an existing build is in queue or building.

          We have tested with over 100 jobs polling every minute in a master/master setup with 12 executers.

          Show
          bue Bue Petersen added a comment - We fixed a threading issues in our libraries, and we made sure polling does not start a new build while an existing build is in queue or building. We have tested with over 100 jobs polling every minute in a master/master setup with 12 executers.

            People

            • Assignee:
              bue Bue Petersen
              Reporter:
              bue Bue Petersen
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: