Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57258

Lost builds historical data after updating to Jenkins v2.175

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Cannot Reproduce
    • Component/s: core, pipeline
    • Labels:
      None
    • Environment:
      Jenkins v2.175 / Unix master / Java 64bit 1.8.0_192
    • Similar Issues:

      Description

      I'm using Multibranch Pipeline to automatically build changes in my Enterprise GitHub Organization repositories.

      After updating Jenkins to 2.175 and restarting, ALL the successful builds of ALL the branches of ALL my GitHub org repositories (that's around 75 repositories and approximately 2,000 builds) have been retriggered and instantly failed. All the builds failed with the same error, see attached console log. Now, all the projects and branches and builds are marked as Failed, and some build information such as the build date appear to be lost (see screenshots).

      In the example log, we can see the build was successfully completed a week ago, but restarted and failed today after restarting Jenkins.  I can consistenly reproduce this issue by re-building a project and restarting Jenkins.

      The issue does not happen if I downgrade to Jenkins 2.174.

        Attachments

        1. branches.PNG
          branches.PNG
          68 kB
        2. build_after.xml
          16 kB
        3. build_before.xml
          25 kB
        4. build_history.PNG
          build_history.PNG
          17 kB
        5. failed_build_console_log.txt
          3 kB

          Activity

          Hide
          nico_echegut Nicolas Echegut added a comment -

          I grabbed the build.xml from a build before and after restart. 

          I updated workflow-support from 3.2 to 3.3 right before upgrading Jenkins, but restarted Jenkins in-between. I was already on the latest version of workflow-job-plugin and workflow-cps-plugin.

          Show
          nico_echegut Nicolas Echegut added a comment - I grabbed the build.xml from a build before and after restart.  I updated workflow-support from 3.2 to 3.3 right before upgrading Jenkins, but restarted Jenkins in-between. I was already on the latest version of workflow-job-plugin and workflow-cps-plugin.
          Hide
          dnusbaum Devin Nusbaum added a comment -

          Based on those files, the pre-update run and flow execution have a result, the run is marked as completed and the flow execution is marked as done, which all looks good. The only things that seem off is that the flow execution does not have persistedClean = true, but that by itself shouldn't be a problem, and that the CloverHtmlBuildAction and checkout information were nulled out as part of the update. This does remind me of JENKINS-57254, which was also reported against 2.175 recently, although apparently that was caused by the SCoverage plugin which doesn't appear in your build.xml files. Do you see any errors in your Jenkins system logs around the time of the upgrade?

          Show
          dnusbaum Devin Nusbaum added a comment - Based on those files, the pre-update run and flow execution have a result, the run is marked as completed and the flow execution is marked as done, which all looks good. The only things that seem off is that the flow execution does not have persistedClean = true , but that by itself shouldn't be a problem, and that the CloverHtmlBuildAction and checkout information were nulled out as part of the update. This does remind me of JENKINS-57254 , which was also reported against 2.175 recently, although apparently that was caused by the SCoverage plugin which doesn't appear in your build.xml files. Do you see any errors in your Jenkins system logs around the time of the upgrade?
          Hide
          nico_echegut Nicolas Echegut added a comment -

          I don't see anything useful in the log and error files from the day of the upgrade.

          I'm not using the SCoverage plugin.

          Show
          nico_echegut Nicolas Echegut added a comment - I don't see anything useful in the log and error files from the day of the upgrade. I'm not using the SCoverage plugin.
          Hide
          danielbeck Daniel Beck added a comment -

          Does this problem still occur with Jenkins 2.176? I assume instances were restored from backup when this happened, so what happens when you skip 2.175?

          Show
          danielbeck Daniel Beck added a comment - Does this problem still occur with Jenkins 2.176? I assume instances were restored from backup when this happened, so what happens when you skip 2.175?
          Hide
          nico_echegut Nicolas Echegut added a comment -

          Unfortunately this Jenkins instance did not have any backup so nothing was recovered. I skipped 2.175 and 2.176 and upgraded to 2.177. The good news is the issue with builds automatically restarting and failing after a Jenkins restart is not occurring anymore. The bad news is we probably will never know what happened now  It may have been related to JENKINS-57254.

          Anyway, I will go ahead and close this Jira since the issue cannot be reproduced anymore. Thanks for looking into this.

          Show
          nico_echegut Nicolas Echegut added a comment - Unfortunately this Jenkins instance did not have any backup so nothing was recovered. I skipped 2.175 and 2.176 and upgraded to 2.177. The good news is the issue with builds automatically restarting and failing after a Jenkins restart is not occurring anymore. The bad news is we probably will never know what happened now   It may have been related to  JENKINS-57254 . Anyway, I will go ahead and close this Jira since the issue cannot be reproduced anymore. Thanks for looking into this.

            People

            • Assignee:
              Unassigned
              Reporter:
              nico_echegut Nicolas Echegut
            • Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: