Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18866

Symlink prevens backup from ThinBackup

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Blocker
    • Resolution: Unresolved
    • Component/s: thinbackup-plugin
    • Labels:
      None
    • Environment:
      windows7 & 8
    • Similar Issues:

      Description

      Removal of the symlinks lastFailedBuild,lastUnstableBuild and lastUnsuccessfulBuild does result in a correct backup.
      Error message:
      Jul 22, 2013 1:13:27 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork backupNow
      SEVERE: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path 'c:\work\backup'.
      java.io.FileNotFoundException: Source 'C:\Program Files (x86)\Jenkins\jobs\Build Job 1\builds\lastFailedBuild' does not exist
      at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866)
      at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:814)
      at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildFiles(HudsonBackup.java:302)
      at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildsFor(HudsonBackup.java:270)
      at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJob(HudsonBackup.java:208)
      at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobsDirectory(HudsonBackup.java:196)
      at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobs(HudsonBackup.java:175)
      at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:142)
      at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:87)
      at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76)
      at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:54)
      at java.lang.Thread.run(Unknown Source)

        Attachments

          Issue Links

            Activity

            frankroghair Frank Roghair created issue -
            Hide
            frankroghair Frank Roghair added a comment -

            version is 1.523

            Show
            frankroghair Frank Roghair added a comment - version is 1.523
            frankroghair Frank Roghair made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-17681 [ JENKINS-17681 ]
            Hide
            axelabs Bart Bialek added a comment - - edited

            I have the same problem with Jenkins 1.550 and thinBackup 1.7 on CentOS 6.3:

            Jul 11, 2014 4:22:43 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork backupNow
            SEVERE: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path '/var/lib/jenkins/backups'.
            java.io.FileNotFoundException: Source '/var/lib/jenkins/jobs/Cron_AWD_Sitemap_Generation/builds/lastFailedBuild' does not exist
            at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866)
            at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:814)
            at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildFiles(HudsonBackup.java:280)
            at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildsFor(HudsonBackup.java:263)
            at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJob(HudsonBackup.java:202)
            at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobsDirectory(HudsonBackup.java:190)
            at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobs(HudsonBackup.java:169)
            at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:136)
            at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:87)
            at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76)
            at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:54)
            at java.lang.Thread.run(Thread.java:722)

            The lastFailed build link points to a non existant file:
            /var/lib/jenkins/jobs/Cron_AWD_Sitemap_Generation/builds/lastFailedBuild -> -1

            There are many other jobs that have these broken symlinks. I will manually remove them, but it would be nice for thinBackup to ignore them instead.

            Show
            axelabs Bart Bialek added a comment - - edited I have the same problem with Jenkins 1.550 and thinBackup 1.7 on CentOS 6.3: Jul 11, 2014 4:22:43 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork backupNow SEVERE: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path '/var/lib/jenkins/backups'. java.io.FileNotFoundException: Source '/var/lib/jenkins/jobs/Cron_AWD_Sitemap_Generation/builds/lastFailedBuild' does not exist at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866) at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:814) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildFiles(HudsonBackup.java:280) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildsFor(HudsonBackup.java:263) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJob(HudsonBackup.java:202) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobsDirectory(HudsonBackup.java:190) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobs(HudsonBackup.java:169) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:136) at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:87) at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76) at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:54) at java.lang.Thread.run(Thread.java:722) The lastFailed build link points to a non existant file: /var/lib/jenkins/jobs/Cron_AWD_Sitemap_Generation/builds/lastFailedBuild -> -1 There are many other jobs that have these broken symlinks. I will manually remove them, but it would be nice for thinBackup to ignore them instead.
            Hide
            axelabs Bart Bialek added a comment -

            thinBackup 1.7.4 seems to have fixed this error for me.

            Show
            axelabs Bart Bialek added a comment - thinBackup 1.7.4 seems to have fixed this error for me.
            danielbeck Daniel Beck made changes -
            Component/s thinBackup [ 15780 ]
            Component/s plugin [ 15491 ]
            Hide
            jaytradertee Jack T added a comment -

            I am running into this issue with thinBackup 1.7.4 is not saving all the symbolic links in Windows 2008. The lastFailedBuild, lastUnstableBuild and lastUnsuccessfulBuild symbolic links are retained but the ones pointing to the build directories are not kept. For those build directories it copies the contents of the symbolic links but doesn't keep the symbolic link itself. This is fine if you restore the backup to the original location, but if you try to change paths it causes issues.

            Not a big issue, but it took me some time to figure out why all my backups had wrong dates and times. My workaround is to use xcopy with the /B switch to keeps all the symbolic links.

            Show
            jaytradertee Jack T added a comment - I am running into this issue with thinBackup 1.7.4 is not saving all the symbolic links in Windows 2008. The lastFailedBuild, lastUnstableBuild and lastUnsuccessfulBuild symbolic links are retained but the ones pointing to the build directories are not kept. For those build directories it copies the contents of the symbolic links but doesn't keep the symbolic link itself. This is fine if you restore the backup to the original location, but if you try to change paths it causes issues. Not a big issue, but it took me some time to figure out why all my backups had wrong dates and times. My workaround is to use xcopy with the /B switch to keeps all the symbolic links.
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 150313 ] JNJira + In-Review [ 177608 ]
            Hide
            piotrk Piotr Krukowiecki added a comment - - edited

            Same issue.

            Additionally, this causes ThinBackup to not delete backups set ("Max number of backup sets" option ignored). This filled our build server disks (40 GB of ThinBackup logs).

            That is, we have set limited number of backups set to be kept. But it seems the backup sets are not deleted. When I click "Backup Now", a new backup folder is created (for example "FULL-2018-03-02_09-26"), but old folders are not deleted. In Jenkins logs I can see following:

            Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path '/tmp/jenkins_backup'. java.io.FileNotFoundException: /var/lib/jenkins/jobs/Reference Tests - Run Defragmentation/lastSuccessful (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:1138) at org.apache.commons.io.FileUtils.doCopyDirectory(FileUtils.java:1428) at org.apache.commons.io.FileUtils.doCopyDirectory(FileUtils.java:1426) at org.apache.commons.io.FileUtils.doCopyDirectory(FileUtils.java:1426) at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1389) at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1310) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupAdditionalFiles(HudsonBackup.java:289) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:171) at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:89) at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76) at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:53) at java.lang.Thread.run(Thread.java:748)

            The "/var/lib/jenkins/jobs/Reference Tests - Run Defragmentation/lastSuccessful" is a link to non-existing file (because the job was not run yet, or simply the builds were deleted).

            After deleting all links to non-existing files, running ThinBackup didn't result in any exceptions in Jenkins log, and old backup sets were deleted ("Max number of backup sets" honored).

            Also, it looks like the created backup is not complete (backup process stops when exception is thrown?)

            Due to this, I am increasing priority to Critical, as backup process is unreliable, and the broken links are common / normal in jenkins.

             

             

             

             

             

            Show
            piotrk Piotr Krukowiecki added a comment - - edited Same issue. Additionally, this causes ThinBackup to not delete backups set ("Max number of backup sets" option ignored). This filled our build server disks (40 GB of ThinBackup logs). That is, we have set limited number of backups set to be kept. But it seems the backup sets are not deleted. When I click "Backup Now", a new backup folder is created (for example "FULL-2018-03-02_09-26"), but old folders are not deleted. In Jenkins logs I can see following: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path '/tmp/jenkins_backup'. java.io.FileNotFoundException: /var/lib/jenkins/jobs/Reference Tests - Run Defragmentation/lastSuccessful (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:1138) at org.apache.commons.io.FileUtils.doCopyDirectory(FileUtils.java:1428) at org.apache.commons.io.FileUtils.doCopyDirectory(FileUtils.java:1426) at org.apache.commons.io.FileUtils.doCopyDirectory(FileUtils.java:1426) at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1389) at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1310) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupAdditionalFiles(HudsonBackup.java:289) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:171) at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:89) at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76) at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:53) at java.lang.Thread.run(Thread.java:748) The "/var/lib/jenkins/jobs/Reference Tests - Run Defragmentation/lastSuccessful" is a link to non-existing file (because the job was not run yet, or simply the builds were deleted). After deleting all links to non-existing files, running ThinBackup didn't result in any exceptions in Jenkins log, and old backup sets were deleted ("Max number of backup sets" honored). Also, it looks like the created backup is not complete (backup process stops when exception is thrown?) Due to this, I am increasing priority to Critical, as backup process is unreliable, and the broken links are common / normal in jenkins.          
            piotrk Piotr Krukowiecki made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            Hide
            jglick Jesse Glick added a comment -

            Probably obsolete as of JENKINS-37862. (No new build symlinks are created or updated. See the upgrade guide; existing symlinks are not removed automatically.)

            Show
            jglick Jesse Glick added a comment - Probably obsolete as of JENKINS-37862 . (No new build symlinks are created or updated. See the upgrade guide; existing symlinks are not removed automatically.)
            jglick Jesse Glick made changes -
            Link This issue duplicates JENKINS-37862 [ JENKINS-37862 ]
            angelfighter Helene W made changes -
            Priority Critical [ 2 ] Blocker [ 1 ]
            Hide
            angelfighter Helene W added a comment -

            Hi,

            I have the same issue that Piotr Krukowiecki described.

            All of our backups are corrupt due to errors like this:

            Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path 'E:\JBackup'.
            java.io.IOException: Failed to list contents of E:\Jenkins\jobs\Experiments\jobs\Company\jobs\RollingBuildRerun\jobs\Rerun_Rolling_Build_param\lastSuccessful

             

            This also causes that the moving of old backups to ZIP files is not performed.

             

            I'm setting this bug to Blocker as the plugin is simply not usable if it does not create a backup.

             

            System data:

            Windows Server 2008 R2 SP1

            Jenkins version 2.195

            ThinBackup Plugin version 1.9

             

             

            Show
            angelfighter Helene W added a comment - Hi, I have the same issue that Piotr Krukowiecki described. All of our backups are corrupt due to errors like this: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path 'E:\JBackup'. java.io.IOException: Failed to list contents of E:\Jenkins\jobs\Experiments\jobs\Company\jobs\RollingBuildRerun\jobs\Rerun_Rolling_Build_param\lastSuccessful   This also causes that the moving of old backups to ZIP files is not performed.   I'm setting this bug to Blocker as the plugin is simply not usable if it does not create a backup.   System data: Windows Server 2008 R2 SP1 Jenkins version 2.195 ThinBackup Plugin version 1.9    

              People

              • Assignee:
                Unassigned
                Reporter:
                frankroghair Frank Roghair
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: