Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10986

Symlinks in a shelved project are archived as directories on Unix platforms

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      When archived symlinks are converted to directories in the shelved project archive. After unshelved the project has about twice the size of the original project. Potentially editing build descriptions and names in the unshelved project can cause discrepancies, since the builds information is duplicated.

      Steps to reproduce:
      1. Create an empty test project (in this example project name is "empty")
      2. Run one or more builds
      3. Open shell on jenkins server in the home directory(/var/lib/jenkins/). Open dir jobs/builds/empty
      4. Run commands :
      ls -la .
      ls -la builds .
      du -hs *
      Save the output on the screen
      5. Open "empty" project page. Open "Shelve Project" link , Click on "Shelve Project" button
      6. Open jenkins start page. Open "Shelved Project" link. Checkmark "empty" project. Click on "Unshelve Project" button (BTW should be "Unshelve Projects"
      7. Repeat step 4. Compare the output with that at the step 4
      Expected : outputs should be the same.
      Actual: All symlinks are replaced with dir copies, the project size increased

        Attachments

          Issue Links

            Activity

            Hide
            jameshowe James Howe added a comment - - edited

            From my testing, it appears that any job that's been unshelved will break Jenkins if loaded after 1.597, as the layout migration fails.

            Failed to update hudson.model.FreeStyleProject@61d2876c[Job1] lastSuccessfulBuild permalink for Job1 #7
            java.nio.file.DirectoryNotEmptyException: /var/lib/jenkins/jobs/Job1/builds/lastSuccessfulBuild
            at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242)
            at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
            at java.nio.file.Files.delete(Files.java:1079)

            Show
            jameshowe James Howe added a comment - - edited From my testing, it appears that any job that's been unshelved will break Jenkins if loaded after 1.597, as the layout migration fails. Failed to update hudson.model.FreeStyleProject@61d2876c [Job1] lastSuccessfulBuild permalink for Job1 #7 java.nio.file.DirectoryNotEmptyException: /var/lib/jenkins/jobs/Job1/builds/lastSuccessfulBuild at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242) at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) at java.nio.file.Files.delete(Files.java:1079)
            Hide
            tom_ghyselinck Tom Ghyselinck added a comment -

            Hi guys,

            This is a really big issue for us too.

            Using `.symlink` files can be tricky. There is always (an whether small or not) possibility that the `.symlink` file already exists....

            Why not using another archiver instead of `zip`?
            For example `tar` + `gzip` is supported on most platforms.

            Another safe procedure is to create the `zip` archive as follows:

            1. Add a subfolder and store the actual Job files here instead of in the archive root (excluding the symlinks!)
            2. Add a file containing all symlinks (relative to the Job root folder)

            I.e.

            <archive>.zip
              +- <actual_archive_contents>
              |    +- dir1
              |    +- dir2
              |    +- file1
              |    +- file2
              |    +- ...
              |
              +- symlinks.map
            

            where symlinks.map contains all the symlinks (relative to <actual_archive_contents>)

            Show
            tom_ghyselinck Tom Ghyselinck added a comment - Hi guys, This is a really big issue for us too. Using ` .symlink ` files can be tricky. There is always ( an whether small or not ) possibility that the ` .symlink ` file already exists.... Why not using another archiver instead of ` zip `? For example ` tar ` + ` gzip ` is supported on most platforms. Another safe procedure is to create the ` zip ` archive as follows: Add a subfolder and store the actual Job files here instead of in the archive root ( excluding the symlinks !) Add a file containing all symlinks ( relative to the Job root folder) I.e. <archive>.zip +- <actual_archive_contents> | +- dir1 | +- dir2 | +- file1 | +- file2 | +- ... | +- symlinks.map where symlinks.map contains all the symlinks ( relative to <actual_archive_contents> )
            Hide
            pierrebtz Pierre Beitz added a comment -

            2.1 release switched to a tar for the archive which means the symlinks will now be kept.

            Show
            pierrebtz Pierre Beitz added a comment - 2.1 release switched to a tar for the archive which means the symlinks will now be kept.
            Hide
            prifiz Valentin Baranov added a comment -

            Hi guys.

            When can we expect this fix to be released? We extremely need it

            Show
            prifiz Valentin Baranov added a comment - Hi guys. When can we expect this fix to be released? We extremely need it
            Hide
            pierrebtz Pierre Beitz added a comment -

            Valentin Baranov

            This issue is fixed in the plugin. There is however an issue in Jenkins core with the tar function breaking symlinks: https://issues.jenkins-ci.org/browse/JENKINS-52781

            I made a fix for this, and there is a PR waiting for review here: https://github.com/jenkinsci/jenkins/pull/3569

            Show
            pierrebtz Pierre Beitz added a comment - Valentin Baranov This issue is fixed in the plugin. There is however an issue in Jenkins core with the tar function breaking symlinks: https://issues.jenkins-ci.org/browse/JENKINS-52781 I made a fix for this, and there is a PR waiting for review here: https://github.com/jenkinsci/jenkins/pull/3569

              People

              • Assignee:
                pierrebtz Pierre Beitz
                Reporter:
                yashma Adil Akhund-Zade
              • Votes:
                10 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: