Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10629

Tar implimentation can't handle > 8GB and doesn't error out.

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      During archiving from a slave the untaring can fail with files > 8GB. This is because the tar creator makes a corrupt archive. I think it doesn't properly null terminate the size value when the file size is gets to the 12th digit - which is a violation of tar spec (from Wikipedia tar article: Numeric values are encoded in octal numbers using ASCII digits, with leading zeroes. For historical reasons, a final NUL or space character should be used. Thus although there are 12 bytes reserved for storing the file size, only 11 octal digits can be stored. This gives a maximum file size of 8 gigabytes on archived files.)
      Changing to gnutar might be nice to handle large files, or at least making the library error out when it hits large files rather than happily building corrupt archives.

      Error caused by this bug (due to reading the corrupt header):

      Archiving artifacts
      ERROR: Publisher hudson.tasks.ArtifactArchiver aborted due to exception
      java.lang.IllegalArgumentException: Negative time
      at java.io.File.setLastModified(File.java:1258)
      at hudson.FilePath.readFromTar(FilePath.java:1605)
      at hudson.FilePath.copyRecursiveTo(FilePath.java:1530)
      at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:117)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:649)
      at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:625)
      at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:603)
      at hudson.model.Build$RunnerImpl.post2(Build.java:161)
      at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:572)
      at hudson.model.Run.run(Run.java:1386)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:145)
      Finished: FAILURE

        Attachments

          Issue Links

            Activity

            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kanstantsin Shautsou
            Path:
            core/pom.xml
            core/src/main/java/hudson/FilePath.java
            core/src/main/java/hudson/org/apache/tools/tar/TarInputStream.java
            core/src/main/java/hudson/org/apache/tools/tar/TarOutputStream.java
            core/src/main/java/hudson/util/io/TarArchiver.java
            http://jenkins-ci.org/commit/jenkins/6e5740eb5be85ede0cbd01d900cb6d5fbf71ee62
            Log:
            Revert "Revert "JENKINS-10629 - Migrate the Tar archives handling code to commons-compress""

            This reverts commit 8f1280a85c54ea6150b15c38303464ab23b32e92.

            (cherry picked from commit 03bd5959cb94a0e626c5ace910d14c3083a9db2c)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kanstantsin Shautsou Path: core/pom.xml core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/org/apache/tools/tar/TarInputStream.java core/src/main/java/hudson/org/apache/tools/tar/TarOutputStream.java core/src/main/java/hudson/util/io/TarArchiver.java http://jenkins-ci.org/commit/jenkins/6e5740eb5be85ede0cbd01d900cb6d5fbf71ee62 Log: Revert "Revert " JENKINS-10629 - Migrate the Tar archives handling code to commons-compress"" This reverts commit 8f1280a85c54ea6150b15c38303464ab23b32e92. (cherry picked from commit 03bd5959cb94a0e626c5ace910d14c3083a9db2c)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kanstantsin Shautsou
            Path:
            core/src/main/java/hudson/util/io/TarArchiver.java
            http://jenkins-ci.org/commit/jenkins/2b63fea84739c1ae900e6997efc18658bcdbaee0
            Log:
            Revert "Revert "FIXED JENKINS-10629] - Enable BigNumber mode to support archiving of files with size >8Gb""

            This reverts commit ee57300963ca0137565f61fe314d459b627ad74b.

            (cherry picked from commit 2ef29215c2b311a9d5c2e515268285b5623ce833)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kanstantsin Shautsou Path: core/src/main/java/hudson/util/io/TarArchiver.java http://jenkins-ci.org/commit/jenkins/2b63fea84739c1ae900e6997efc18658bcdbaee0 Log: Revert "Revert "FIXED JENKINS-10629 ] - Enable BigNumber mode to support archiving of files with size >8Gb"" This reverts commit ee57300963ca0137565f61fe314d459b627ad74b. (cherry picked from commit 2ef29215c2b311a9d5c2e515268285b5623ce833)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kanstantsin Shautsou
            Path:
            core/src/main/java/hudson/util/io/TarArchiver.java
            core/src/test/java/hudson/FilePathTest.java
            http://jenkins-ci.org/commit/jenkins/fd1e392fd4cb9d9feade9da7540c00921d32d816
            Log:
            [FIXED JENKINS-10629] Unbroke stream with new tar implementation.

            Causes bytes lost and truncated tar archive.
            TarBuffer not used in TarArchiveOutputStream.

            (cherry picked from commit 3e187a026408d5ca74202f7e26dd565cde2e87d0)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kanstantsin Shautsou Path: core/src/main/java/hudson/util/io/TarArchiver.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/fd1e392fd4cb9d9feade9da7540c00921d32d816 Log: [FIXED JENKINS-10629] Unbroke stream with new tar implementation. Causes bytes lost and truncated tar archive. TarBuffer not used in TarArchiveOutputStream. (cherry picked from commit 3e187a026408d5ca74202f7e26dd565cde2e87d0)
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #4358
            Revert "Revert "JENKINS-10629 - Migrate the Tar archives handling code to commons-compress"" (Revision 6e5740eb5be85ede0cbd01d900cb6d5fbf71ee62)
            Revert "Revert "FIXED JENKINS-10629] - Enable BigNumber mode to support archiving of files with size >8Gb"" (Revision 2b63fea84739c1ae900e6997efc18658bcdbaee0)
            [FIXED JENKINS-10629] Unbroke stream with new tar implementation. (Revision fd1e392fd4cb9d9feade9da7540c00921d32d816)

            Result = UNSTABLE
            ogondza : 6e5740eb5be85ede0cbd01d900cb6d5fbf71ee62
            Files :

            • core/src/main/java/hudson/org/apache/tools/tar/TarInputStream.java
            • core/src/main/java/hudson/FilePath.java
            • core/src/main/java/hudson/util/io/TarArchiver.java
            • core/src/main/java/hudson/org/apache/tools/tar/TarOutputStream.java
            • core/pom.xml

            ogondza : 2b63fea84739c1ae900e6997efc18658bcdbaee0
            Files :

            • core/src/main/java/hudson/util/io/TarArchiver.java

            ogondza : fd1e392fd4cb9d9feade9da7540c00921d32d816
            Files :

            • core/src/test/java/hudson/FilePathTest.java
            • core/src/main/java/hudson/util/io/TarArchiver.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #4358 Revert "Revert " JENKINS-10629 - Migrate the Tar archives handling code to commons-compress"" (Revision 6e5740eb5be85ede0cbd01d900cb6d5fbf71ee62) Revert "Revert "FIXED JENKINS-10629 ] - Enable BigNumber mode to support archiving of files with size >8Gb"" (Revision 2b63fea84739c1ae900e6997efc18658bcdbaee0) [FIXED JENKINS-10629] Unbroke stream with new tar implementation. (Revision fd1e392fd4cb9d9feade9da7540c00921d32d816) Result = UNSTABLE ogondza : 6e5740eb5be85ede0cbd01d900cb6d5fbf71ee62 Files : core/src/main/java/hudson/org/apache/tools/tar/TarInputStream.java core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/util/io/TarArchiver.java core/src/main/java/hudson/org/apache/tools/tar/TarOutputStream.java core/pom.xml ogondza : 2b63fea84739c1ae900e6997efc18658bcdbaee0 Files : core/src/main/java/hudson/util/io/TarArchiver.java ogondza : fd1e392fd4cb9d9feade9da7540c00921d32d816 Files : core/src/test/java/hudson/FilePathTest.java core/src/main/java/hudson/util/io/TarArchiver.java
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: tfennelly
            Path:
            test/src/main/java/hudson/core/PluginManagerOverrideTest.java
            test/src/main/resources/hudson/core/PluginManagerOverrideTest/BasicPluginManagerOverride/newview.jelly
            http://jenkins-ci.org/commit/jenkins-test-harness/9823d43e2d9ebd9cccad7efca8eb965bc4aafedf
            Log:
            Merge branch 'master' into upgrade-htmlunit

            • master:
              commons-compress -> 1.10
              Noting #1788
              Remove hardcode in tar test
              [FIXED JENKINS-10629] Unbroke stream with new tar implementation.
              Revert "Revert "FIXED JENKINS-10629] - Enable BigNumber mode to support archiving of files with size >8Gb""
              Revert "Revert "JENKINS-10629 - Migrate the Tar archives handling code to commons-compress""
              Fix at-since from PR #1788
              Merge PR #1788: Make plugin manager pluggable

            Originally-Committed-As: 3f98011bd4aae0ee6956d264c952782452389f79

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: tfennelly Path: test/src/main/java/hudson/core/PluginManagerOverrideTest.java test/src/main/resources/hudson/core/PluginManagerOverrideTest/BasicPluginManagerOverride/newview.jelly http://jenkins-ci.org/commit/jenkins-test-harness/9823d43e2d9ebd9cccad7efca8eb965bc4aafedf Log: Merge branch 'master' into upgrade-htmlunit master: commons-compress -> 1.10 Noting #1788 Remove hardcode in tar test [FIXED JENKINS-10629] Unbroke stream with new tar implementation. Revert "Revert "FIXED JENKINS-10629 ] - Enable BigNumber mode to support archiving of files with size >8Gb"" Revert "Revert " JENKINS-10629 - Migrate the Tar archives handling code to commons-compress"" Fix at-since from PR #1788 Merge PR #1788: Make plugin manager pluggable Originally-Committed-As: 3f98011bd4aae0ee6956d264c952782452389f79

              People

              • Assignee:
                integer Kanstantsin Shautsou
                Reporter:
                keibaker keith baker
              • Votes:
                6 Vote for this issue
                Watchers:
                27 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: