Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-23507

FilePath.installIfNecessaryFrom does not follow HTTP 30x redirects

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: core
    • Labels:
      None
    • Similar Issues:

      Description

      I'm using a tool installer which, for some downloads, the URL returns a 302 redirect.

      However, Jenkins reads the contents of the redirect page (i.e. some text like "This page has moved to http://foo/bar") and attempts to parse that as the zip/tar/whatever file.

      The stacktrace users see when this happens is pretty cryptic:

      Unpacking Go from http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3 on Jenkins
      FATAL: Failed to install http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3
      hudson.util.IOException2: Failed to install http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3
      	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768)
      	at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:53)
      	at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:61)
      	at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:107)
      	at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:204)
      	at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:41)
      	at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34)
              ...
      Caused by: hudson.util.IOException2: Failed to unpack http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz (28 bytes read of total -1)
      	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:762)
      	... 12 more
      Caused by: hudson.util.IOException2: Failed to extract input stream
      	at hudson.FilePath.readFromTar(FilePath.java:2071)
      	...
      Caused by: java.io.IOException: incorrect header check
      	at com.jcraft.jzlib.InflaterInputStream.read(InflaterInputStream.java:112)
      	at org.apache.tools.tar.TarBuffer.readBlock(TarBuffer.java:257)
      	...
      

      I will try and fix the tool installer in question to resolve the end URL, but this isn't a great idea in the long run — Jenkins should follow the 30x redirects itself.

        Attachments

          Issue Links

            Activity

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: 15knots Path: cmake.groovy http://jenkins-ci.org/commit/backend-crawler/9e81b09be1e950d2feed836fc5fe9bb0ae92ad00 Log: Workarounbd for JENKINS-23507 Compare: https://github.com/jenkinsci/backend-crawler/compare/2dd4c0054441...9e81b09be1e9
            Hide
            vlatombe Vincent Latombe added a comment -
            Show
            vlatombe Vincent Latombe added a comment - Filed https://github.com/jenkinsci/jenkins/pull/2331 to fix this issue.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Vincent Latombe
            Path:
            core/src/main/java/hudson/FilePath.java
            core/src/test/java/hudson/FilePathTest.java
            http://jenkins-ci.org/commit/jenkins/23a37feec061c4def89b1ae65976b55ae7bc3be4
            Log:
            JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Vincent Latombe Path: core/src/main/java/hudson/FilePath.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/23a37feec061c4def89b1ae65976b55ae7bc3be4 Log: JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Vincent Latombe
            Path:
            core/src/main/java/hudson/FilePath.java
            core/src/test/java/hudson/FilePathTest.java
            http://jenkins-ci.org/commit/jenkins/0c16c8576bf662b2af049d8d9720ea7d30085c3a
            Log:
            JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects (#2331)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Vincent Latombe Path: core/src/main/java/hudson/FilePath.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/0c16c8576bf662b2af049d8d9720ea7d30085c3a Log: JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects (#2331)
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Released as Jenkins-2.4

            Show
            oleg_nenashev Oleg Nenashev added a comment - Released as Jenkins-2.4
            Hide
            cmelo Carlos Melo added a comment -

            Oleg Nenashev is this issue back?

            I just got the following on jenkins 2.15:

            Unpacking Go from https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 on Jenkins
            FATAL: Failed to install https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4
            java.io.IOException: Failed to install https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4
            	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:860)
            	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768)
            	at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:56)
            	at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:68)
            	at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109)
            	at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206)
            	at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:43)
            	at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34)
            	at hudson.model.Build$BuildExecution.doRun(Build.java:156)
            	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
            	at hudson.model.Run.execute(Run.java:1720)
            	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
            	at hudson.model.ResourceController.execute(ResourceController.java:98)
            	at hudson.model.Executor.run(Executor.java:404)
            Caused by: java.io.IOException: Failed to unpack https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz (152490 bytes read of total 80053651)
            	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:854)
            	... 13 more
            Caused by: java.io.IOException: Failed to extract input stream
            	at hudson.FilePath.readFromTar(FilePath.java:2328)
            	at hudson.FilePath.access$400(FilePath.java:195)
            	at hudson.FilePath$10.invoke(FilePath.java:730)
            	at hudson.FilePath$10.invoke(FilePath.java:728)
            	at hudson.FilePath.act(FilePath.java:1018)
            	at hudson.FilePath.act(FilePath.java:996)
            	at hudson.FilePath.untarFrom(FilePath.java:728)
            	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:852)
            	... 13 more
            Caused by: java.io.IOException: Truncated TAR archive
            	at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:618)
            	at java.io.InputStream.read(InputStream.java:101)
            	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
            	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
            	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
            	at hudson.util.IOUtils.copy(IOUtils.java:40)
            	at hudson.FilePath.readFromTar(FilePath.java:2318)
            	... 20 more
            
            Show
            cmelo Carlos Melo added a comment - Oleg Nenashev is this issue back? I just got the following on jenkins 2.15: Unpacking Go from https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 on Jenkins FATAL: Failed to install https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 java.io.IOException: Failed to install https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 at hudson.FilePath.installIfNecessaryFrom(FilePath.java:860) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768) at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:56) at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:68) at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109) at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206) at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:43) at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34) at hudson.model.Build$BuildExecution.doRun(Build.java:156) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534) at hudson.model.Run.execute(Run.java:1720) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) Caused by: java.io.IOException: Failed to unpack https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz (152490 bytes read of total 80053651) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:854) ... 13 more Caused by: java.io.IOException: Failed to extract input stream at hudson.FilePath.readFromTar(FilePath.java:2328) at hudson.FilePath.access$400(FilePath.java:195) at hudson.FilePath$10.invoke(FilePath.java:730) at hudson.FilePath$10.invoke(FilePath.java:728) at hudson.FilePath.act(FilePath.java:1018) at hudson.FilePath.act(FilePath.java:996) at hudson.FilePath.untarFrom(FilePath.java:728) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:852) ... 13 more Caused by: java.io.IOException: Truncated TAR archive at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:618) at java.io.InputStream.read(InputStream.java:101) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at hudson.util.IOUtils.copy(IOUtils.java:40) at hudson.FilePath.readFromTar(FilePath.java:2318) ... 20 more
            Hide
            vlatombe Vincent Latombe added a comment -

            Carlos Melo

            Looks to me the url is resolved properly. I get an HTTP 200 from curl -I https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz

            So whatever is your problem, it's probably unrelated to this issue.

            Show
            vlatombe Vincent Latombe added a comment - Carlos Melo Looks to me the url is resolved properly. I get an HTTP 200 from curl -I https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz So whatever is your problem, it's probably unrelated to this issue.

              People

              • Assignee:
                vlatombe Vincent Latombe
                Reporter:
                orrc Christopher Orr
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: