Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-17330

FilePath.installIfNecessaryFrom routes download over remoting channel

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Installing a large downloaded tool on a network slave can be quite slow:

      "Executor #... for ... : executing ... #... / waiting for hudson.remoting.Channel@...:HTTP full-duplex channel ..." Id=... Group=main TIMED_WAITING on hudson.remoting.UserRequest@...
      	at java.lang.Object.wait(Native Method)
      	-  waiting on hudson.remoting.UserRequest@...
      	at hudson.remoting.Request.call(Request.java:146)
      	at hudson.remoting.Channel.call(Channel.java:672)
      	at hudson.FilePath.act(FilePath.java:831)
      	at hudson.FilePath.act(FilePath.java:824)
      	at hudson.FilePath.unzipFrom(FilePath.java:485)
      	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:718)
      	at hudson.tools.DownloadFromUrlInstaller.performInstallation(DownloadFromUrlInstaller.java:72)
      	at com.soasta.jenkins.MakeAppTouchTestableInstaller.performInstallation(MakeAppTouchTestableInstaller.java:49)
      	at com.soasta.jenkins.MakeAppTouchTestable.perform(MakeAppTouchTestable.java:94)
      	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:728)
      	at hudson.model.Build$BuildExecution.build(Build.java:199)
      	at hudson.model.Build$BuildExecution.doRun(Build.java:160)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
      	at hudson.model.Run.execute(Run.java:1505)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:237)
      

      JENKINS-7813 is partly to blame, but the main issue here is that the URL is being opened on master, a RemoteInputStream constructed, and the archive unpacked on the slave. It would be much faster to open the URL directly from the slave and use the remoting channel only for control.

      In typical cases the slave can access a tool URL. There are some rarer cases where it is really intentional that the URL is resolved from the master; it might be a file-protocol URL, the slave might have no public internet access, the download location might be limited to particular IPs, etc. So installIfNecessaryFrom should first try to open from the slave, then fall back to the current implementation.

        Attachments

          Issue Links

            Activity

            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/hudson/FilePath.java
            http://jenkins-ci.org/commit/jenkins/b67272e7261cd588e2bb51ad8759584a9b3ffa49
            Log:
            [FIXED JENKINS-17330] FilePath.installIfNecessaryFrom should avoid routing download over remoting channel.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/FilePath.java http://jenkins-ci.org/commit/jenkins/b67272e7261cd588e2bb51ad8759584a9b3ffa49 Log: [FIXED JENKINS-17330] FilePath.installIfNecessaryFrom should avoid routing download over remoting channel.
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #2416
            [FIXED JENKINS-17330] FilePath.installIfNecessaryFrom should avoid routing download over remoting channel. (Revision b67272e7261cd588e2bb51ad8759584a9b3ffa49)

            Result = SUCCESS
            Jesse Glick : b67272e7261cd588e2bb51ad8759584a9b3ffa49
            Files :

            • core/src/main/java/hudson/FilePath.java
            • changelog.html
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2416 [FIXED JENKINS-17330] FilePath.installIfNecessaryFrom should avoid routing download over remoting channel. (Revision b67272e7261cd588e2bb51ad8759584a9b3ffa49) Result = SUCCESS Jesse Glick : b67272e7261cd588e2bb51ad8759584a9b3ffa49 Files : core/src/main/java/hudson/FilePath.java changelog.html
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/hudson/FilePath.java
            http://jenkins-ci.org/commit/jenkins/3aaca054b8fb65452d5e338b4ecab0d1b26d0d4f
            Log:
            [FIXED JENKINS-17330] FilePath.installIfNecessaryFrom should avoid routing download over remoting channel.(cherry picked from commit b67272e7261cd588e2bb51ad8759584a9b3ffa49)

            Conflicts:
            changelog.html

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/FilePath.java http://jenkins-ci.org/commit/jenkins/3aaca054b8fb65452d5e338b4ecab0d1b26d0d4f Log: [FIXED JENKINS-17330] FilePath.installIfNecessaryFrom should avoid routing download over remoting channel.(cherry picked from commit b67272e7261cd588e2bb51ad8759584a9b3ffa49) Conflicts: changelog.html

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                jglick Jesse Glick
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: