Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3799

Slave-to-master copies can be extremely slow

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Component/s: core
    • Labels:
      None
    • Environment:
      Platform: All, OS: All
    • Similar Issues:

      Attachments

        Issue Links

          Activity

          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          I was approached by someone in the IRC channel that said he's going to take a
          look at it, and after I said thank you to him and he left, I realized that I was
          actually supposed to fix this problem for my colleagues.

          So I went ahead and take a look anyway.

          The root cause was that the TarOutputStream used in FilePath.copyRecursiveTo()
          was flushing buffers too often, and that was causing the unnecessary network
          traffic. I also removed the recording of where Command objects are created for
          ProxyOutputStream.Chunk to cut the overhead.

          This fix will be in 1.311.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - I was approached by someone in the IRC channel that said he's going to take a look at it, and after I said thank you to him and he left, I realized that I was actually supposed to fix this problem for my colleagues. So I went ahead and take a look anyway. The root cause was that the TarOutputStream used in FilePath.copyRecursiveTo() was flushing buffers too often, and that was causing the unnecessary network traffic. I also removed the recording of where Command objects are created for ProxyOutputStream.Chunk to cut the overhead. This fix will be in 1.311.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/core/src/main/java/hudson/FilePath.java
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/Command.java
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java
          trunk/www/changelog.html
          http://fisheye4.cenqua.com/changelog/hudson/?cs=18816
          Log:
          [FIXED JENKINS-3799] Fix for performance problem in FilePath.copyRecursiveTo() over the network.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/main/java/hudson/FilePath.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/Command.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java trunk/www/changelog.html http://fisheye4.cenqua.com/changelog/hudson/?cs=18816 Log: [FIXED JENKINS-3799] Fix for performance problem in FilePath.copyRecursiveTo() over the network.
          Hide
          abayer Andrew Bayer added a comment -

          I'll try this out once 1.311 comes out, but I'm not 100% sure this will fix the
          specific problem I'm seeing - it's showing up for Maven artifacts, which are
          getting copied using FilePath.copyTo(), not FilePath.copyRecursiveTo().

          Show
          abayer Andrew Bayer added a comment - I'll try this out once 1.311 comes out, but I'm not 100% sure this will fix the specific problem I'm seeing - it's showing up for Maven artifacts, which are getting copied using FilePath.copyTo(), not FilePath.copyRecursiveTo().
          Hide
          abayer Andrew Bayer added a comment -

          This is still going on.

          Show
          abayer Andrew Bayer added a comment - This is still going on.
          Hide
          zynasis zynasis added a comment -

          Having the same issue. Using v1.366

          Show
          zynasis zynasis added a comment - Having the same issue. Using v1.366
          Hide
          asimmons Aaron Simmons added a comment -

          This problem is still occurring as of 1.403

          Show
          asimmons Aaron Simmons added a comment - This problem is still occurring as of 1.403
          Hide
          jglick Jesse Glick added a comment -

          Probably improved by JENKINS-7813 fixes.

          Show
          jglick Jesse Glick added a comment - Probably improved by JENKINS-7813 fixes.

            People

            • Assignee:
              Unassigned
              Reporter:
              abayer Andrew Bayer
            • Votes:
              13 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: