Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3799

Slave-to-master copies can be extremely slow

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Component/s: core
    • Labels:
      None
    • Environment:
      Platform: All, OS: All

      Issue Links

        Activity

        Hide
        kohsuke Kohsuke Kawaguchi added a comment -

        I was approached by someone in the IRC channel that said he's going to take a
        look at it, and after I said thank you to him and he left, I realized that I was
        actually supposed to fix this problem for my colleagues.

        So I went ahead and take a look anyway.

        The root cause was that the TarOutputStream used in FilePath.copyRecursiveTo()
        was flushing buffers too often, and that was causing the unnecessary network
        traffic. I also removed the recording of where Command objects are created for
        ProxyOutputStream.Chunk to cut the overhead.

        This fix will be in 1.311.

        Show
        kohsuke Kohsuke Kawaguchi added a comment - I was approached by someone in the IRC channel that said he's going to take a look at it, and after I said thank you to him and he left, I realized that I was actually supposed to fix this problem for my colleagues. So I went ahead and take a look anyway. The root cause was that the TarOutputStream used in FilePath.copyRecursiveTo() was flushing buffers too often, and that was causing the unnecessary network traffic. I also removed the recording of where Command objects are created for ProxyOutputStream.Chunk to cut the overhead. This fix will be in 1.311.
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in hudson
        User: : kohsuke
        Path:
        trunk/hudson/main/core/src/main/java/hudson/FilePath.java
        trunk/hudson/main/remoting/src/main/java/hudson/remoting/Command.java
        trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java
        trunk/www/changelog.html
        http://fisheye4.cenqua.com/changelog/hudson/?cs=18816
        Log:
        [FIXED JENKINS-3799] Fix for performance problem in FilePath.copyRecursiveTo() over the network.

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/main/java/hudson/FilePath.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/Command.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java trunk/www/changelog.html http://fisheye4.cenqua.com/changelog/hudson/?cs=18816 Log: [FIXED JENKINS-3799] Fix for performance problem in FilePath.copyRecursiveTo() over the network.
        Hide
        abayer abayer added a comment -

        I'll try this out once 1.311 comes out, but I'm not 100% sure this will fix the
        specific problem I'm seeing - it's showing up for Maven artifacts, which are
        getting copied using FilePath.copyTo(), not FilePath.copyRecursiveTo().

        Show
        abayer abayer added a comment - I'll try this out once 1.311 comes out, but I'm not 100% sure this will fix the specific problem I'm seeing - it's showing up for Maven artifacts, which are getting copied using FilePath.copyTo(), not FilePath.copyRecursiveTo().
        Hide
        abayer abayer added a comment -

        This is still going on.

        Show
        abayer abayer added a comment - This is still going on.
        Hide
        zynasis zynasis added a comment -

        Having the same issue. Using v1.366

        Show
        zynasis zynasis added a comment - Having the same issue. Using v1.366
        Hide
        asimmons Aaron Simmons added a comment -

        This problem is still occurring as of 1.403

        Show
        asimmons Aaron Simmons added a comment - This problem is still occurring as of 1.403
        Hide
        jglick Jesse Glick added a comment -

        Probably improved by JENKINS-7813 fixes.

        Show
        jglick Jesse Glick added a comment - Probably improved by JENKINS-7813 fixes.

          People

          • Assignee:
            Unassigned
            Reporter:
            abayer abayer
          • Votes:
            13 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: