Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10520

clone-workspace-scm performance is poor

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • Solaris/x86, cluster of 5 nodes, all machines in cluster are identical

      Clone workspace performance is poor. Whilst compilation takes around 4 minutes, archiving the workspace afterwards takes the total job time out to 35 minutes! Similarly un-archiving it at the start of downstream jobs.

      We have 4 steps, Compile, quick test, package, full test.
      Compile uses Perforce SCM, we use clone workspace after that to ensure downstream builds use identical files but other compiles can continue in parallel. So we need almost the files in the entire workspace.
      The filesystem workspace size is about 1.6Gb
      The archived workspace.zip file size is about 1.4Gb

      An "exclude" filter may help a little but I think there is something slow going on.
      Note that we use slaves as well so the piped data may have an impact, but all the machines are very close together.
      When the compile runs on the master node it doesn't seem any quicker.

            Unassigned Unassigned
            edrandall Ed Randall
            Votes:
            13 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated: