Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34946

P4 Plugin runs out of memory reverting files

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • p4-plugin
    • Windows 7 64-bit
      JRE 1.8.0_91
      Jenkins version 1.656, running as Windows service
      Initially had the default -Xmx256, raised to -Xmx1024 but didn't help
      P4 Plugin version 1.3.9
      Web browser is Chome

      An out of memory exception was triggered when P4 Plugin tried to revert some files. It looks like it actually tries to load the files into memory while reverting.

      Our depot has some very large files, so the Populate options are as follows:
      Sync only
      Populate have list
      Revert any open or unshelved files

      Usually the workspace is left clean without any files left checked out, but on one occasion there were a couple of files totalling about 1GB in size checked out in the master node's workspace when it tried to start a build. It got as far as trying to revert the checked out files and then threw an out of memory exception.

      From that point on, it retried every 2 minutes (which is our SCM poll time) and every build on the master node failed with the same issue (all our builds on a given node share the same P4 workspace). I took a heap dump when Jenkins was idle to see if there were any leaks, but its memory usage was only 30MB.

      I then triggered a build and took another heap after a couple of seconds to try to catch the memory spike, and that time I found a single instance of byte[], 113MB in size, was in the heap (the contents of the array was all zeroes), but not attached to the GC root. One of the files it was attempting to revert was about that size, and there was another file of about 800MB that it also needed to revert, so I suspect it successfully allocated space for the 113MB file and then threw an exception trying to allocate memory for the larger file.

      If my analysis is correct, this bug will occur whenever P4 Plugin has to revert a file larger than the Jenkins heap, so to repro the bug I suggest creating a very large file, checking it out and then having P4 Plugin try to revert it. If that doesn't trigger the bug, I should be able to provide more info to help you track it down.

      I've attached a build log from one of the failures showing the out of memory message (which is actually a ConnectionException).

      By the way, if it makes any difference, the files it was trying to revert needed resolving, as they had been modified in another workspace since being checked out.

            Unassigned Unassigned
            henryfalconer Henry Falconer
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: