Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-2142

FATAL: Java heap space in Perforce Plugin

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: perforce-plugin
    • Labels:
      None
    • Environment:
      Platform: PC, OS: Windows XP
    • Similar Issues:

      Description

      During Hudson build I receive the following error
      ------
      [workspace] $ p4 sync -f //depot/Synchro/dev/main/...#head
      FATAL: Java heap space
      java.lang.OutOfMemoryError: Java heap space
      at java.util.Arrays.copyOf(Unknown Source)
      at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
      at java.lang.AbstractStringBuilder.append(Unknown Source)
      at java.lang.StringBuilder.append(Unknown Source)
      at
      com.tek42.perforce.parse.AbstractPerforceTemplate.getPerforceResponse(AbstractPerforceTemplate.java:194)
      at com.tek42.perforce.parse.Workspaces.syncTo(Workspaces.java:90)
      at com.tek42.perforce.parse.Workspaces.syncToHead(Workspaces.java:71)
      at hudson.plugins.perforce.PerforceSCM.checkout(PerforceSCM.java:318)
      at hudson.model.AbstractProject.checkout(AbstractProject.java:596)
      at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:251)
      at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:225)
      at hudson.model.Run.run(Run.java:784)
      at hudson.model.Build.run(Build.java:85)
      at hudson.model.ResourceController.execute(ResourceController.java:70)
      at hudson.model.Executor.run(Executor.java:88)
      ------
      Hudson is currently running with CATALINA_OPTS="-DHUDSON_HOME=C:\.hudson
      -Xmx2048M -Xms2048M"

      Are there any recommend settings for the Perforce plugin?

        Attachments

          Activity

          Hide
          ssbarnea Sorin Sbarnea added a comment -

          Now the question is who is getting out of memory: the server or the slave. I think it is the slave and in this case I don't know how to increase the memory of the slave (node).

          Anyway, the p4 plugin doesn't need so much memory.

          Show
          ssbarnea Sorin Sbarnea added a comment - Now the question is who is getting out of memory: the server or the slave. I think it is the slave and in this case I don't know how to increase the memory of the slave (node). Anyway, the p4 plugin doesn't need so much memory.
          Hide
          rpetti Rob Petti added a comment -

          ssbarnea, you have 3GB of memory, but have you configured your java VM to actually use it? You need to supply the jvm flag: -Xmx3000m in order to use the full 3GB (the default is something considerably smaller.)

          The output of the p4 commands is stored and parsed on the master, not the slave, so that's where the memory is needed.

          Fixing this basically requires me to rewrite large portions of the tek42 library code, so for the time being please just sync the workspace manually the first time. I can't imagine you are editing so many files at a time that you'd run into this again after the initial sync.

          Show
          rpetti Rob Petti added a comment - ssbarnea, you have 3GB of memory, but have you configured your java VM to actually use it? You need to supply the jvm flag: -Xmx3000m in order to use the full 3GB (the default is something considerably smaller.) The output of the p4 commands is stored and parsed on the master , not the slave, so that's where the memory is needed. Fixing this basically requires me to rewrite large portions of the tek42 library code, so for the time being please just sync the workspace manually the first time. I can't imagine you are editing so many files at a time that you'd run into this again after the initial sync.
          Hide
          ssbarnea Sorin Sbarnea added a comment -

          My options were made on JAVA_OPTS but I discovered that when I installed tomcat I forgot to `tune` the starter catalina.sh to use 3GB of RAM.

          Thanks for letting me know that this was a server side issue, previously I tried to change the settings on the node config.

          I'm wondering why this still happens considering that 100.000 lines or 200 bytes would take only ~20mb or RAM.

          Aren't any easier workarounds, like not storing the p4 sync results at all?

          Show
          ssbarnea Sorin Sbarnea added a comment - My options were made on JAVA_OPTS but I discovered that when I installed tomcat I forgot to `tune` the starter catalina.sh to use 3GB of RAM. Thanks for letting me know that this was a server side issue, previously I tried to change the settings on the node config. I'm wondering why this still happens considering that 100.000 lines or 200 bytes would take only ~20mb or RAM. Aren't any easier workarounds, like not storing the p4 sync results at all?
          Hide
          rpetti Rob Petti added a comment -

          Not storing the p4 sync results is the part that would require me to rewrite large portions of the tek42 code. It's built into how the library deals with output from all perforce commands, not just the sync code. All the output is stored, then parsed, then split, then parsed again. Certainly not efficient when you have over 2 million files being synced at once like some people seem to have. We would need to rewrite it so it would use a buffered stream that's provided to all levels of abstraction simultaneously, which would also likely require multithreading. A quick hack would be to throw out all output when 'sync' is in the command line, but then you would have the unfortunate side effect of not having any feedback if something goes wrong.

          The tek42 library simply wasn't designed to handle large operations like this, but given that the official p4java doesn't work correctly on all platforms, it's the best we've got.

          It would only take ~20mb if it was pure data. The tek42 libraries also make a copy of it and split it into an array of lines. At the end of the day, it could be taking up as much as 3 or 4 times the amound of ram you would normally expect. Depending on the JVM you are using, the default max heap size value could be as low as 64mb-128mb, so that would easily fill it up in no time. We run with -Xmx2000m, and have never run into this issue.

          Getting this fixed is on my list of things to do, but I'm getting slammed at work and haven't had much time for OSS at the moment.

          Show
          rpetti Rob Petti added a comment - Not storing the p4 sync results is the part that would require me to rewrite large portions of the tek42 code. It's built into how the library deals with output from all perforce commands, not just the sync code. All the output is stored, then parsed, then split, then parsed again. Certainly not efficient when you have over 2 million files being synced at once like some people seem to have. We would need to rewrite it so it would use a buffered stream that's provided to all levels of abstraction simultaneously, which would also likely require multithreading. A quick hack would be to throw out all output when 'sync' is in the command line, but then you would have the unfortunate side effect of not having any feedback if something goes wrong. The tek42 library simply wasn't designed to handle large operations like this, but given that the official p4java doesn't work correctly on all platforms, it's the best we've got. It would only take ~20mb if it was pure data. The tek42 libraries also make a copy of it and split it into an array of lines. At the end of the day, it could be taking up as much as 3 or 4 times the amound of ram you would normally expect. Depending on the JVM you are using, the default max heap size value could be as low as 64mb-128mb, so that would easily fill it up in no time. We run with -Xmx2000m, and have never run into this issue. Getting this fixed is on my list of things to do, but I'm getting slammed at work and haven't had much time for OSS at the moment.
          Hide
          rpetti Rob Petti added a comment -

          85b8bf8 should solve this issue for most if not all users. It basically throws away all but the first 50 lines of the response from 'p4 sync'. Errors are still checked for on the fly, but "synced"/"refreshed" notifications are ignored.

          Show
          rpetti Rob Petti added a comment - 85b8bf8 should solve this issue for most if not all users. It basically throws away all but the first 50 lines of the response from 'p4 sync'. Errors are still checked for on the fly, but "synced"/"refreshed" notifications are ignored.

            People

            • Assignee:
              Unassigned
              Reporter:
              fowlesp fowlesp
            • Votes:
              3 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: