Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15382

Memory exhaustion parsing large test stdio from Surefire

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      A user with some *-output.txt files pushing 70Mb received errors when publishing test results:

      [ERROR] FATAL ERROR 
      [INFO] ------------------------------------------------------------------------ 
      [INFO] Java heap space 
      [INFO] ------------------------------------------------------------------------ 
      [INFO] Trace 
      java.lang.OutOfMemoryError: Java heap space 
      	at java.util.Arrays.copyOf(Arrays.java:2882) 
      	at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) 
      	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515) 
      	at java.lang.StringBuffer.append(StringBuffer.java:306) 
      	at java.io.StringWriter.write(StringWriter.java:77) 
      	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1129) 
      	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1104) 
      	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1050) 
      	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1075) 
      	at org.apache.commons.io.IOUtils.toString(IOUtils.java:382) 
      	at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1103) 
      	at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1120) 
      	at hudson.tasks.junit.SuiteResult.<init>(SuiteResult.java:195) 
      	at hudson.tasks.junit.SuiteResult.parseSuite(SuiteResult.java:130) 
      	at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:115) 
      	at hudson.tasks.junit.TestResult.parse(TestResult.java:227) 
      	at hudson.tasks.junit.TestResult.parse(TestResult.java:163) 
      	at hudson.maven.reporters.SurefireArchiver.postExecute(SurefireArchiver.java:141) 
      	at hudson.maven.Maven2Builder.postExecute(Maven2Builder.java:155) 
      	at hudson.maven.MavenBuilder$Adapter.postExecute(MavenBuilder.java:310) 
      	at hudson.maven.agent.PluginManagerInterceptor$1MojoIntercepterImpl.callPost(PluginManagerInterceptor.java:170) 
      	at hudson.maven.agent.PluginManagerInterceptor.executeMojo(PluginManagerInterceptor.java:183) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180) 
      	at org.apache.maven.lifecycle.LifecycleExecutorInterceptor.execute(LifecycleExecutorInterceptor.java:65) 
      	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328) 
      	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138) 
      	at org.apache.maven.cli.MavenCli.main(MavenCli.java:362) 
      [INFO] ------------------------------------------------------------------------ 
      [INFO] ...
      [INFO] Final Memory: 223M/1224M 
      [INFO] ------------------------------------------------------------------------
      

      Indeed it is silly to call FileUtils.readFileToString only to throw away the great majority of what was just read.

        Attachments

          Issue Links

            Activity

            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            changelog.html
            core/src/main/java/hudson/tasks/junit/CaseResult.java
            core/src/main/java/hudson/tasks/junit/SuiteResult.java
            core/src/main/java/hudson/tasks/junit/TestResult.java
            core/src/main/java/hudson/util/TextFile.java
            test/src/test/groovy/hudson/util/TextFileTest.groovy
            test/src/test/resources/hudson/util/ascii.txt
            http://jenkins-ci.org/commit/jenkins/a3512b9fe983cbbdb3b78597086822a61abb81ff
            Log:
            [FIXED JENKINS-15382]

            Revisiting the fix. Avoid parsing the entire text by only looking at the tail portion.

            (cherry picked from commit 14d980c4377dc703df8db76f192de19246a0266e)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/tasks/junit/CaseResult.java core/src/main/java/hudson/tasks/junit/SuiteResult.java core/src/main/java/hudson/tasks/junit/TestResult.java core/src/main/java/hudson/util/TextFile.java test/src/test/groovy/hudson/util/TextFileTest.groovy test/src/test/resources/hudson/util/ascii.txt http://jenkins-ci.org/commit/jenkins/a3512b9fe983cbbdb3b78597086822a61abb81ff Log: [FIXED JENKINS-15382] Revisiting the fix. Avoid parsing the entire text by only looking at the tail portion. (cherry picked from commit 14d980c4377dc703df8db76f192de19246a0266e)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            core/src/main/java/hudson/util/TextFile.java
            http://jenkins-ci.org/commit/jenkins/fca0ea8aa87ab18a1b398ce7bb8ba20bb8bde186
            Log:
            JENKINS-15382 Forgot to close the file

            (cherry picked from commit 43ecda6d2d4aba44144d12fb89ad1ed3e77bc7f3)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/util/TextFile.java http://jenkins-ci.org/commit/jenkins/fca0ea8aa87ab18a1b398ce7bb8ba20bb8bde186 Log: JENKINS-15382 Forgot to close the file (cherry picked from commit 43ecda6d2d4aba44144d12fb89ad1ed3e77bc7f3)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            core/src/main/java/hudson/tasks/junit/CaseResult.java
            http://jenkins-ci.org/commit/jenkins/bcf9a84f18e42d41f268bd62e6e62d56d2f9b23b
            Log:
            JENKINS-15382 not sure which code is triggering animal sniffer violation

            (cherry picked from commit f7671e621e374bc51a81ec75c056d15ee359093a)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/tasks/junit/CaseResult.java http://jenkins-ci.org/commit/jenkins/bcf9a84f18e42d41f268bd62e6e62d56d2f9b23b Log: JENKINS-15382 not sure which code is triggering animal sniffer violation (cherry picked from commit f7671e621e374bc51a81ec75c056d15ee359093a)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            core/src/main/java/hudson/util/TextFile.java
            http://jenkins-ci.org/commit/jenkins/36ef7c646734582b18178f83b6bb2c13e598b610
            Log:
            JENKINS-15382 removed Java 1.6 dependency

            (cherry picked from commit 82404407f2c26b3b0509e392ec35dee8ea7dc571)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/util/TextFile.java http://jenkins-ci.org/commit/jenkins/36ef7c646734582b18178f83b6bb2c13e598b610 Log: JENKINS-15382 removed Java 1.6 dependency (cherry picked from commit 82404407f2c26b3b0509e392ec35dee8ea7dc571)
            Hide
            rajesh55_5 rajesh r added a comment -

            Still getting the issue while running maven project in jenkins :
            testcase name="testrunners.CukeTestOne" classname="org.apache.maven.surefire.junit4.JUnit4Provider" time="0.767">
            <error message="Java heap space" type="java.lang.OutOfMemoryError">java.lang.OutOfMemoryError: Java heap space
            at java.util.Arrays.copyOf(Arrays.java:2367)
            at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
            at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
            at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
            at java.lang.StringBuffer.append(StringBuffer.java:237)
            at java.io.StringWriter.write(StringWriter.java:112)
            at gherkin.deps.com.google.gson.stream.JsonWriter.string(JsonWriter.java:538)
            at gherkin.deps.com.google.gson.stream.JsonWriter.value(JsonWriter.java:404)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapters$13.write(TypeAdapters.java:353)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapters$13.write(TypeAdapters.java:337)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
            at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:209)
            at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:146)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
            at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96)
            at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
            at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:209)
            at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:146)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
            at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96)
            at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
            at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:209)
            at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:146)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
            at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96)
            at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
            at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:209)
            at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:146)
            at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
            </error>

            Show
            rajesh55_5 rajesh r added a comment - Still getting the issue while running maven project in jenkins : testcase name="testrunners.CukeTestOne" classname="org.apache.maven.surefire.junit4.JUnit4Provider" time="0.767"> <error message="Java heap space" type="java.lang.OutOfMemoryError">java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2367) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415) at java.lang.StringBuffer.append(StringBuffer.java:237) at java.io.StringWriter.write(StringWriter.java:112) at gherkin.deps.com.google.gson.stream.JsonWriter.string(JsonWriter.java:538) at gherkin.deps.com.google.gson.stream.JsonWriter.value(JsonWriter.java:404) at gherkin.deps.com.google.gson.internal.bind.TypeAdapters$13.write(TypeAdapters.java:353) at gherkin.deps.com.google.gson.internal.bind.TypeAdapters$13.write(TypeAdapters.java:337) at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:209) at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:146) at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96) at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60) at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:209) at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:146) at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96) at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60) at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:209) at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:146) at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96) at gherkin.deps.com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60) at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:209) at gherkin.deps.com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:146) at gherkin.deps.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) </error>

              People

              • Assignee:
                Unassigned
                Reporter:
                jglick Jesse Glick
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: