Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6516

Heavy memory consumption for test results, esp. stdout/stderr

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Component/s: junit-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      Looking through a heap dump from our server which crashed with an OOME recently, I noticed a number of large String's. The second largest, at around 300k characters, is the standard error from a test suite (one of thousands) in one build (maybe 20 builds old) in one job.

      Why should this be loaded in memory? In the unlikely event that someone wants to look through details of these old test results, they can be loaded from disk. The junitResult.xml from this build is 3.1 Mb, so it is ridiculous to keep it in heap. Most of that is stderr.

      Alternately, discard stdout/stderr for test suites that passed, since it is unlikely you would want to look at output in that case anyway.

        Attachments

          Issue Links

            Activity

            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in hudson
            User: : jglick
            Path:
            trunk/hudson/main/core/src/main/java/hudson/tasks/junit/CaseResult.java
            trunk/hudson/main/core/src/main/java/hudson/tasks/junit/SuiteResult.java
            trunk/hudson/main/core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
            http://jenkins-ci.org/commit/30978
            Log:
            [FIXED JENKINS-6516] Trim stdout/stderr of passing tests to around 1Kb. Otherwise Hudson easily gets OOME.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : jglick Path: trunk/hudson/main/core/src/main/java/hudson/tasks/junit/CaseResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/SuiteResult.java trunk/hudson/main/core/src/test/java/hudson/tasks/junit/SuiteResultTest.java http://jenkins-ci.org/commit/30978 Log: [FIXED JENKINS-6516] Trim stdout/stderr of passing tests to around 1Kb. Otherwise Hudson easily gets OOME.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in hudson
            User: : jglick
            Path:
            trunk/www/changelog.html
            http://jenkins-ci.org/commit/30979
            Log:
            JENKINS-6516 Noting.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : jglick Path: trunk/www/changelog.html http://jenkins-ci.org/commit/30979 Log: JENKINS-6516 Noting.
            Hide
            huybrechts huybrechts added a comment -

            It's not right to just remove test output. I still want to see mine (even when it's large), e.g. to compare the output of bad runs to good runs.

            Please make this optional.

            Show
            huybrechts huybrechts added a comment - It's not right to just remove test output. I still want to see mine (even when it's large), e.g. to compare the output of bad runs to good runs. Please make this optional.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in hudson
            User: : jglick
            Path:
            trunk/hudson/main/core/src/main/java/hudson/tasks/junit/CaseResult.java
            trunk/hudson/main/core/src/main/java/hudson/tasks/junit/JUnitParser.java
            trunk/hudson/main/core/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java
            trunk/hudson/main/core/src/main/java/hudson/tasks/junit/SuiteResult.java
            trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TestResult.java
            trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly
            trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/JUnitResultArchiver/help-keepLongStdio.html
            trunk/hudson/main/core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
            http://jenkins-ci.org/commit/31016
            Log:
            [FIXED JENKINS-6516] Make stdio trimming optional.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : jglick Path: trunk/hudson/main/core/src/main/java/hudson/tasks/junit/CaseResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/JUnitParser.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/SuiteResult.java trunk/hudson/main/core/src/main/java/hudson/tasks/junit/TestResult.java trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly trunk/hudson/main/core/src/main/resources/hudson/tasks/junit/JUnitResultArchiver/help-keepLongStdio.html trunk/hudson/main/core/src/test/java/hudson/tasks/junit/SuiteResultTest.java http://jenkins-ci.org/commit/31016 Log: [FIXED JENKINS-6516] Make stdio trimming optional.
            Hide
            ytsolar ytsolar added a comment - - edited

            can this information like stdout, stderr, errorStacktrace etc be loaded from disk when needed? why should this be loaded in memory?

            Show
            ytsolar ytsolar added a comment - - edited can this information like stdout, stderr, errorStacktrace etc be loaded from disk when needed? why should this be loaded in memory?
            Hide
            jglick Jesse Glick added a comment -

            In principle it could be kept in disk and loaded on demand, but due to the automagical use of XStream I could not see any straightforward way to do that.

            Show
            jglick Jesse Glick added a comment - In principle it could be kept in disk and loaded on demand, but due to the automagical use of XStream I could not see any straightforward way to do that.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/tasks/junit/CaseResult.java
            core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
            http://jenkins-ci.org/commit/jenkins/4872423170a59170e253f41f4041e64c6bbcf2b0
            Log:
            JENKINS-6516 Place stdio truncation message on its own line to make it more prominent.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/tasks/junit/CaseResult.java core/src/test/java/hudson/tasks/junit/SuiteResultTest.java http://jenkins-ci.org/commit/jenkins/4872423170a59170e253f41f4041e64c6bbcf2b0 Log: JENKINS-6516 Place stdio truncation message on its own line to make it more prominent.
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #2010
            JENKINS-6516 Place stdio truncation message on its own line to make it more prominent. (Revision 4872423170a59170e253f41f4041e64c6bbcf2b0)

            Result = SUCCESS
            Jesse Glick : 4872423170a59170e253f41f4041e64c6bbcf2b0
            Files :

            • core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
            • core/src/main/java/hudson/tasks/junit/CaseResult.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2010 JENKINS-6516 Place stdio truncation message on its own line to make it more prominent. (Revision 4872423170a59170e253f41f4041e64c6bbcf2b0) Result = SUCCESS Jesse Glick : 4872423170a59170e253f41f4041e64c6bbcf2b0 Files : core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/main/java/hudson/tasks/junit/CaseResult.java

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                jglick Jesse Glick
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: