Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10424

PerJobCredentialStore kept in ExportTable (potential OOM)

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: core, subversion-plugin
    • Labels:
      None

      Description

      PerJobCredentialStore is kept in ExportTable and is never released which can leads to OOM, for details see [1] and also [2].
      [1]http://jenkins.361315.n4.nabble.com/jenkins-running-out-of-memory-heapspace-I-have-a-memory-leak-td3658830.html
      [2]http://jenkins.361315.n4.nabble.com/Severe-memory-leak-in-hudson-td3162254.html

        Activity

        Hide
        vjuranek vjuranek added a comment -

        IMHO it's cause by PerJobCredentialStore.writeReplace() which increases reference count, but later on it's not decreased and there it's kept in ExportTable all the time (but still not compltely about it).

        Show
        vjuranek vjuranek added a comment - IMHO it's cause by PerJobCredentialStore.writeReplace() which increases reference count, but later on it's not decreased and there it's kept in ExportTable all the time (but still not compltely about it).
        Hide
        kohsuke Kohsuke Kawaguchi added a comment -

        Fix is locally confirmed in tests and currently field tested. If anyone is interested in helping to verify the fix, please let me know.
        Currently the fix is targeted in 1.424.

        Show
        kohsuke Kohsuke Kawaguchi added a comment - Fix is locally confirmed in tests and currently field tested. If anyone is interested in helping to verify the fix, please let me know. Currently the fix is targeted in 1.424.
        Hide
        vjuranek vjuranek added a comment -

        I'm happy to try to verify the fix. Is the patch available somewhere?

        Show
        vjuranek vjuranek added a comment - I'm happy to try to verify the fix. Is the patch available somewhere?
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        changelog.html
        pom.xml
        http://jenkins-ci.org/commit/jenkins/c797e28b7b2df90cec240c159200077880903271
        Log:
        [FIXED JENKINS-10424] integrating a new version of remoting.jar that
        fixes reference count leaks.

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html pom.xml http://jenkins-ci.org/commit/jenkins/c797e28b7b2df90cec240c159200077880903271 Log: [FIXED JENKINS-10424] integrating a new version of remoting.jar that fixes reference count leaks.
        Hide
        kohsuke Kohsuke Kawaguchi added a comment -

        The fix is in remoting.jar 2.2. So interested people please build Jenkins with this remoting jar and let us know how it goes.

        I can build 1.422+this fix very easily, but I don't have a good place to put it, as jenkins.war is bulky.

        Show
        kohsuke Kohsuke Kawaguchi added a comment - The fix is in remoting.jar 2.2. So interested people please build Jenkins with this remoting jar and let us know how it goes. I can build 1.422+this fix very easily, but I don't have a good place to put it, as jenkins.war is bulky.
        Hide
        dogfood dogfood added a comment -

        Integrated in jenkins_main_trunk #984
        [FIXED JENKINS-10424] integrating a new version of remoting.jar that

        Kohsuke Kawaguchi : c797e28b7b2df90cec240c159200077880903271
        Files :

        • pom.xml
        • changelog.html
        Show
        dogfood dogfood added a comment - Integrated in jenkins_main_trunk #984 [FIXED JENKINS-10424] integrating a new version of remoting.jar that Kohsuke Kawaguchi : c797e28b7b2df90cec240c159200077880903271 Files : pom.xml changelog.html
        Hide
        cbos Cees Bos added a comment -

        What is the expected delivery date for this fix?
        Is there a way we can use the 1.422+ version?

        We are suffering each day from this issue, every day is a day too much.

        Show
        cbos Cees Bos added a comment - What is the expected delivery date for this fix? Is there a way we can use the 1.422+ version? We are suffering each day from this issue, every day is a day too much.
        Hide
        vjuranek vjuranek added a comment -

        I can confirm, that the issue was fixed.

        @Cees Bos: you can download snapshot build from Jenkins CI, in [1] and later builds it should be fixed

        [1] http://ci.jenkins-ci.org/job/jenkins_main_trunk/984/artifact/war/target/jenkins.war

        Show
        vjuranek vjuranek added a comment - I can confirm, that the issue was fixed. @Cees Bos: you can download snapshot build from Jenkins CI, in [1] and later builds it should be fixed [1] http://ci.jenkins-ci.org/job/jenkins_main_trunk/984/artifact/war/target/jenkins.war
        Hide
        cbos Cees Bos added a comment -

        I have upgraded our Jenkins environment to the war mentioned above.
        Heap usage is back to previous normal level (1.5 Gb instead of 3Gb before the fix)

        Jenkins is working much much better now as we were used to before. Thanks

        Show
        cbos Cees Bos added a comment - I have upgraded our Jenkins environment to the war mentioned above. Heap usage is back to previous normal level (1.5 Gb instead of 3Gb before the fix) Jenkins is working much much better now as we were used to before. Thanks
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Vojtech Juranek
        Path:
        remoting/src/main/java/hudson/remoting/Channel.java
        remoting/src/main/java/hudson/remoting/ExportTable.java
        remoting/src/test/java/hudson/remoting/ChannelTest.java
        http://jenkins-ci.org/commit/jenkins/b130a59b6b237b05e2be693e34a456ea28187765
        Log:
        Backporting fix for JENKINS-10424 from remoting project (https://github.com/jenkinsci/remoting/commit/4cd0ec976d350b22d4b9180ae9858777abd3097a)

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Vojtech Juranek Path: remoting/src/main/java/hudson/remoting/Channel.java remoting/src/main/java/hudson/remoting/ExportTable.java remoting/src/test/java/hudson/remoting/ChannelTest.java http://jenkins-ci.org/commit/jenkins/b130a59b6b237b05e2be693e34a456ea28187765 Log: Backporting fix for JENKINS-10424 from remoting project ( https://github.com/jenkinsci/remoting/commit/4cd0ec976d350b22d4b9180ae9858777abd3097a )
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Vojtech Juranek
        Path:
        changelog.html
        http://jenkins-ci.org/commit/jenkins/0f3f59c05be0311cda2cf924cccfb8b481f4e7cf
        Log:
        Added changelog item for backported fix for JENKINS-10424

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Vojtech Juranek Path: changelog.html http://jenkins-ci.org/commit/jenkins/0f3f59c05be0311cda2cf924cccfb8b481f4e7cf Log: Added changelog item for backported fix for JENKINS-10424
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        changelog.html
        pom.xml
        http://jenkins-ci.org/commit/jenkins/c797e28b7b2df90cec240c159200077880903271
        Log:
        [FIXED JENKINS-10424] integrating a new version of remoting.jar that
        fixes reference count leaks.

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html pom.xml http://jenkins-ci.org/commit/jenkins/c797e28b7b2df90cec240c159200077880903271 Log: [FIXED JENKINS-10424] integrating a new version of remoting.jar that fixes reference count leaks.
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        changelog.html
        pom.xml
        http://jenkins-ci.org/commit/jenkins/c797e28b7b2df90cec240c159200077880903271
        Log:
        [FIXED JENKINS-10424] integrating a new version of remoting.jar that
        fixes reference count leaks.

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html pom.xml http://jenkins-ci.org/commit/jenkins/c797e28b7b2df90cec240c159200077880903271 Log: [FIXED JENKINS-10424] integrating a new version of remoting.jar that fixes reference count leaks.

          People

          • Assignee:
            kohsuke Kohsuke Kawaguchi
            Reporter:
            vjuranek vjuranek
          • Votes:
            4 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: