Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54603

Memory leak in remoting causes Jenkins to crash

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Major Major
    • remoting
    • None

      Some of our jobs rely on an external slave. This has been working for a while without any issues. Recently, the number of jobs that run daily has been increased. This is when our problems started. After a while Jenkins has consumed all the memory available to the VM, and it locks up as a result.

      The log is full of this exception:

      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: SEVERE: This command is created here
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: Nov 12, 2018 5:03:53 PM hudson.remoting.Channel$1 handle
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: SEVERE: Failed to execute command Pipe.Flush(-1) (channel PLTSTSRV001)
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: java.util.concurrent.ExecutionException: Invalid object ID -1 iota=1723
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:478)
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.ExportTable.get(ExportTable.java:397)
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.Channel.getExportedObject(Channel.java:780)
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.ProxyOutputStream$Flush.execute(ProxyOutputStream.java:307)
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.Channel$1.handle(Channel.java:565)
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:85)
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: Caused by: java.lang.Exception: Object appears to be deallocated at lease before Mon Nov 12 16:39:51 CET 2018
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:474)
      Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: ... 5 more

      I think this is what's causing Jenkins to leak.

        1. heap-histogram.txt
          15 kB
        2. Jenkins Job Output
          27 kB
        3. Pipeline.txt
          7 kB

            smokeythebandit Benjamin Martens
            smokeythebandit Benjamin Martens
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: