Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19507

Jenkins runs out of threads when using SVN post commit hook with svn+ssh URL

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • subversion-plugin
    • Linux Red Hat Enterprise Linux Server release 6.4 (Santiago), Jenkins V 1.509.2 Installed via provided rpm, Subversion Plugin V 1.5, Subversion 1.6.9

      We have a running a Jenkins-Instance with about 900 build jobs. Most of them are configured with SCM Trigger (Subversion). Due to performance issues with the SCM we employ the svn+ssh protocol for the coummincation with the SCM. In order to reduce load on Subversion we try to trigger our CI jenkins jobs with the post commit hook. When we trigger the Jenkins-URL in order to notify jenkins on SCM change we get the following exception in the log file and the jenkins server is unaccessible until jenkins server restart:

      Sep 05, 2013 9:10:30 AM winstone.Logger logInternal
      SEVERE: Error while serving http://jenkins-qa.fiducia.de/jenkins/subversion/9f1f1913-8d78-0410-adcc-cdd7385d28bb/notifyCommit
      java.lang.reflect.InvocationTargetException
      at sun.reflect.GeneratedMethodAccessor690.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:601)
      at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:288)
      at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:151)
      at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:90)
      at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:111)
      at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:384)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:384)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:488)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:162)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
      at winstone.ServletConfiguration.execute(ServletConfiguration.java:248)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:376)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
      at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
      at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:54)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:331)
      at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:215)
      at winstone.RequestHandlerThread.run(RequestHandlerThread.java:138)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:722)
      Caused by: java.lang.OutOfMemoryError: unable to create new native thread
      at java.lang.Thread.start0(Native Method)
      at java.lang.Thread.start(Thread.java:691)
      at com.trilead.ssh2.transport.TransportManager.initialize(TransportManager.java:516)
      at com.trilead.ssh2.Connection.connect(Connection.java:733)
      at org.tmatesoft.svn.core.internal.io.svn.ssh.SshHost.openConnection(SshHost.java:217)
      at org.tmatesoft.svn.core.internal.io.svn.ssh.SshHost.openSession(SshHost.java:145)
      at org.tmatesoft.svn.core.internal.io.svn.ssh.SshSessionPool.openSession(SshSessionPool.java:81)
      at org.tmatesoft.svn.core.internal.io.svn.SVNSSHConnector.open(SVNSSHConnector.java:129)
      at org.tmatesoft.svn.core.internal.io.svn.SVNConnection.open(SVNConnection.java:77)
      at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.openConnection(SVNRepositoryImpl.java:1252)
      at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.testConnection(SVNRepositoryImpl.java:95)
      at hudson.scm.SubversionSCM$ModuleLocation.getUUID(SubversionSCM.java:2363)
      at hudson.scm.SubversionRepositoryStatus.doNotifyCommit(SubversionRepositoryStatus.java:138)
      ... 76 more

      Sep 05, 2013 9:10:30 AM org.kohsuke.stapler.compression.CompressionFilter reportException
      WARNING: Untrapped servlet exception
      javax.servlet.ServletException: java.lang.OutOfMemoryError: unable to create new native thread
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:616)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:384)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:384)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:488)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:162)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
      at winstone.ServletConfiguration.execute(ServletConfiguration.java:248)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:376)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
      at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
      at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:54)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:331)
      at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:215)
      at winstone.RequestHandlerThread.run(RequestHandlerThread.java:138)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:722)
      Caused by: java.lang.OutOfMemoryError: unable to create new native thread
      at java.lang.Thread.start0(Native Method)
      at java.lang.Thread.start(Thread.java:691)
      at com.trilead.ssh2.transport.TransportManager.initialize(TransportManager.java:516)
      at com.trilead.ssh2.Connection.connect(Connection.java:733)
      at org.tmatesoft.svn.core.internal.io.svn.ssh.SshHost.openConnection(SshHost.java:217)
      at org.tmatesoft.svn.core.internal.io.svn.ssh.SshHost.openSession(SshHost.java:145)
      at org.tmatesoft.svn.core.internal.io.svn.ssh.SshSessionPool.openSession(SshSessionPool.java:81)
      at org.tmatesoft.svn.core.internal.io.svn.SVNSSHConnector.open(SVNSSHConnector.java:129)
      at org.tmatesoft.svn.core.internal.io.svn.SVNConnection.open(SVNConnection.java:77)
      at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.openConnection(SVNRepositoryImpl.java:1252)
      at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.testConnection(SVNRepositoryImpl.java:95)
      at hudson.scm.SubversionSCM$ModuleLocation.getUUID(SubversionSCM.java:2363)
      at hudson.scm.SubversionRepositoryStatus.doNotifyCommit(SubversionRepositoryStatus.java:138)
      at sun.reflect.GeneratedMethodAccessor690.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:601)
      at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:288)
      at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:151)
      at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:90)
      at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:111)
      at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
      ... 67 more

            Unassigned Unassigned
            vbuehner Valentin Buehner
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: