Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35160

Unable to delete project using NFS. File descriptor leak in build folder

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I tried to delete a project, but getting following error:

      http://cheesecake.factory.corp.yahoo.com:9999/jenkins/user/jennyhung/my-views/view/All/job/GeminiE2E_SearchSharedLibrarySiteLinkTestSingle-verifyAdServingNoUse/doDelete

      A problem occurred while processing the request. Please check our bug tracker to see if a similar problem has already been reported. If it is already reported, please vote and put a comment on it to let us gauge the impact of the problem. If you think this is a new issue, please file a new issue. When you file an issue, make sure to add the entire stack trace, along with the version of Jenkins and relevant plugins. The users list might be also useful in understanding what has happened.
      Stack trace

      java.nio.file.FileSystemException: /home/y/var/jenkins/jobs/GeminiE2E_SearchSharedLibrarySiteLinkTestSingle-verifyAdServingNoUse/builds/2016-05-18_03-38-23/.nfs000000000fbbd28800000082: Device or resource busy
      	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
      	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
      	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
      	at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
      	at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
      	at java.nio.file.Files.delete(Files.java:1126)
      	at sun.reflect.GeneratedMethodAccessor3466.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at hudson.Util.deleteFile(Util.java:247)
      	at hudson.Util.deleteRecursive(Util.java:310)
      	at hudson.Util.deleteContentsRecursive(Util.java:212)
      	at hudson.Util.deleteRecursive(Util.java:301)
      	at hudson.Util.deleteContentsRecursive(Util.java:212)
      	at hudson.Util.deleteRecursive(Util.java:301)
      	at hudson.Util.deleteContentsRecursive(Util.java:212)
      	at hudson.Util.deleteRecursive(Util.java:301)
      	at hudson.model.AbstractItem.performDelete(AbstractItem.java:596)
      	at hudson.model.Job.performDelete(Job.java:259)
      	at hudson.model.AbstractProject.performDelete(AbstractProject.java:372)
      	at hudson.model.AbstractItem.delete(AbstractItem.java:585)
      	at hudson.model.Job.delete(Job.java:648)
      	at hudson.model.AbstractItem.doDoDelete(AbstractItem.java:546)
      	at sun.reflect.GeneratedMethodAccessor3013.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298)
      	at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:46)
      	at org.kohsuke.stapler.Function$InterceptedFunction.invoke(Function.java:399)
      	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161)
      	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96)
      	at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:121)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:249)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:249)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:391)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:249)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
      	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
      	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:96)
      	at com.cloudbees.jenkins.support.slowrequest.SlowRequestFilter.doFilter(SlowRequestFilter.java:37)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99)
      	at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:117)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99)
      	at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:95)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99)
      	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:102)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:85)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at hudson.security.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:114)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
      	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:168)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at yjava.servlet.filter.BouncerFilter.doFilter(BouncerFilter.java:217)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at yjava.servlet.filter.SSLCrimeFilter.doFilter(SSLCrimeFilter.java:30)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at yjava.servlet.filter.YHdrsFilter.doFilter(YHdrsFilter.java:73)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at yjava.cookie.CookieDataFilter.doFilter(CookieDataFilter.java:152)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at yjava.servlet.filter.DoNotTrackFilter.doFilter(DoNotTrackFilter.java:114)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at yjava.remote.ip.RemoteIPFilter.doFilter(RemoteIPFilter.java:126)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at yjava.security.yiv.servlet.InputValidationFilter.doFilter(InputValidationFilter.java:228)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at yjava.servlet.filter.StatsFilter.doFilter(StatsFilter.java:73)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
      	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      	at org.eclipse.jetty.webapp.logging.ContextLogHandler.handle(ContextLogHandler.java:62)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      	at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:159)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:309)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      	at org.eclipse.jetty.server.Server.handle(Server.java:497)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
      	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
      	at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Issue Links

            Activity

            Hide
            teilo James Nord added a comment -

            Jenkins needs to be smarter about (not) allowing you to delete a project if it is building or scheduled.
            It should either abort the build and then delete, or prevent the deletion to begin with (and if a job is in the queue for the job, it should be cancelled - otherwise you may have an exception later on in the queue).

            Show
            teilo James Nord added a comment - Jenkins needs to be smarter about (not) allowing you to delete a project if it is building or scheduled. It should either abort the build and then delete, or prevent the deletion to begin with (and if a job is in the queue for the job, it should be cancelled - otherwise you may have an exception later on in the queue).
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Stephen Connolly
            Path:
            src/main/java/com/cloudbees/hudson/plugins/folder/AbstractFolder.java
            src/main/java/com/cloudbees/hudson/plugins/folder/ItemDeletion.java
            src/main/java/com/cloudbees/hudson/plugins/folder/computed/ComputedFolder.java
            src/main/resources/com/cloudbees/hudson/plugins/folder/Messages.properties
            http://jenkins-ci.org/commit/cloudbees-folder-plugin/7d0c7811196d44726fb1b91d96ecd39ed357de19
            Log:
            JENKINS-35112 Re-implement based on JENKINS-35160 impl in core

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: src/main/java/com/cloudbees/hudson/plugins/folder/AbstractFolder.java src/main/java/com/cloudbees/hudson/plugins/folder/ItemDeletion.java src/main/java/com/cloudbees/hudson/plugins/folder/computed/ComputedFolder.java src/main/resources/com/cloudbees/hudson/plugins/folder/Messages.properties http://jenkins-ci.org/commit/cloudbees-folder-plugin/7d0c7811196d44726fb1b91d96ecd39ed357de19 Log: JENKINS-35112 Re-implement based on JENKINS-35160 impl in core
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Stephen Connolly
            Path:
            src/main/java/com/cloudbees/hudson/plugins/folder/AbstractFolder.java
            src/main/java/com/cloudbees/hudson/plugins/folder/ItemDeletion.java
            src/main/java/com/cloudbees/hudson/plugins/folder/computed/ComputedFolder.java
            src/main/resources/com/cloudbees/hudson/plugins/folder/Messages.properties
            http://jenkins-ci.org/commit/cloudbees-folder-plugin/37eeaf496b6e30450bc59158a0ae91bb37b9074b
            Log:
            Merge pull request #95 from stephenc/jenkins-35112-redux

            JENKINS-35112 Re-implement based on JENKINS-35160 impl in core

            Compare: https://github.com/jenkinsci/cloudbees-folder-plugin/compare/d43aa4d30bd6...37eeaf496b6e

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: src/main/java/com/cloudbees/hudson/plugins/folder/AbstractFolder.java src/main/java/com/cloudbees/hudson/plugins/folder/ItemDeletion.java src/main/java/com/cloudbees/hudson/plugins/folder/computed/ComputedFolder.java src/main/resources/com/cloudbees/hudson/plugins/folder/Messages.properties http://jenkins-ci.org/commit/cloudbees-folder-plugin/37eeaf496b6e30450bc59158a0ae91bb37b9074b Log: Merge pull request #95 from stephenc/jenkins-35112-redux JENKINS-35112 Re-implement based on JENKINS-35160 impl in core Compare: https://github.com/jenkinsci/cloudbees-folder-plugin/compare/d43aa4d30bd6...37eeaf496b6e
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Stephen Connolly
            Path:
            core/src/main/java/hudson/model/AbstractItem.java
            core/src/main/java/hudson/model/Computer.java
            core/src/main/java/hudson/model/Executor.java
            core/src/main/java/hudson/model/Job.java
            core/src/main/java/hudson/model/RestartListener.java
            core/src/main/java/hudson/model/queue/Tasks.java
            core/src/main/java/jenkins/model/queue/ItemDeletion.java
            core/src/main/resources/hudson/model/Messages.properties
            test/src/test/java/hudson/model/JobTest.java
            http://jenkins-ci.org/commit/jenkins/52a1a1035fa71a2e023e93c083a626898bcb0d47
            Log:
            [FIXED JENKINS-35160] - Job deletion: Wait up to 15 seconds for interrupted builds to complete (#2789)

            • [FIXED JENKINS-35160] Wait up to 15 seconds for interrupted builds to complete
            • Also now aware of concurrent builds
            • JENKINS-35160 We should do the interrupt for any Item not just Jobs

            Left over references before I settled on a better name

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: core/src/main/java/hudson/model/AbstractItem.java core/src/main/java/hudson/model/Computer.java core/src/main/java/hudson/model/Executor.java core/src/main/java/hudson/model/Job.java core/src/main/java/hudson/model/RestartListener.java core/src/main/java/hudson/model/queue/Tasks.java core/src/main/java/jenkins/model/queue/ItemDeletion.java core/src/main/resources/hudson/model/Messages.properties test/src/test/java/hudson/model/JobTest.java http://jenkins-ci.org/commit/jenkins/52a1a1035fa71a2e023e93c083a626898bcb0d47 Log: [FIXED JENKINS-35160] - Job deletion: Wait up to 15 seconds for interrupted builds to complete (#2789) [FIXED JENKINS-35160] Wait up to 15 seconds for interrupted builds to complete Also now aware of concurrent builds JENKINS-35160 Tests are good, they catch bugs JENKINS-35160 We should do the interrupt for any Item not just Jobs JENKINS-35160 s/DeleteBlocker/ItemDeletion/g Left over references before I settled on a better name JENKINS-35160 Switch to Failure for better HTML rendering JENKINS-35160 Align the i18n key with owning class
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            The fix has been integrated towards 2.55

            Show
            oleg_nenashev Oleg Nenashev added a comment - The fix has been integrated towards 2.55

              People

              • Assignee:
                stephenconnolly Stephen Connolly
                Reporter:
                jennyhung Jenny Hung
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: