Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-12308

Successful Jenkins job will not terminate requiring a restart of the system

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Labels:
    • Environment:
      Running Master-Slave configuration on Windows 2003 server with Jenkins Master & Slaves running as windows services

      Description

      When someone which the Jenkins system has trouble authenticating against AD (see issue: https://issues.jenkins-ci.org/browse/JENKINS-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel) checks in code that triggers a build, the build hangs, even after a successful completion. This locks any further builds by that project or any builds that are locked out by that project running from building until Jenkins is restarted.

      The log contains the following information which shows the job completed successfully then later throws the LDAP exception.

      INFO: SCM changes detected in ClaimApplicationLog. Triggering #95
      Jan 5, 2012 8:07:33 AM hudson.model.Run run
      INFO: ClaimApplicationLog #95 main build action completed: SUCCESS
      Jan 5, 2012 8:07:33 AM hudson.plugins.analysis.core.BuildResult loadResult
      INFO: Loaded data file D:\HudsonWorkingDirectory\jobs\ClaimApplicationLog\builds\2012-01-05_02-31-30\compiler-warnings.xml for build 94
      Jan 5, 2012 8:07:37 AM com.mtvi.plateng.hudson.regex.Configuration findMailAddressFor
      WARNING: RegExMailAddressResolver configuration for regex null and email pattern null is not valid.
      .....
      Jan 5, 2012 8:18:58 AM hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider retrieveUser
      WARNING: Failed to retrieve user information for XXXXXXXXX
      javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 525, vece|]; remaining name 'DC=XXXXX,DC=XXXXX,DC=net'
      at com.sun.jndi.ldap.LdapCtx.mapErrorCode(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.processReturnCode(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.processReturnCode(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.connect(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.ensureOpen(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.ensureOpen(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.doSearch(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.searchAux(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.c_search(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.c_search(Unknown Source)
      at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(Unknown Source)
      at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(Unknown Source)
      at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(Unknown Source)
      at hudson.plugins.active_directory.LDAPSearchBuilder.search(LDAPSearchBuilder.java:52)
      at hudson.plugins.active_directory.LDAPSearchBuilder.searchOne(LDAPSearchBuilder.java:42)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:140)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:105)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:64)
      at org.acegisecurity.providers.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:119)
      at org.acegisecurity.providers.ProviderManager.doAuthentication(ProviderManager.java:195)
      at org.acegisecurity.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:45)
      at org.acegisecurity.ui.webapp.AuthenticationProcessingFilter.attemptAuthentication(AuthenticationProcessingFilter.java:71)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:252)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
      at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
      at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
      at java.lang.Thread.run(Unknown Source)
      Jan 5, 2012 8:18:58 AM hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider retrieveUser
      org.acegisecurity.BadCredentialsException: Failed to retrieve user information for sgupta30; nested exception is javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 525, vece|]; remaining name 'DC=XXXXX,DC=XXXXX,DC=net'
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:179)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:105)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:64)
      at org.acegisecurity.providers.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:119)
      at org.acegisecurity.providers.ProviderManager.doAuthentication(ProviderManager.java:195)
      at org.acegisecurity.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:45)
      at org.acegisecurity.ui.webapp.AuthenticationProcessingFilter.attemptAuthentication(AuthenticationProcessingFilter.java:71)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:252)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
      at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
      at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
      at java.lang.Thread.run(Unknown Source)
      Caused by: javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 525, vece|]; remaining name 'DC=XXXXX,DC=XXXXX,DC=net'
      at com.sun.jndi.ldap.LdapCtx.mapErrorCode(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.processReturnCode(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.processReturnCode(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.connect(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.ensureOpen(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.ensureOpen(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.doSearch(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.searchAux(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.c_search(Unknown Source)
      at com.sun.jndi.ldap.LdapCtx.c_search(Unknown Source)
      at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(Unknown Source)
      at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(Unknown Source)
      at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(Unknown Source)
      at hudson.plugins.active_directory.LDAPSearchBuilder.search(LDAPSearchBuilder.java:52)
      at hudson.plugins.active_directory.LDAPSearchBuilder.searchOne(LDAPSearchBuilder.java:42)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:140)
      ... 26 more

      The job then retains a running thread on the Master and when you drill into the Console Output you see the Ajax spinning wheel showing that the system still has not completed whatever it is doing even though the job looks like it completed. The only way to clear the lock is to restart Jenkins.

      1. 1-9-2012 dump 1.txt
        141 kB
        alexlombardi
      2. 1-9-2012 dump 2.txt
        98 kB
        alexlombardi

        Activity

        Hide
        kohsuke Kohsuke Kawaguchi added a comment -

        Those log entries are caused by HTTP requests that try to authenticate themselves and likely unrelated.

        When a hang happens next time, please obtain the thread dump. There's an article on Wiki that shows you how.

        Show
        kohsuke Kohsuke Kawaguchi added a comment - Those log entries are caused by HTTP requests that try to authenticate themselves and likely unrelated. When a hang happens next time, please obtain the thread dump. There's an article on Wiki that shows you how.
        Hide
        alexlombardi alexlombardi added a comment -

        This weekend, after purging the users directory and still not clearing out the hanging issue, I uninstalled the Active Directory plug in and re-installed the 1.24 version (newest). I am now seeing error messages trying to validate all users in the matrix where permissions are configured indicating some password issue (see below), including the ones that have logged into the system already. The hang now happesn on all jobs. Thread dumps included.

        Failed to test the validity of the user name XXXXXX (show details)
        hudson.plugins.active_directory.MultiCauseBadCredentialsException: Either no such user 'XXXXXX' or incorrect password
        at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:100)
        at hudson.plugins.active_directory.AbstractActiveDirectoryAuthenticationProvider.loadUserByUsername(AbstractActiveDirectoryAuthenticationProvider.java:27)
        at hudson.plugins.active_directory.ActiveDirectorySecurityRealm.loadUserByUsername(ActiveDirectorySecurityRealm.java:533)
        at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName_(GlobalMatrixAuthorizationStrategy.java:303)
        at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName(GlobalMatrixAuthorizationStrategy.java:288)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282)
        at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149)
        at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88)
        at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:104)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648)
        at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:234)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477)
        at org.kohsuke.stapler.Stapler.service(Stapler.java:159)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
        at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
        at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
        at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:74)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
        at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
        at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:85)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
        at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
        at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
        at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
        at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
        at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
        at java.lang.Thread.run(Unknown Source)
        Cause #1: hudson.security.UserMayOrMayNotExistException: Unable to retrieve the user information without bind DN/password configured
        at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:154)
        at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:119)
        at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:84)
        at hudson.plugins.active_directory.AbstractActiveDirectoryAuthenticationProvider.loadUserByUsername(AbstractActiveDirectoryAuthenticationProvider.java:27)
        at hudson.plugins.active_directory.ActiveDirectorySecurityRealm.loadUserByUsername(ActiveDirectorySecurityRealm.java:533)
        at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName_(GlobalMatrixAuthorizationStrategy.java:303)
        at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName(GlobalMatrixAuthorizationStrategy.java:288)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282)
        at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149)
        at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88)
        at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:104)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648)
        at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:234)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477)
        at org.kohsuke.stapler.Stapler.service(Stapler.java:159)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
        at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
        at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
        at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:74)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
        at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
        at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:85)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
        at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
        at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
        at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
        at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
        at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
        at java.lang.Thread.run(Unknown Source)
        Cause #2: hudson.security.UserMayOrMayNotExistException: Unable to retrieve the user information without bind DN/password configured
        at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:154)
        at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:119)
        at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:84)
        at hudson.plugins.active_directory.AbstractActiveDirectoryAuthenticationProvider.loadUserByUsername(AbstractActiveDirectoryAuthenticationProvider.java:27)
        at hudson.plugins.active_directory.ActiveDirectorySecurityRealm.loadUserByUsername(ActiveDirectorySecurityRealm.java:533)
        at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName_(GlobalMatrixAuthorizationStrategy.java:303)
        at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName(GlobalMatrixAuthorizationStrategy.java:288)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282)
        at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149)
        at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88)
        at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:104)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648)
        at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:234)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477)
        at org.kohsuke.stapler.Stapler.service(Stapler.java:159)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
        at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
        at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
        at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:74)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
        at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
        at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:85)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
        at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
        at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
        at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
        at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
        at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
        at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
        at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
        at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
        at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
        at java.lang.Thread.run(Unknown Source)

        Show
        alexlombardi alexlombardi added a comment - This weekend, after purging the users directory and still not clearing out the hanging issue, I uninstalled the Active Directory plug in and re-installed the 1.24 version (newest). I am now seeing error messages trying to validate all users in the matrix where permissions are configured indicating some password issue (see below), including the ones that have logged into the system already. The hang now happesn on all jobs. Thread dumps included. Failed to test the validity of the user name XXXXXX (show details) hudson.plugins.active_directory.MultiCauseBadCredentialsException: Either no such user 'XXXXXX' or incorrect password at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:100) at hudson.plugins.active_directory.AbstractActiveDirectoryAuthenticationProvider.loadUserByUsername(AbstractActiveDirectoryAuthenticationProvider.java:27) at hudson.plugins.active_directory.ActiveDirectorySecurityRealm.loadUserByUsername(ActiveDirectorySecurityRealm.java:533) at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName_(GlobalMatrixAuthorizationStrategy.java:303) at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName(GlobalMatrixAuthorizationStrategy.java:288) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:104) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648) at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:234) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477) at org.kohsuke.stapler.Stapler.service(Stapler.java:159) at javax.servlet.http.HttpServlet.service(HttpServlet.java:45) at winstone.ServletConfiguration.execute(ServletConfiguration.java:249) at winstone.RequestDispatcher.forward(RequestDispatcher.java:335) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:74) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:85) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at winstone.RequestDispatcher.forward(RequestDispatcher.java:333) at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244) at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) at java.lang.Thread.run(Unknown Source) Cause #1: hudson.security.UserMayOrMayNotExistException: Unable to retrieve the user information without bind DN/password configured at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:154) at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:119) at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:84) at hudson.plugins.active_directory.AbstractActiveDirectoryAuthenticationProvider.loadUserByUsername(AbstractActiveDirectoryAuthenticationProvider.java:27) at hudson.plugins.active_directory.ActiveDirectorySecurityRealm.loadUserByUsername(ActiveDirectorySecurityRealm.java:533) at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName_(GlobalMatrixAuthorizationStrategy.java:303) at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName(GlobalMatrixAuthorizationStrategy.java:288) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:104) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648) at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:234) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477) at org.kohsuke.stapler.Stapler.service(Stapler.java:159) at javax.servlet.http.HttpServlet.service(HttpServlet.java:45) at winstone.ServletConfiguration.execute(ServletConfiguration.java:249) at winstone.RequestDispatcher.forward(RequestDispatcher.java:335) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:74) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:85) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at winstone.RequestDispatcher.forward(RequestDispatcher.java:333) at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244) at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) at java.lang.Thread.run(Unknown Source) Cause #2: hudson.security.UserMayOrMayNotExistException: Unable to retrieve the user information without bind DN/password configured at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:154) at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:119) at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:84) at hudson.plugins.active_directory.AbstractActiveDirectoryAuthenticationProvider.loadUserByUsername(AbstractActiveDirectoryAuthenticationProvider.java:27) at hudson.plugins.active_directory.ActiveDirectorySecurityRealm.loadUserByUsername(ActiveDirectorySecurityRealm.java:533) at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName_(GlobalMatrixAuthorizationStrategy.java:303) at hudson.security.GlobalMatrixAuthorizationStrategy$DescriptorImpl.doCheckName(GlobalMatrixAuthorizationStrategy.java:288) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:104) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648) at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:234) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:563) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:648) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477) at org.kohsuke.stapler.Stapler.service(Stapler.java:159) at javax.servlet.http.HttpServlet.service(HttpServlet.java:45) at winstone.ServletConfiguration.execute(ServletConfiguration.java:249) at winstone.RequestDispatcher.forward(RequestDispatcher.java:335) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:74) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:85) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at winstone.RequestDispatcher.forward(RequestDispatcher.java:333) at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244) at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) at java.lang.Thread.run(Unknown Source)
        Hide
        alexlombardi alexlombardi added a comment - - edited

        After some additional trial & error I have confirmed that the locking issue is being caused because Jenkins has a list of e-mail recipients identified in the extended e-mail plug-in that have NOT logged into Jenkins yet. This did not use to be a problem before and mail recipients that had not logged into Jenkins used to get e-mails just fine. The locking seems to occur whenever a job runs that is either triggered by someone that has not logged into Jenkins before checking code into Subversion and starting a build or when e-mail is sent to someone that has never logged into Jenkins.

        Also I have 2 domain names: one under which the windows service runs, another under which all users authorities are defined, comma separated as the instructions say they should be. However this results in authentication failure as follows:

        Domain=xxxx.yyyy.net,zzzz.yyyy.net site=null
        Failing to resolve domain controllers
        javax.naming.NamingException: No SRV record found for _ldap._tcp.xxxx.yyyy.net,zzzz.yyyy.net [Root exception is javax.naming.NameNotFoundException: DNS name not found [response code 3]; remaining name '_ldap._tcp.xxxx.yyyy.net,zzzz.yyyy.net']
        at hudson.plugins.active_directory.ActiveDirectorySecurityRealm$DesciprotrImpl.obtainLDAPServer(ActiveDirectorySecurityRealm.java:489)
        at hudson.plugins.active_directory.ActiveDirectorySecurityRealm$DesciprotrImpl.obtainLDAPServer(ActiveDirectorySecurityRealm.java:416)

        Adding any kind domain controller information fails as well as whatever is trying to parse the controller list fails to do so correctly.

        Show
        alexlombardi alexlombardi added a comment - - edited After some additional trial & error I have confirmed that the locking issue is being caused because Jenkins has a list of e-mail recipients identified in the extended e-mail plug-in that have NOT logged into Jenkins yet. This did not use to be a problem before and mail recipients that had not logged into Jenkins used to get e-mails just fine. The locking seems to occur whenever a job runs that is either triggered by someone that has not logged into Jenkins before checking code into Subversion and starting a build or when e-mail is sent to someone that has never logged into Jenkins. Also I have 2 domain names: one under which the windows service runs, another under which all users authorities are defined, comma separated as the instructions say they should be. However this results in authentication failure as follows: Domain=xxxx.yyyy.net,zzzz.yyyy.net site=null Failing to resolve domain controllers javax.naming.NamingException: No SRV record found for _ldap._tcp.xxxx.yyyy.net,zzzz.yyyy.net [Root exception is javax.naming.NameNotFoundException: DNS name not found [response code 3] ; remaining name '_ldap._tcp.xxxx.yyyy.net,zzzz.yyyy.net'] at hudson.plugins.active_directory.ActiveDirectorySecurityRealm$DesciprotrImpl.obtainLDAPServer(ActiveDirectorySecurityRealm.java:489) at hudson.plugins.active_directory.ActiveDirectorySecurityRealm$DesciprotrImpl.obtainLDAPServer(ActiveDirectorySecurityRealm.java:416) Adding any kind domain controller information fails as well as whatever is trying to parse the controller list fails to do so correctly.
        Hide
        kohsuke Kohsuke Kawaguchi added a comment -

        When the hang happens in the build, please obtain the thread dump by following instructions at https://wiki.jenkins-ci.org/display/JENKINS/Build+is+hanging

        Show
        kohsuke Kohsuke Kawaguchi added a comment - When the hang happens in the build, please obtain the thread dump by following instructions at https://wiki.jenkins-ci.org/display/JENKINS/Build+is+hanging
        Hide
        kohsuke Kohsuke Kawaguchi added a comment -

        Downgrading since I'm not hearing back from the reporter.

        Show
        kohsuke Kohsuke Kawaguchi added a comment - Downgrading since I'm not hearing back from the reporter.

          People

          • Assignee:
            alexlombardi alexlombardi
            Reporter:
            alexlombardi alexlombardi
          • Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated: