Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37737

Intermittent login failures with Active Directory / Matrix-based security

    Details

    • Similar Issues:

      Description

      Helllo! We are experiencing intermittent login issues since early August, 2016 for all users from any browser or workstation (location does not seem to be an issue). We have a cross domain - VPN tunnel, which has not experienced recent outages to cause failed logons or AD lookups. Other systems relying on the VPN tunnel are not experiencing authentication issues. Successful manual telnet tests between the Domain Controllers were successful during Jenkins failed logins. We are not ruling out a network issue but we can't see any problems. We have not recently upgraded Jenkins or the Active Directory Plugin.

      Looking forward to any help to resolve our issue.

      Output from log:

      Aug 27, 2016 7:11:51 AM hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider retrieveUser
      WARNING: Credential exception trying to authenticate against ####### domain
      org.acegisecurity.BadCredentialsException: Failed to retrieve user information for ##############; nested exception is javax.naming.PartialResultException Root exception is javax.naming.CommunicationException: DomainDnsZones.######## [Root exception is java.net.ConnectException: Connection timed out: connect]
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider$1.call(ActiveDirectoryUnixAuthenticationProvider.java:332)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider$1.call(ActiveDirectoryUnixAuthenticationProvider.java:235)
      at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
      at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
      at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
      at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
      at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
      at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
      at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:235)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:200)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.retrieveUser(ActiveDirectoryUnixAuthenticationProvider.java:142)
      at org.acegisecurity.providers.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:122)
      at org.acegisecurity.providers.ProviderManager.doAuthentication(ProviderManager.java:200)
      at org.acegisecurity.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:47)
      at org.acegisecurity.ui.webapp.AuthenticationProcessingFilter.attemptAuthentication(AuthenticationProcessingFilter.java:74)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:252)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
      at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
      at org.eclipse.jetty.server.Server.handle(Server.java:370)
      at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
      at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
      at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
      at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
      at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
      at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
      at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
      at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
      at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
      at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at java.lang.Thread.run(Unknown Source)
      Caused by: javax.naming.PartialResultException Root exception is javax.naming.CommunicationException: DomainDnsZones.####### [Root exception is java.net.ConnectException: Connection timed out: connect]
      at com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMoreImpl(Unknown Source)
      at com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMore(Unknown Source)
      at hudson.plugins.active_directory.LDAPSearchBuilder.searchOne(LDAPSearchBuilder.java:86)
      at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider$1.call(ActiveDirectoryUnixAuthenticationProvider.java:280)
      ... 55 more

        Attachments

          Activity

          Hide
          jr Jens Runge added a comment -

          Hi Derek,

          I admit its only a dirty workaround. Our Teams isn't that big, so I have done it every time somone can't login.
          Meanwhile our Administrators cleaned up the DNS, so that DomainDnsZones.<domain> only points to available systems.

          I assume, your Jenkins Service is running with a local server account.
          Then you can try to use a domain account as Jenkins service account on your Jenkins Server.
          Maybe then the windows native authentication for directory listing should not fail and would not fallback to non native authentication.

          Show
          jr Jens Runge added a comment - Hi Derek, I admit its only a dirty workaround. Our Teams isn't that big, so I have done it every time somone can't login. Meanwhile our Administrators cleaned up the DNS, so that DomainDnsZones.<domain> only points to available systems. I assume, your Jenkins Service is running with a local server account. Then you can try to use a domain account as Jenkins service account on your Jenkins Server. Maybe then the windows native authentication for directory listing should not fail and would not fallback to non native authentication.
          Hide
          dsakauye Derek Sakauye added a comment -

          Hi JR,

          Our team is pretty small too. So we're in a similar pickle. We actually are running Jenkins with a domain service account. I'll try your "flushdns" workaround next time we can't login. I'll check with Domain Admins about cleaning up the DNS too.

          Thanks,

          • Derek
          Show
          dsakauye Derek Sakauye added a comment - Hi JR, Our team is pretty small too. So we're in a similar pickle. We actually are running Jenkins with a domain service account. I'll try your "flushdns" workaround next time we can't login. I'll check with Domain Admins about cleaning up the DNS too. Thanks, Derek
          Hide
          vkadiri Vijaya Bhaskar Kadiri added a comment -

          Hello!

          We have the same problems with Jenkins ver. 2.7.1. Any ETA for the fix?

          Show
          vkadiri Vijaya Bhaskar Kadiri added a comment - Hello! We have the same problems with Jenkins ver. 2.7.1. Any ETA for the fix?
          Hide
          romgo Hugo added a comment -

          Hello,

          I have the same issue, Jenkins can only reach 2 controller i my domain, so I use the servers option to point to the server it can reach (don't want to use round robin from Active directory).
          But I can see request to other domain controller which are not configured in my list.
          This cause login failure and various timeout.

          Here is my configuration :

            <securityRealm class="hudson.plugins.active_directory.ActiveDirectorySecurityRealm" plugin="active-directory@2.0">
              <domains>
                <hudson.plugins.active__directory.ActiveDirectoryDomain>
                  <!-- <name>domain.local</name> -->
                  <servers>192.168.1.2:636,192.168.1.3:636</servers>
                </hudson.plugins.active__directory.ActiveDirectoryDomain>
              </domains>
              <bindName>CN=LDAP,OU=Users,DC=domain,DC=local</bindName>
              <bindPassword>DDHCNCNCCC/r9Rxf0HvCqt0QVuU=</bindPassword>
              <groupLookupStrategy>AUTO</groupLookupStrategy>
              <removeIrrelevantGroups>false</removeIrrelevantGroups>
            </securityRealm>
          

          plugin : 2.0
          jenkins : 2.34

          Show
          romgo Hugo added a comment - Hello, I have the same issue, Jenkins can only reach 2 controller i my domain, so I use the servers option to point to the server it can reach (don't want to use round robin from Active directory). But I can see request to other domain controller which are not configured in my list. This cause login failure and various timeout. Here is my configuration : <securityRealm class= "hudson.plugins.active_directory.ActiveDirectorySecurityRealm" plugin= "active-directory@2.0" > <domains> <hudson.plugins.active__directory.ActiveDirectoryDomain> <!-- <name>domain.local</name> --> <servers>192.168.1.2:636,192.168.1.3:636</servers> </hudson.plugins.active__directory.ActiveDirectoryDomain> </domains> <bindName>CN=LDAP,OU=Users,DC=domain,DC=local</bindName> <bindPassword>DDHCNCNCCC/r9Rxf0HvCqt0QVuU=</bindPassword> <groupLookupStrategy>AUTO</groupLookupStrategy> <removeIrrelevantGroups> false </removeIrrelevantGroups> </securityRealm> plugin : 2.0 jenkins : 2.34
          Hide
          alphamikevictor Agustin Munoz added a comment -

          Hello,

          I am on the same situation, I was able to perform a small workaround by setting the environment variable com.sun.jndi.ldap.connect.timeout to 100 on advanced properties, now instead of getting stuck in the login screen for several minutes it only takes a few seconds to authenticate the user.

          plugin: 2.4

          jenkins: 2.46.2

          Show
          alphamikevictor Agustin Munoz added a comment - Hello, I am on the same situation, I was able to perform a small workaround by setting the environment variable com.sun.jndi.ldap.connect.timeout to 100 on advanced properties, now instead of getting stuck in the login screen for several minutes it only takes a few seconds to authenticate the user. plugin: 2.4 jenkins: 2.46.2

            People

            • Assignee:
              fbelzunc Félix Belzunce Arcos
              Reporter:
              dsakauye Derek Sakauye
            • Votes:
              3 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: