Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55070

"Jenkins.instance is missing" error when restarting Jenkins

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: core
    • Labels:
    • Environment:
      Windows Server 2016
      Java 1.8.0_191-b12
      Jenkins 2.150.1
    • Similar Issues:

      Description

      Sometimes when I try to restart Jenkins (eg. after upgrading the plugins), it just starts returning HTTP 500 errors, and I need to restart the Jenkins service.

      When this happens, I see lots of these errors in the logs:

      Dec 07, 2018 12:03:45 AM org.eclipse.jetty.server.HttpChannel handleException
      WARNING: /
      java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
       at jenkins.model.Jenkins.get(Jenkins.java:758)
       at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
       at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
       at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
       at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
       at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
       at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
       at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
       at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
       at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
       at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
       at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
       at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
       at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
       at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
       at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
       at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
       at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
       at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
       at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
       at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
       at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
       at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
       at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
       at org.eclipse.jetty.server.Server.handle(Server.java:503)
       at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
       at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
       at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
       at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
       at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
       at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
       at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
       at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
       at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
       at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
       at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
       at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
       at java.lang.Thread.run(Unknown Source)

       

        Attachments

          Activity

          Hide
          danielbeck Daniel Beck added a comment -

          This happens after other errors. Look for those.

          Show
          danielbeck Daniel Beck added a comment - This happens after other errors. Look for those.
          Hide
          daniel15 Daniel Lo Nigro added a comment -

          I saw this again today while restarting after installing a plugin... I didn't see any other errors before it though. Seems like it happens after "INFO: Jenkins stopped" and then it doesn't restart properly?

          I did see "java.lang.IllegalStateException: Error already set" and "java.lang.NullPointerException at hudson.security.TokenBasedRememberMeServices2.autoLogin" errors in the stack of the second instance of the error though.

          Dec 07, 2018 9:40:48 PM jenkins.model.Jenkins$23 run
          INFO: Restart in 10 seconds
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$23 run
          INFO: Restarting VM as requested by daniel15
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins cleanUp
          INFO: Stopping Jenkins
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$18 onAttained
          INFO: Started termination
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$18 onAttained
          INFO: Completed termination
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins _cleanUpDisconnectComputers
          INFO: Starting node disconnection
          Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpShutdownPluginManager
          INFO: Stopping plugin manager
          Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpPersistQueue
          INFO: Persisting build queue
          Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpAwaitDisconnects
          INFO: Waiting for node disconnection completion
          Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins cleanUp
          INFO: Jenkins stopped
          Dec 07, 2018 9:42:45 PM org.eclipse.jetty.server.HttpChannel handleException
          WARNING: /
          java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          	at jenkins.model.Jenkins.get(Jenkins.java:758)
          	at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
          	at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
          	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
          	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
          	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
          	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
          	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
          	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
          	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
          	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
          	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
          	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
          	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          	at org.eclipse.jetty.server.Server.handle(Server.java:503)
          	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
          	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
          	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
          	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
          	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
          	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
          	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
          	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
          	at java.lang.Thread.run(Unknown Source)Dec 07, 2018 9:42:45 PM org.eclipse.jetty.server.HttpChannel handleException
          WARNING: ERROR dispatch failed
          java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          	at jenkins.model.Jenkins.get(Jenkins.java:758)
          	at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
          	at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
          	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
          	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
          	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
          	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
          	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
          	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
          	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
          	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
          	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
          	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
          	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          	at org.eclipse.jetty.server.Server.handle(Server.java:503)
          	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
          	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
          	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
          	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
          	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
          	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
          	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
          	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
          	at java.lang.Thread.run(Unknown Source)
          	Suppressed: java.lang.IllegalStateException: Error already set
          		at org.eclipse.jetty.server.HttpChannelState.onError(HttpChannelState.java:758)
          		at org.eclipse.jetty.server.HttpChannel.handleException(HttpChannel.java:590)
          		at org.eclipse.jetty.server.HttpChannelOverHttp.handleException(HttpChannelOverHttp.java:489)
          		at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:526)
          		... 12 more
          	Caused by: java.lang.NullPointerException
          		at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:126)
          		at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
          		at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
          		at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
          		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          		at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
          		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          		at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
          		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          		at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
          		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
          		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
          		at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
          		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          		at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
          		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
          		at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
          		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
          		at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
          		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
          		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
          		at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
          		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
          		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
          		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          		at org.eclipse.jetty.server.Server.handle(Server.java:503)
          		at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
          		... 12 more
          
          Show
          daniel15 Daniel Lo Nigro added a comment - I saw this again today while restarting after installing a plugin... I didn't see any other errors before it though. Seems like it happens after "INFO: Jenkins stopped" and then it doesn't restart properly? I did see "java.lang.IllegalStateException: Error already set" and "java.lang.NullPointerException at hudson.security.TokenBasedRememberMeServices2.autoLogin" errors in the stack of the second instance of the error though. Dec 07, 2018 9:40:48 PM jenkins.model.Jenkins$23 run INFO: Restart in 10 seconds Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$23 run INFO: Restarting VM as requested by daniel15 Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins cleanUp INFO: Stopping Jenkins Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$18 onAttained INFO: Started termination Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$18 onAttained INFO: Completed termination Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins _cleanUpDisconnectComputers INFO: Starting node disconnection Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpShutdownPluginManager INFO: Stopping plugin manager Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpPersistQueue INFO: Persisting build queue Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpAwaitDisconnects INFO: Waiting for node disconnection completion Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins cleanUp INFO: Jenkins stopped Dec 07, 2018 9:42:45 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: / java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang. Thread .run(Unknown Source)Dec 07, 2018 9:42:45 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: ERROR dispatch failed java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang. Thread .run(Unknown Source) Suppressed: java.lang.IllegalStateException: Error already set at org.eclipse.jetty.server.HttpChannelState.onError(HttpChannelState.java:758) at org.eclipse.jetty.server.HttpChannel.handleException(HttpChannel.java:590) at org.eclipse.jetty.server.HttpChannelOverHttp.handleException(HttpChannelOverHttp.java:489) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:526) ... 12 more Caused by: java.lang.NullPointerException at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:126) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) ... 12 more
          Hide
          danielbeck Daniel Beck added a comment -

          Yep, Jenkins apparently cannot restart itself, and just stopped.

          Potential problem with winsw / how you run Jenkins. Check the Windows event log, the Jenkins service log, and provide complete information how and when (what version originally) Jenkins was installed.

          Poking Oleg Nenashev in case he's still maintaining winsw.

          Show
          danielbeck Daniel Beck added a comment - Yep, Jenkins apparently cannot restart itself, and just stopped. Potential problem with winsw / how you run Jenkins. Check the Windows event log, the Jenkins service log, and provide complete information how and when (what version originally) Jenkins was installed. Poking Oleg Nenashev in case he's still maintaining winsw .
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          Daniel Beck yes, I somewhat maintain it. Though I do not see evidence that the issue caused by WinSW:

           

           

          Caused by: java.lang.NullPointerException
          		at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:126)
          		at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
          

          Due to whatever reason the autoLogin() logic is invoked before Jenkins holder class is fully initialized: https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/security/TokenBasedRememberMeServices2.java#L126 . This is a regression caused by the SECURITY-996 fix in this commit by Wadeck Follonierhttps://github.com/jenkinsci/jenkins/commit/ef9583a24abc4de157e1570cb32d7a273d327f36 

           

           

           

          Show
          oleg_nenashev Oleg Nenashev added a comment - Daniel Beck yes, I somewhat maintain it. Though I do not see evidence that the issue caused by WinSW:     Caused by: java.lang.NullPointerException at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:126) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) Due to whatever reason the autoLogin() logic is invoked before Jenkins holder class is fully initialized: https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/security/TokenBasedRememberMeServices2.java#L126 . This is a regression caused by the SECURITY-996 fix in this commit by Wadeck Follonier :  https://github.com/jenkinsci/jenkins/commit/ef9583a24abc4de157e1570cb32d7a273d327f36        
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          The root exception is also a bug IMHO, but it existed in the code for a while

          java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          	at jenkins.model.Jenkins.get(Jenkins.java:758)
                  at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
          

          Instead of invoking Jenkins.get(), the code should use the Jenkins instance passed by the initializer

          For me it's hard to say what is a root cause, but we really need to cleanup the code a bit

          Show
          oleg_nenashev Oleg Nenashev added a comment - The root exception is also a bug IMHO, but it existed in the code for a while java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) Instead of invoking Jenkins.get(), the code should use the Jenkins instance passed by the initializer For me it's hard to say what is a root cause, but we really need to cleanup the code a bit
          Hide
          danielbeck Daniel Beck added a comment -

          This does not look like it would affect a Jenkins restart. Note this occurred two minutes after Jenkins was stopped as part of a restart.

          Show
          danielbeck Daniel Beck added a comment - This does not look like it would affect a Jenkins restart. Note this occurred two minutes after Jenkins was stopped as part of a restart.
          Hide
          daniel15 Daniel Lo Nigro added a comment - - edited

          Check the Windows event log, the Jenkins service log, and provide complete information how and when (what version originally) Jenkins was installed.

          Daniel Beck - I originally installed Jenkins 2.138.3 via the Windows installer. The only change I've made is that I've changed the service to run as a separate Jenkins user rather than as SYSTEM for better security (maybe that's causing issues?).

          I do see some errors like this in the Event Viewer:

          SIGINT to 6536 failed - Killing as fallback

          What is the "Jenkins service log"? Is that different to the C:\Program Files (x86)\Jenkins\jenkins.err.log file I've been looking at?

           

          Show
          daniel15 Daniel Lo Nigro added a comment - - edited Check the Windows event log, the Jenkins service log, and provide complete information how and when (what version originally) Jenkins was installed. Daniel Beck - I originally installed Jenkins 2.138.3 via the Windows installer. The only change I've made is that I've changed the service to run as a separate Jenkins user rather than as SYSTEM for better security (maybe that's causing issues?). I do see some errors like this in the Event Viewer: SIGINT to 6536 failed - Killing as fallback What is the "Jenkins service log"? Is that different to the C:\Program Files (x86)\Jenkins\jenkins.err.log file I've been looking at?  
          Hide
          mcardenasblanco Manuel Cardenas added a comment - - edited

          Daniel Beck , we have seen this issue in a non windows installation, Centos 7.x

           * OpenJDK 64-Bit Server VM 25.161-b14

          The complete context: We are upgrading Jenkins from 2.138.3.1 to 2.138.4.3 using a war file. Once that we use the new war file, the service is not able to start. We can see the trace below.

           

          Dec 11, 2018 3:22:10 PM org.eclipse.jetty.server.HttpChannel handleException
           WARNING: /login
           java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
           at jenkins.model.Jenkins.get(Jenkins.java:758) 
          ...
          
          Dec 11, 2018 3:22:42 PM org.eclipse.jetty.server.HttpChannel handleException
           WARNING: /tcpSlaveAgentListener/
           java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
           at jenkins.model.Jenkins.get(Jenkins.java:758)
           at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
           at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
           at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
           at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
           at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
           at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
           at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
           at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
           at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.Server.handle(Server.java:531)
           at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
           at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
           at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
           at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
           at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
           at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
           at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
           at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
           at java.lang.Thread.run(Thread.java:748)
           Dec 11, 2018 3:22:42 PM org.eclipse.jetty.server.HttpChannel handleException
           WARNING: ERROR dispatch failed
           java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
           at jenkins.model.Jenkins.get(Jenkins.java:758)
           at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
           at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
           at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
           at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
           at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
           at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
           at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
           at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
           at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.Server.handle(Server.java:531)
           at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
           Stopping Jenkins Daemon...
           jenkins.service: main process exited, code=exited, status=143/n/a
           Stopped Jenkins Daemon.
           Unit jenkins.service entered failed state.
           jenkins.service failed.
          
          Show
          mcardenasblanco Manuel Cardenas added a comment - - edited Daniel Beck , we have seen this issue in a non windows installation, Centos 7.x  * OpenJDK 64-Bit Server VM 25.161-b14 The complete context: We are upgrading Jenkins from 2.138.3.1 to 2.138.4.3 using a war file. Once that we use the new war file, the service is not able to start. We can see the trace below.   Dec 11, 2018 3:22:10 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: /login java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) ... Dec 11, 2018 3:22:42 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: /tcpSlaveAgentListener/ java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang. Thread .run( Thread .java:748) Dec 11, 2018 3:22:42 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: ERROR dispatch failed java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) Stopping Jenkins Daemon... jenkins.service: main process exited, code=exited, status=143/n/a Stopped Jenkins Daemon. Unit jenkins.service entered failed state. jenkins.service failed.
          Hide
          wfollonier Wadeck Follonier added a comment -

          Manuel Cardenas From my PoV, you are not affected by the same exception (rememberMe) but another introduced in previous release. It seems very likely that the code in rememberMe is not the culprit.

          In your case the full stacktrace shows:

          Caused by: java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          at jenkins.model.Jenkins.get(Jenkins.java:758)
          at jenkins.telemetry.impl.StaplerDispatches$StaplerTrace.record(StaplerDispatches.java:105)
          at org.kohsuke.stapler.EvaluationTrace$ApplicationTracer.trace(EvaluationTrace.java:77)
          at org.kohsuke.stapler.Dispatcher.anonymizedTraceEval(Dispatcher.java:76)
          at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:685)
          at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870)
          at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668)
          at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
          at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
          at com.cloudbees.jenkins.ha.HAHealthCheckFilter.doFilter(HAHealthCheckFilter.java:35)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
          at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
          at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:64)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:169)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
          at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
          at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
          at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
          at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
          at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
          at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
          at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
          at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
          at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
          at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
          at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
          at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
          at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56)
          at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          at org.eclipse.jetty.server.Server.handle(Server.java:531)
          at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
          ... 12 more
          

          So I created 2 PR for this ticket, 3805 to correct the rememberMe but I highly suspect to not correct the problem and 3806 to correct the legacy behavior there.
          From my PoV, the root cause is something else, in the startup mechanism that caused both exceptions.

          Show
          wfollonier Wadeck Follonier added a comment - Manuel Cardenas From my PoV, you are not affected by the same exception (rememberMe) but another introduced in previous release. It seems very likely that the code in rememberMe is not the culprit. In your case the full stacktrace shows: Caused by: java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at jenkins.telemetry.impl.StaplerDispatches$StaplerTrace.record(StaplerDispatches.java:105) at org.kohsuke.stapler.EvaluationTrace$ApplicationTracer.trace(EvaluationTrace.java:77) at org.kohsuke.stapler.Dispatcher.anonymizedTraceEval(Dispatcher.java:76) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:685) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) at com.cloudbees.jenkins.ha.HAHealthCheckFilter.doFilter(HAHealthCheckFilter.java:35) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:64) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:169) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) ... 12 more So I created 2 PR for this ticket, 3805 to correct the rememberMe but I highly suspect to not correct the problem and 3806 to correct the legacy behavior there. From my PoV, the root cause is something else, in the startup mechanism that caused both exceptions.
          Hide
          mcardenasblanco Manuel Cardenas added a comment -

          Thank you for your analysis Wadeck Follonier, we will try to dig a little bit more on what is happening as it is preventing us from applying a security upgrade.

          Show
          mcardenasblanco Manuel Cardenas added a comment - Thank you for your analysis Wadeck Follonier , we will try to dig a little bit more on what is happening as it is preventing us from applying a security upgrade.
          Hide
          dnusbaum Devin Nusbaum added a comment -

          I think the problem in Manuel Cardenas's case would be fixed by https://github.com/jenkinsci/jenkins/pull/3712 which made it into the 2.150 LTS line but not 2.138. Even so, it's not clear to me why that issue would happen consistently during a restart.

          Show
          dnusbaum Devin Nusbaum added a comment - I think the problem in Manuel Cardenas 's case would be fixed by  https://github.com/jenkinsci/jenkins/pull/3712  which made it into the 2.150 LTS line but not 2.138. Even so, it's not clear to me why that issue would happen consistently during a restart.
          Hide
          mcardenasblanco Manuel Cardenas added a comment -

          Devin Nusbaum are you aware of any potential workaround for this?

          Show
          mcardenasblanco Manuel Cardenas added a comment - Devin Nusbaum are you aware of any potential workaround for this?
          Hide
          dnusbaum Devin Nusbaum added a comment - - edited

          No, but maybe if the problem is inbound HTTP requests hitting the servlet/Stapler layer while Jenkins is shutting down/restarting then blocking all HTTP traffic to Jenkins while the restart is happening might help (only really possible if you have some kind of reverse proxy in front of Jenkins or something, and I am not sure what kind of configuration would be needed).

          Edit: Manuel Cardenas thinking about it some more, I think you could also work around the issue by passing -Dhudson.model.UsageStatistics.disabled=true as a system property when starting Jenkins so that this comparison short circuits before it gets to the Jenkins.get() call.

          Show
          dnusbaum Devin Nusbaum added a comment - - edited No, but maybe if the problem is inbound HTTP requests hitting the servlet/Stapler layer while Jenkins is shutting down/restarting then blocking all HTTP traffic to Jenkins while the restart is happening might help (only really possible if you have some kind of reverse proxy in front of Jenkins or something, and I am not sure what kind of configuration would be needed). Edit: Manuel Cardenas thinking about it some more, I think you could also work around the issue by passing -Dhudson.model.UsageStatistics.disabled=true as a system property when starting Jenkins so that this comparison short circuits before it gets to the Jenkins.get() call.

            People

            • Assignee:
              wfollonier Wadeck Follonier
              Reporter:
              daniel15 Daniel Lo Nigro
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: