Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49147

JEP-200 location-based whitelisting broken in obsolete versions of Tomcat

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: core
    • Environment:
      Jenkins 2.103 running on Debian 8 with OpenJDK 1.8.0_131 and Tomcat8
    • Similar Issues:

      Description

      I followed my normal procedure in updating our Jenkins builder:

      • Update all Jenkins plugins first.
      • Update jenkins core (using .war file)

      I got the error message about JEP-200 and XStream, so I browsed to the wiki page. It had a note in there about -Dhudson.remoting.ClassFilter=, with a comma-separated list of class names from the log file. I got the list of classes:

      # grep -ri rejecting /var/log/tomcat8/catalina.out | awk -F ' ' '{print $2}' | sort | uniq
      hudson.model.Cause$UserIdCause
      hudson.model.Hudson$CloudList
      hudson.model.MyViewsProperty
      hudson.model.PaneStatusProperties
      hudson.model.Queue$State
      hudson.model.UpdateSite
      hudson.model.View$PropertyList
      hudson.node_monitors.ArchitectureMonitor
      hudson.node_monitors.ClockMonitor
      hudson.node_monitors.DiskSpaceMonitor
      hudson.node_monitors.ResponseTimeMonitor
      hudson.node_monitors.SwapSpaceMonitor
      hudson.node_monitors.TemporarySpaceMonitor
      hudson.remoting.RemoteInvocationHandler$RPCRequest
      hudson.scm.SCMRevisionState$None
      hudson.search.UserSearchProperty
      hudson.slaves.JNLPLauncher
      hudson.slaves.RetentionStrategy$2
      hudson.tasks.LogRotator
      hudson.tasks.Shell$DescriptorImpl
      hudson.triggers.SCMTrigger$BuildAction
      hudson.triggers.SCMTrigger$DescriptorImpl
      hudson.triggers.SCMTrigger$SCMTriggerCause
      hudson.util.CopyOnWriteMap$Hash
      jenkins.model.BuildDiscarderProperty
      jenkins.model.ProjectNamingStrategy$DefaultProjectNamingStrategy
      jenkins.security.ApiTokenProperty
      jenkins.security.LastGrantedAuthoritiesProperty
      jenkins.slaves.RemotingWorkDirSettings
      

      So, I added those as a comma-separated list to -Dhudson.remoting.ClassFilter= and restarted Tomcat. Jenkins came back (authentication worked, but no build information is available, and slaves cannot connect), but I am now seeing a message about "You have data stored in an older format and/or unreadable data.". I am a bit afraid I will lose my build history and other metadata if I click on "Discard Unreadable Data". Is that a "safe" operation for my builds metadata?

      Also, why do I need to add so many exclusions to the hudson.remoting.ClassFilter, some of which seem to be internal to jenkins/hudson? Shouldn't that "just work"? Did I do something wrong in the upgrade?

        Attachments

          Issue Links

            Activity

            Hide
            kmott Kyle Mott added a comment -

            To clarify a bit further, I did take a snapshot of my Jenkins VM after the upgrade to 2.103 and my hudson.remoting.ClassFilter changes, but before I did "Discard Old Data". I did click on "Discard Old Data", and that doesn't seem to have changed anything. My slave machines are still all disconnected.

            Show
            kmott Kyle Mott added a comment - To clarify a bit further, I did take a snapshot of my Jenkins VM after the upgrade to 2.103 and my hudson.remoting.ClassFilter changes, but before I did "Discard Old Data". I did click on "Discard Old Data", and that doesn't seem to have changed anything. My slave machines are still all disconnected.
            Hide
            jglick Jesse Glick added a comment -

            Shouldn't that "just work"?

            Yes of course. Nothing in Jenkins core needs to be whitelisted.

            I suspect ClassFilterImpl.isLocationWhitelisted is failing to grasp that your Jenkins core is really Jenkins core, for some reason related to an unusual packaging. Perhaps because you are running on Tomcat—almost everyone uses the built-in Winstone instead, and I do not know of any automated tests which check compatibility with other containers.

            You will need to create a FINE logger on jenkins.security.ClassFilterImpl to find out. Normally you will see a single message early on to the effect of

            file:/usr/share/jenkins/WEB-INF/lib/jenkins-core-2.103.jar seems to be the location of Jenkins core, OK
            
            Show
            jglick Jesse Glick added a comment - Shouldn't that "just work"? Yes of course. Nothing in Jenkins core needs to be whitelisted. I suspect ClassFilterImpl.isLocationWhitelisted is failing to grasp that your Jenkins core is really Jenkins core, for some reason related to an unusual packaging. Perhaps because you are running on Tomcat—almost everyone uses the built-in Winstone instead, and I do not know of any automated tests which check compatibility with other containers. You will need to create a FINE logger on jenkins.security.ClassFilterImpl to find out. Normally you will see a single message early on to the effect of file:/usr/share/jenkins/WEB-INF/lib/jenkins-core-2.103.jar seems to be the location of Jenkins core, OK
            Hide
            kmott Kyle Mott added a comment -

            Thank you for the help Jesse Glick, really appreciate it. I tried creating the FINE logger, but when I went to save, it threw this in the UI:

            java.lang.UnsupportedOperationException: Refusing to marshal hudson.logging.LogRecorder for security reasons; see https://jenkins.io/redirect/class-filter/
            	at hudson.util.XStream2$BlacklistedTypesConverter.marshal(XStream2.java:530)
            	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
            	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
            	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
            	at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
            	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
            	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
            	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
            	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
            	at hudson.XmlFile.write(XmlFile.java:194)
            Caused: java.io.IOException
            	at hudson.XmlFile.write(XmlFile.java:201)
            	at hudson.logging.LogRecorder.save(LogRecorder.java:328)
            	at hudson.logging.LogRecorder.doConfigSubmit(LogRecorder.java:303)
            	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
            	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:343)
            	at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:77)
            	at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26)
            	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:184)
            	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:117)
            	at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:129)
            	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
            	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
            	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
            	at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374)
            	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
            	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
            	at org.kohsuke.stapler.MetaClass$3.doDispatch(MetaClass.java:209)
            	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
            	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
            	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
            	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
            	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
            	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
            	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
            	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
            	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
            	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
            	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:225)
            	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
            	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
            	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
            	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
            	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
            	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
            	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
            	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:99)
            	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
            	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
            	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
            	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
            	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
            	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
            	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
            	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
            	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
            	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
            	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
            	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
            	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
            	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
            	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
            	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
            	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:618)
            	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
            	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
            	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
            	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
            	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537)
            	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
            	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658)
            	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
            	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1570)
            	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1527)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
            	at java.lang.Thread.run(Thread.java:748)
            
            Show
            kmott Kyle Mott added a comment - Thank you for the help Jesse Glick , really appreciate it. I tried creating the FINE logger, but when I went to save, it threw this in the UI: java.lang.UnsupportedOperationException: Refusing to marshal hudson.logging.LogRecorder for security reasons; see https: //jenkins.io/redirect/ class- filter/ at hudson.util.XStream2$BlacklistedTypesConverter.marshal(XStream2.java:530) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43) at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82) at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37) at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026) at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015) at com.thoughtworks.xstream.XStream.toXML(XStream.java:988) at hudson.XmlFile.write(XmlFile.java:194) Caused: java.io.IOException at hudson.XmlFile.write(XmlFile.java:201) at hudson.logging.LogRecorder.save(LogRecorder.java:328) at hudson.logging.LogRecorder.doConfigSubmit(LogRecorder.java:303) at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627) at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:343) at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:77) at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:184) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:117) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:129) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.MetaClass$3.doDispatch(MetaClass.java:209) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154) at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:225) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:99) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:618) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1570) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1527) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang. Thread .run( Thread .java:748)
            Hide
            kmott Kyle Mott added a comment -

            It looks like even though the UI threw the error, it did save the log recorder settings. Went back in to it, and this was in there:

            Jan 24, 2018 1:26:09 PM FINE jenkins.security.ClassFilterImpl lambda$isLocationWhitelisted$2
            jar:file:/var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar!/hudson/logging/LogRecorder$Target.class is not recognized; rejecting
            Jan 24, 2018 1:26:09 PM WARNING jenkins.security.ClassFilterImpl lambda$isBlacklisted$1
            hudson.logging.LogRecorder$Target in jar:file:/var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar!/hudson/logging/LogRecorder$Target.class might be dangerous, so rejecting; see https://jenkins.io/redirect/class-filter/
            
            Show
            kmott Kyle Mott added a comment - It looks like even though the UI threw the error, it did save the log recorder settings. Went back in to it, and this was in there: Jan 24, 2018 1:26:09 PM FINE jenkins.security.ClassFilterImpl lambda$isLocationWhitelisted$2 jar:file:/ var /lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar!/hudson/logging/LogRecorder$Target. class is not recognized; rejecting Jan 24, 2018 1:26:09 PM WARNING jenkins.security.ClassFilterImpl lambda$isBlacklisted$1 hudson.logging.LogRecorder$Target in jar:file:/ var /lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar!/hudson/logging/LogRecorder$Target. class might be dangerous, so rejecting; see https: //jenkins.io/redirect/ class- filter/
            Hide
            jglick Jesse Glick added a comment -

            Seems to be a bug in Tomcat. A valid code source would be jar:file:/var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar!/ or file:/var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar or even (stretching the limits of logic) jar:file:/var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar. In no case should it include the resource path within the code source.

            At any rate, sounds easy enough to work around, probably using hudson.remoting.Which, assuming I can get a working test environment to reproduce. I recommend you just use the built-in Winstone, though.

            Show
            jglick Jesse Glick added a comment - Seems to be a bug in Tomcat. A valid code source would be jar:file: /var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar!/ or file: /var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar or even (stretching the limits of logic) jar:file: /var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/jenkins-core-2.103.jar . In no case should it include the resource path within the code source. At any rate, sounds easy enough to work around, probably using hudson.remoting.Which , assuming I can get a working test environment to reproduce. I recommend you just use the built-in Winstone, though.
            Hide
            jglick Jesse Glick added a comment -

            Hmm, I cannot reproduce any problem using Tomcat 8.5.27 via the tomcat:latest Docker image. I get for example

            permitting hudson.logging.LogRecorder due to its location in file:/usr/local/tomcat/webapps/jenkins/WEB-INF/lib/jenkins-core-2.104-SNAPSHOT.jar
            

            From your log file, seems you are using 8.0.14, so I guess I will try to dig up something older.

            Show
            jglick Jesse Glick added a comment - Hmm, I cannot reproduce any problem using Tomcat 8.5.27 via the tomcat:latest Docker image. I get for example permitting hudson.logging.LogRecorder due to its location in file:/usr/local/tomcat/webapps/jenkins/WEB-INF/lib/jenkins-core-2.104-SNAPSHOT.jar From your log file, seems you are using 8.0.14, so I guess I will try to dig up something older.
            Hide
            jglick Jesse Glick added a comment -

            Well I put together https://github.com/jglick/tomcat-jenkins but it is working fine for me on any Tomcat version I could find—7, 8.0.x, 8.5.x, 9.

            The fix is probably easy enough but I would not like to make a change I cannot test. Is there some way to reproduce this from scratch?

            Show
            jglick Jesse Glick added a comment - Well I put together https://github.com/jglick/tomcat-jenkins  but it is working fine for me on any Tomcat version I could find—7, 8.0.x, 8.5.x, 9. The fix is probably easy enough but I would not like to make a change I cannot test. Is there some way to reproduce this from scratch?
            Hide
            kmott Kyle Mott added a comment - - edited

            I am able to reproduce this by installing a fresh copy of Tomcat8 on Debian 8 Jessie (which is 8.0.15), and creating a simple config.xml in /home/jenkins/config.xml with the content <hudson/>. It consistently throws the same errors about not recognizing jenkins-core-2.103.jar.

            If I remove the config.xml and restart Tomcat8, everything works as expected.

            However, no matter what Docker version of Tomcat8 I use, I cannot repro (if I go all the way down to 8.0.15, Jenkins refuses to start at all, and complains about AWT being misconfigured and needing to enable -Djava.awt.headless=true).

            At this point, I guess I'll have to switch to running Winstone directly, instead of using Tomcat8, unless you have any other ideas?

            Is it safe to point Winstone at my existing /home/jenkins directory (with my config.xml and other metadata)? Will it pick up the previous settings from Tomcat8 (which will be stopped/disabled)?

            Show
            kmott Kyle Mott added a comment - - edited I am able to reproduce this by installing a fresh copy of Tomcat8 on Debian 8 Jessie (which is 8.0.15), and creating a simple config.xml in /home/jenkins/config.xml with the content <hudson/> . It consistently throws the same errors about not recognizing jenkins-core-2.103.jar. If I remove the config.xml and restart Tomcat8, everything works as expected. However, no matter what Docker version of Tomcat8 I use, I cannot repro (if I go all the way down to 8.0.15, Jenkins refuses to start at all, and complains about AWT being misconfigured and needing to enable -Djava.awt.headless=true ). At this point, I guess I'll have to switch to running Winstone directly, instead of using Tomcat8, unless you have any other ideas? Is it safe to point Winstone at my existing /home/jenkins directory (with my config.xml and other metadata)? Will it pick up the previous settings from Tomcat8 (which will be stopped/disabled)?
            Hide
            jglick Jesse Glick added a comment -

            no matter what Docker version of Tomcat8 I use, I cannot repro

            Odd. I will try to reproduce using a non-Docker-based installation, and/or using the Jessie image to install Tomcat from update sources. Again, if I can reproduce, the fix is probably straightforward and might help users of even more exotic containers.

            Is it safe to point Winstone at my existing /home/jenkins directory (with my config.xml and other metadata)? Will it pick up the previous settings from Tomcat8

            It should. Probably goes without saying, but—create a backup first.

            Show
            jglick Jesse Glick added a comment - no matter what Docker version of Tomcat8 I use, I cannot repro Odd. I will try to reproduce using a non-Docker-based installation, and/or using the Jessie image to install Tomcat from update sources. Again, if I can reproduce, the fix is probably straightforward and might help users of even more exotic containers. Is it safe to point Winstone at my existing  /home/jenkins  directory (with my config.xml and other metadata)? Will it pick up the previous settings from Tomcat8 It should. Probably goes without saying, but—create a backup first.
            Hide
            jglick Jesse Glick added a comment -

            using the Jessie image to install Tomcat from update sources

            Finally able to reproduce it this way. Perhaps this was just a bug fixed in the 8.0.x line; 8.0.22, 8.0.38, and 8.0.40 mention potentially related fixes in the changelog. So I wonder why Jessie (even jessie-backports) is shipping a version obsolete since late 2014.

            Show
            jglick Jesse Glick added a comment - using the Jessie image to install Tomcat from update sources Finally able to reproduce it this way. Perhaps this was just a bug fixed in the 8.0.x line; 8.0.22, 8.0.38, and 8.0.40 mention potentially related fixes in the changelog . So I wonder why Jessie (even jessie-backports ) is shipping a version obsolete since late 2014.
            Hide
            danielbeck Daniel Beck added a comment -

            LTS-candidating this in case 2.103ish becomes baseline.

            Show
            danielbeck Daniel Beck added a comment - LTS-candidating this in case 2.103ish becomes baseline.
            Hide
            kmott Kyle Mott added a comment - - edited

            Jesse Glick, I migrated our deployment to native Winstone, and everything seems to have come back (slaves connected, jobs show up along with metadata, etc.).

            However, when I go to trigger a job, I am getting this now in the job log console:

            java.lang.ClassCastException: java.util.HashMap cannot be cast to hudson.scm.SCMRevisionState
            	at org.jenkinsci.plugins.workflow.steps.scm.MultiSCMRevisionState.get(MultiSCMRevisionState.java:56)
            	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:110)
            	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:85)
            	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:75)
            	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
            	at hudson.security.ACL.impersonate(ACL.java:274)
            	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            	at java.lang.Thread.run(Thread.java:748)
            Finished: FAILURE
            

            Is there some way to fix this?

            Show
            kmott Kyle Mott added a comment - - edited Jesse Glick , I migrated our deployment to native Winstone, and everything seems to have come back (slaves connected, jobs show up along with metadata, etc.). However, when I go to trigger a job, I am getting this now in the job log console: java.lang.ClassCastException: java.util.HashMap cannot be cast to hudson.scm.SCMRevisionState at org.jenkinsci.plugins.workflow.steps.scm.MultiSCMRevisionState.get(MultiSCMRevisionState.java:56) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:110) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:85) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:75) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47) at hudson.security.ACL.impersonate(ACL.java:274) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE Is there some way to fix this?
            Hide
            jglick Jesse Glick added a comment -

            Kyle Mott I have not seen that and it looks unrelated. Best to file separately with steps to reproduce, or send to the users’ list.

            Show
            jglick Jesse Glick added a comment - Kyle Mott I have not seen that and it looks unrelated. Best to file separately with steps to reproduce, or send to the users’ list.
            Hide
            kmott Kyle Mott added a comment -

            If I trigger the build again, it goes away, so I'm guessing some transient turd left by Tomcat. Thanks again for all of the help Jesse!

            Show
            kmott Kyle Mott added a comment - If I trigger the build again, it goes away, so I'm guessing some transient turd left by Tomcat. Thanks again for all of the help Jesse!
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/jenkins/security/ClassFilterImpl.java
            http://jenkins-ci.org/commit/jenkins/655be64a1753a3077a81fc0c34573bca74dcf5a0
            Log:
            JENKINS-49147 Tolerate unusual CodeSource.location format from old versions of Tomcat.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/security/ClassFilterImpl.java http://jenkins-ci.org/commit/jenkins/655be64a1753a3077a81fc0c34573bca74dcf5a0 Log: JENKINS-49147 Tolerate unusual CodeSource.location format from old versions of Tomcat.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            core/src/main/java/jenkins/security/ClassFilterImpl.java
            http://jenkins-ci.org/commit/jenkins/cd875f0e5aa8191e414a8b9940b02137d8ffb2f4
            Log:
            Merge pull request #3264 from jglick/Tomcat-JENKINS-49147

            JENKINS-49147 Tolerate unusual CodeSource.location format from old versions of Tomcat

            Compare: https://github.com/jenkinsci/jenkins/compare/dbc75125c565...cd875f0e5aa8

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/jenkins/security/ClassFilterImpl.java http://jenkins-ci.org/commit/jenkins/cd875f0e5aa8191e414a8b9940b02137d8ffb2f4 Log: Merge pull request #3264 from jglick/Tomcat- JENKINS-49147 JENKINS-49147 Tolerate unusual CodeSource.location format from old versions of Tomcat Compare: https://github.com/jenkinsci/jenkins/compare/dbc75125c565...cd875f0e5aa8
            Hide
            danielbeck Daniel Beck added a comment -

            Resolved towards 2.104.

            Show
            danielbeck Daniel Beck added a comment - Resolved towards 2.104.
            Hide
            danielbeck Daniel Beck added a comment -

            This is in 2.104 so realistic LTS baselines will include this (and we're aiming for 2.107).

            Show
            danielbeck Daniel Beck added a comment - This is in 2.104 so realistic LTS baselines will include this (and we're aiming for 2.107).

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                kmott Kyle Mott
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: