Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53924

Jenkins 2.136+ is not restartable with GLIBC 2.5

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: core
    • Labels:
      None
    • Environment:
      Red Hat Enterprise Linux Server release 5.11 (Tikanga)
      java version "1.8.0_181"
      Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
      Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
    • Similar Issues:

      Description

      When Jenkins 2.136 (or newer) starts up, the log lists problem with loading libc library for com.sun.akuma.CLibrary:

      Oct 05, 2018 4:08:34 PM jenkins.diagnosis.HsErrPidList <init>
      WARNING: Failed to list up hs_err_pid files
      java.lang.UnsatisfiedLinkError: /tmp/jna-578977101/jna607378441395487448.tmp: /lib64/libc.so.6: version `GLIBC_2.7' not found (required by /tmp/jna-578977101/jna607378441395487448.tmp)
              at java.lang.ClassLoader$NativeLibrary.load(Native Method)
              at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
              at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
              at java.lang.Runtime.load0(Runtime.java:809)
              at java.lang.System.load(System.java:1086)
              at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:947)
              at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922)
              at com.sun.jna.Native.<clinit>(Native.java:190)
              at com.sun.akuma.CLibrary.<clinit>(CLibrary.java:89)
              at com.sun.akuma.JavaVMArguments.resolvePID(JavaVMArguments.java:128)
              at com.sun.akuma.JavaVMArguments.ofLinux(JavaVMArguments.java:116)
              at com.sun.akuma.JavaVMArguments.of(JavaVMArguments.java:104)
              at com.sun.akuma.JavaVMArguments.current(JavaVMArguments.java:92)
      ...
      

      When visiting /safeRestart URL, there is no option to restart and instead there is a message saying: "Jenkins cannot restart itself as currently configured."

      Attempt to restart from command line fails with:

      $ java -jar jenkins-cli.jar -s <jenkins_url> -ssh -user <user> safe-restart
      
      ERROR: Unexpected exception occurred while performing safe-restart command.
      java.lang.NoClassDefFoundError: Could not initialize class com.sun.akuma.CLibrary
              at com.sun.akuma.JavaVMArguments.resolvePID(JavaVMArguments.java:128)
              at com.sun.akuma.JavaVMArguments.ofLinux(JavaVMArguments.java:116)
              at com.sun.akuma.JavaVMArguments.of(JavaVMArguments.java:104)
              at com.sun.akuma.JavaVMArguments.current(JavaVMArguments.java:92)
              at hudson.lifecycle.UnixLifecycle.<init>(UnixLifecycle.java:55)
              at hudson.lifecycle.Lifecycle.get(Lifecycle.java:94)
              at jenkins.model.Jenkins.getLifecycle(Jenkins.java:2538)
              at org.jenkins.ci.plugins.saferestart.JenkinsUtils.canRestart(JenkinsUtils.java:37)
              at org.jenkins.ci.plugins.saferestart.SafeRestartRootAction.getIconFileName(SafeRestartRootAction.java:47)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
              at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
              at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
              at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
              at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
              at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
              at org.apache.commons.jexl.parser.ASTExpression.value(ASTExpression.java:54)
              at org.apache.commons.jexl.parser.ASTTernaryNode.value(ASTTernaryNode.java:41)
              at org.apache.commons.jexl.parser.ASTExpression.value(ASTExpression.java:54)
              at org.apache.commons.jexl.parser.ASTExpressionExpression.value(ASTExpressionExpression.java:56)
              at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
              at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
              at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
              at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:121)
              at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
              at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
              at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
              at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
              at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
              at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
              at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
              at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
              at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
              at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
              at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
              at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
              at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
              at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
              at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
              at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:147)
              at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
              at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
              at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
              at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
              at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
              at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
              at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
              at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44)
              at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
              at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
              at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
              at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
              at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:112)
              at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:140)
              at org.kohsuke.stapler.IndexViewDispatcher.dispatch(IndexViewDispatcher.java:30)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:734)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:864)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:801)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:864)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668)
              at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
              at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
              at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
              at com.smartcodeltd.jenkinsci.plugin.assetbundler.filters.LessCSS.doFilter(LessCSS.java:47)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
              at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
              at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
              at hudson.plugins.locale.LocaleFilter.doFilter(LocaleFilter.java:42)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
              at hudson.plugins.scm_sync_configuration.extensions.ScmSyncConfigurationFilter$1.call(ScmSyncConfigurationFilter.java:49)
              at hudson.plugins.scm_sync_configuration.extensions.ScmSyncConfigurationFilter$1.call(ScmSyncConfigurationFilter.java:44)
              at hudson.plugins.scm_sync_configuration.ScmSyncConfigurationDataProvider.provideRequestDuring(ScmSyncConfigurationDataProvider.java:106)
              at hudson.plugins.scm_sync_configuration.extensions.ScmSyncConfigurationFilter.doFilter(ScmSyncConfigurationFilter.java:44)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
              at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239)
              at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215)
              at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:88)
              at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:114)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
              at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
              at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:105)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
              at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
              at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
              at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
              at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
              at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
              at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
              at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
              at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
              at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
              at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
              at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
              at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
              at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
              at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
              at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
              at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
              at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
              at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
              at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
              at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
              at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
              at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
              at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
              at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
              at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
              at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
              at org.eclipse.jetty.server.Server.handle(Server.java:531)
              at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
              at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
              at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
              at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
              at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
              at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
              at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
              at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
              at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
              at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
              at winstone.BoundedExecutorService.lambda$scheduleNext$0(BoundedExecutorService.java:80)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      Caused: hudson.lifecycle.RestartNotSupportedException: Failed to obtain the command line arguments of the process
              at hudson.lifecycle.UnixLifecycle.verifyRestartable(UnixLifecycle.java:104)
              at jenkins.model.Jenkins.safeRestart(Jenkins.java:4216)
              at jenkins.model.Jenkins.doSafeRestart(Jenkins.java:4176)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at hudson.cli.declarative.MethodBinder.call(MethodBinder.java:114)
              at hudson.cli.declarative.CLIRegisterer$1.main(CLIRegisterer.java:225)
              at org.jenkinsci.main.modules.sshd.CLICommandAdapter$1.run(CLICommandAdapter.java:37)
              at org.jenkinsci.main.modules.sshd.AsynchronousCommand$1.run(AsynchronousCommand.java:112)
              at java.lang.Thread.run(Thread.java:748)
      

      This is a regression introduced in 2.136. JENKINS-52771 introduced new version of JNA library, which requires GLIBC 2.7 to start. I saw a similar issue JENKINS-10354 was fixed some time ago with recompilation of JNA library against older glibc fixed the issue. Would it be possible to fix it the same way now? RHEL5 extended support is up to 2020.

        Attachments

          Issue Links

            Activity

            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Sames as JENKINS-53832, but for RHEL

            Show
            oleg_nenashev Oleg Nenashev added a comment - Sames as JENKINS-53832 , but for RHEL
            Hide
            oleg_nenashev Oleg Nenashev added a comment -
            Show
            oleg_nenashev Oleg Nenashev added a comment - Documentation patch: https://github.com/jenkins-infra/jenkins.io/pull/1815
            Hide
            igen194 Gordon Daugherty added a comment - - edited

            As info, I had this same issue and found that the cause was that Linux's 'noexec' setting was enabled on the directory that was being used as my JVM's JNI tmpdir. The fix was to use "-Djna.tmpdir=<some path that has execute capability>" in the java command line used to start Jenkins.

             

            Specifically, I added this to my start script:

            #Java's JNA tmpdir cannot be 'noexec' or Jenkins' restart functionality won't work so we're relocating it.

            mkdir -p ${JENKINS_HOME}/tmp
            TMPDIR_ARG=-Djna.tmpdir=${JENKINS_HOME}/tmp

            nohup $JAVA_HOME/bin/java $TMPDIR_ARG -jar jenkins.war <various other arguments omitted as those will differ for you> &

            Show
            igen194 Gordon Daugherty added a comment - - edited As info, I had this same issue and found that the cause was that Linux's 'noexec' setting was enabled on the directory that was being used as my JVM's JNI tmpdir. The fix was to use "-Djna.tmpdir=<some path that has execute capability>" in the java command line used to start Jenkins.   Specifically, I added this to my start script: #Java's JNA tmpdir cannot be 'noexec' or Jenkins' restart functionality won't work so we're relocating it. mkdir -p ${JENKINS_HOME}/tmp TMPDIR_ARG=-Djna.tmpdir=${JENKINS_HOME}/tmp nohup $JAVA_HOME/bin/java $TMPDIR_ARG -jar jenkins.war <various other arguments omitted as those will differ for you> &

              People

              • Assignee:
                Unassigned
                Reporter:
                raspy Krzysztof Malinowski
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: