Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45963

Node monitoring fails when FreeBSD node connected

    Details

    • Type: Improvement
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: monitoring-plugin
    • Labels:
      None
    • Environment:
      Jenkins 2.46.3
      monitoring-plugin 1.67.0
    • Similar Issues:

      Description

      Monitoring plugin fails to collect data from nodes whenever a FreeBSD agent is connected to Jenkins. When the node is disconnected or disabled, plugin is again able to collect data from nodes.

       

      Jenkins log has below warning:

      Aug 03, 2017 7:35:03 AM net.bull.javamelody.JavaLogger warn
      WARNING: exception while collecting data
      java.util.concurrent.ExecutionException: java.lang.InternalError: errno: 2 error: Unable to open /proc/curproc/map
      at hudson.remoting.Channel$2.adapt(Channel.java:862)
      at hudson.remoting.Channel$2.adapt(Channel.java:857)
      at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59)
      at net.bull.javamelody.RemoteCallHelper.collectDataByNodeName(RemoteCallHelper.java:172)
      at net.bull.javamelody.RemoteCallHelper.collectJavaInformationsListByName(RemoteCallHelper.java:182)
      at net.bull.javamelody.NodesCollector.collectWithoutErrorsNow(NodesCollector.java:153)
      at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doMonitoring(HudsonMonitoringFilter.java:160)
      at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:107)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132)
      at org.jenkinsci.plugins.modernstatus.ModernStatusFilter.doFilter(ModernStatusFilter.java:52)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      at org.eclipse.jetty.server.Server.handle(Server.java:499)
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
      at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
      at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.InternalError: errno: 2 error: Unable to open /proc/curproc/map
      at sun.management.OperatingSystemImpl.getCommittedVirtualMemorySize(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at net.bull.javamelody.MemoryInformations.getFromOperatingSystem(MemoryInformations.java:212)
      at net.bull.javamelody.MemoryInformations.getLongFromOperatingSystem(MemoryInformations.java:199)
      at net.bull.javamelody.MemoryInformations.buildMemoryDetails(MemoryInformations.java:156)
      at net.bull.javamelody.MemoryInformations.<init>(MemoryInformations.java:104)
      at net.bull.javamelody.JavaInformations.<init>(JavaInformations.java:137)
      at net.bull.javamelody.RemoteCallHelper$1.call(RemoteCallHelper.java:46)
      at net.bull.javamelody.RemoteCallHelper$1.call(RemoteCallHelper.java:38)
      at net.bull.javamelody.RemoteCallHelper$DelegatingTask.call(RemoteCallHelper.java:132)
      at hudson.remoting.UserRequest.perform(UserRequest.java:153)
      at hudson.remoting.UserRequest.perform(UserRequest.java:50)
      at hudson.remoting.Request$2.run(Request.java:336)
      at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)
      at ......remote call to freebsd_agent(Native Method)
      at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
      at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
      at hudson.remoting.Channel$2.adapt(Channel.java:860)
      ... 57 more
      

        Attachments

          Activity

          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: evernat
          Path:
          src/main/java/net/bull/javamelody/NodesController.java
          src/main/java/net/bull/javamelody/RemoteCallHelper.java
          http://jenkins-ci.org/commit/monitoring-plugin/6de6ddfb66d11fd59da18beb0002e25e1c9e51ff
          Log:
          JENKINS-45963 (FreeBSD): if collect fails for one node, continue with others

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: evernat Path: src/main/java/net/bull/javamelody/NodesController.java src/main/java/net/bull/javamelody/RemoteCallHelper.java http://jenkins-ci.org/commit/monitoring-plugin/6de6ddfb66d11fd59da18beb0002e25e1c9e51ff Log: JENKINS-45963 (FreeBSD): if collect fails for one node, continue with others
          Hide
          evernat evernat added a comment -

          Fixed by ignoring the monitoring data of this failing FreeBSD agent. You can download a new build at: https://javamelody.ci.cloudbees.com/job/jenkins%20plugin/ws/target/monitoring.hpi

          It has not been tried to ignore only the failling call to "getCommittedVirtualMemorySize" since FreeBSD (with OpenJDK) is known to fail for some javax.management calls. See this example.

          Note that according to google, "Unable to open /proc/curproc/map" can be fixed by mounting proc fs. See https://www.freebsd.org/doc/en/articles/linux-users/procfs.html

          Show
          evernat evernat added a comment - Fixed by ignoring the monitoring data of this failing FreeBSD agent. You can download a new build at: https://javamelody.ci.cloudbees.com/job/jenkins%20plugin/ws/target/monitoring.hpi It has not been tried to ignore only the failling call to "getCommittedVirtualMemorySize" since FreeBSD (with OpenJDK) is known to fail for some javax.management calls. See this example . Note that according to google, "Unable to open /proc/curproc/map" can be fixed by mounting proc fs. See https://www.freebsd.org/doc/en/articles/linux-users/procfs.html

            People

            • Assignee:
              evernat evernat
              Reporter:
              xmiklis Mika Karjalainen
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: