Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45410

SSH Slave Agent fails occasionally with java 8 / 9 mismatch between master and slave

    Details

    • Similar Issues:

      Description

      We run a number of jenkins builds on slave machines. Approximately 1 in 20 builds will complete successfully but then fails with the following stack trace - apparently just about as it's ready to report that success. We eventually tracked it down to using java-8-openjdk on the master and java-9-openjdk on the slave. When the slave was switch to java-8-openjdk as well, then the problem disappeared.

      Some apparently similar bugs which look different upon closer inspection:

      https://issues.jenkins-ci.org/browse/JENKINS-21341

      https://issues.jenkins-ci.org/browse/JENKINS-22492

       

      Possible solutions:

      1) Fix this particular incompatibility with java-9-openjdk

      2) Perhaps visibly prompt users to check when different java defaults on master and slave machine

       

      Stack trace for 1 in 20 failure:
      + exit 0
      FATAL: Remote call on ca-g2-01 failed
      java.lang.NoClassDefFoundError: Could not initialize class hudson.util.ProcessTree$UnixReflection
      {{ at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:647)}}
      {{ at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:668)}}
      {{ at hudson.util.ProcessTree$Unix.killAll(ProcessTree.java:589)}}
      {{ at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:996)}}
      {{ at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:987)}}
      {{ at hudson.remoting.UserRequest.perform(UserRequest.java:153)}}
      {{ at hudson.remoting.UserRequest.perform(UserRequest.java:50)}}
      {{ at hudson.remoting.Request$2.run(Request.java:336)}}
      {{ at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)}}
      {{ at java.util.concurrent.FutureTask.run(FutureTask.java:266)}}
      {{ at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1158)}}
      {{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632)}}
      {{ at java.lang.Thread.run(Thread.java:804)}}
      {{ at ......remote call to ca-g2-01(Native Method)}}
      {{ at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)}}
      {{ at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)}}
      {{ at hudson.remoting.Channel.call(Channel.java:830)}}
      Caused: java.io.IOException: Remote call on ca-g2-01 failed
      {{ at hudson.remoting.Channel.call(Channel.java:838)}}
      {{ at hudson.Launcher$RemoteLauncher.kill(Launcher.java:984)}}
      {{ at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:496)}}
      {{ at hudson.model.Run.execute(Run.java:1735)}}
      {{ at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)}}
      {{ at hudson.model.ResourceController.execute(ResourceController.java:97)}}
      {{ at hudson.model.Executor.run(Executor.java:405)}}
      Recording test results
      Finished: FAILURE

        Attachments

          Issue Links

            Activity

            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Java 9 is not supported in Jenkins

            Show
            oleg_nenashev Oleg Nenashev added a comment - Java 9 is not supported in Jenkins
            Hide
            brunobowden Bruno Bowden added a comment -

            Just a more visible warning of the Java 9 compatibility would be enough for now. Bonus points for pointing to documentation on how to switch Java versions.

            Show
            brunobowden Bruno Bowden added a comment - Just a more visible warning of the Java 9 compatibility would be enough for now. Bonus points for pointing to documentation on how to switch Java versions.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Baptiste Mathus I believe this use-case is addressed by the Version Column plugin, right?

            Maybe we can move some bits of it closer to the core, e.g. making the plugin recommended

             

            Show
            oleg_nenashev Oleg Nenashev added a comment - Baptiste Mathus I believe this use-case is addressed by the Version Column plugin, right? Maybe we can move some bits of it closer to the core, e.g. making the plugin recommended  
            Hide
            batmat Baptiste Mathus added a comment - - edited

            IIUC, yes. The plugin (https://wiki.jenkins.io/display/JENKINS/VersionColumn+Plugin) will allow you to automatically disconnect an agent whose Java or remoting version does not satisfy the requirements (can be configured the recommended way, I.e. major Java version matching, or paranoid = exact same version on both sides).

            Maybe we can move some bits of it closer to the core, e.g. making the plugin recommended

            Indeed, might be an idea. I'd need to check the current behaviors to refresh myself with the features (some preexisting and some I added).

            Show
            batmat Baptiste Mathus added a comment - - edited IIUC, yes. The plugin ( https://wiki.jenkins.io/display/JENKINS/VersionColumn+Plugin ) will allow you to automatically disconnect an agent whose Java or remoting version does not satisfy the requirements (can be configured the recommended way, I.e. major Java version matching, or paranoid = exact same version on both sides). Maybe we can move some bits of it closer to the core, e.g. making the plugin recommended Indeed, might be an idea. I'd need to check the current behaviors to refresh myself with the features (some preexisting and some I added).
            Hide
            batmat Baptiste Mathus added a comment -

            I believe we cannot act on something here and this is indeed a use case for versioncolumn-plugin.

            Please follow JENKINS-54741 and install this plugin for checking/aligning versions betwen master and agents.

            We're eager to hear any feedback or concrete idea on how to surface this better though .

            Thanks!

            Show
            batmat Baptiste Mathus added a comment - I believe we cannot act on something here and this is indeed a use case for versioncolumn-plugin. Please follow JENKINS-54741 and install this plugin for checking/aligning versions betwen master and agents. We're eager to hear any feedback or concrete idea on how to surface this better though . Thanks!

              People

              • Assignee:
                Unassigned
                Reporter:
                brunobowden Bruno Bowden
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: