Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48754

Jenkins Slaves do not connect after update from version 2.95 to version 2.99

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Blocker
    • Resolution: Duplicate
    • Component/s: core, remoting
    • Labels:
      None
    • Environment:
    • Similar Issues:

      Description

      After having installed Jenkins Version 2.99, none of my slaves did connect to the server anymore. Even a restart of the java script on the slave did not help.

      Workaround: Update Remoting on the agent side to 3.15

        Attachments

          Issue Links

            Activity

            Hide
            blaoueille Bruno Laoueillé added a comment -

            Hi,

            I've exactly the same problem.

             

            Agent log :

            janv. 02, 2018 3:05:49 PM hudson.remoting.jnlp.Main createEngine
            INFO: Setting up slave: Windows Slave
            janv. 02, 2018 3:05:49 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
            INFO: Using D:\Jenkins\remoting as a remoting work directory
            Both error and output logs will be printed to D:\Jenkins\remoting
            janv. 02, 2018 3:05:49 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
            INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
            janv. 02, 2018 3:06:00 PM hudson.remoting.jnlp.GuiListener$1 run
            INFO: Connecting to ciserver:8081 (retrying:2)
            java.io.IOException: Failed to connect to ciserver:8081
            {{ at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:242)}}
            {{ at hudson.remoting.Engine.connect(Engine.java:686)}}
            {{ at hudson.remoting.Engine.innerRun(Engine.java:547)}}
            {{ at hudson.remoting.Engine.run(Engine.java:469)}}
            Caused by: java.net.ConnectException: Connection refused: connect
            {{ at sun.nio.ch.Net.connect0(Native Method)}}
            {{ at sun.nio.ch.Net.connect(Unknown Source)}}
            {{ at sun.nio.ch.Net.connect(Unknown Source)}}
            {{ at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)}}
            {{ at java.nio.channels.SocketChannel.open(Unknown Source)}}
            {{ at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:203)}}
            {{ ... 3 more}}

             

            And in master log nothing appears...

             

            Regards,

            Bruno.

            Show
            blaoueille Bruno Laoueillé added a comment - Hi, I've exactly the same problem.   Agent log : janv. 02, 2018 3:05:49 PM hudson.remoting.jnlp.Main createEngine INFO: Setting up slave: Windows Slave janv. 02, 2018 3:05:49 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using D:\Jenkins\remoting as a remoting work directory Both error and output logs will be printed to D:\Jenkins\remoting janv. 02, 2018 3:05:49 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping] janv. 02, 2018 3:06:00 PM hudson.remoting.jnlp.GuiListener$1 run INFO: Connecting to ciserver:8081 (retrying:2) java.io.IOException: Failed to connect to ciserver:8081 {{ at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:242)}} {{ at hudson.remoting.Engine.connect(Engine.java:686)}} {{ at hudson.remoting.Engine.innerRun(Engine.java:547)}} {{ at hudson.remoting.Engine.run(Engine.java:469)}} Caused by: java.net.ConnectException: Connection refused: connect {{ at sun.nio.ch.Net.connect0(Native Method)}} {{ at sun.nio.ch.Net.connect(Unknown Source)}} {{ at sun.nio.ch.Net.connect(Unknown Source)}} {{ at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)}} {{ at java.nio.channels.SocketChannel.open(Unknown Source)}} {{ at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:203)}} {{ ... 3 more}}   And in master log nothing appears...   Regards, Bruno.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Which agent version do you use?

            Show
            oleg_nenashev Oleg Nenashev added a comment - Which agent version do you use?
            Hide
            blaoueille Bruno Laoueillé added a comment -

            For me : I've uninstalled my Windows Service agent and click again on "Launch agent from browser". I guess it's the latest with this method ?

            In the agent.jar manifest file it's written 3.15 for the version...

            Show
            blaoueille Bruno Laoueillé added a comment - For me : I've uninstalled my Windows Service agent and click again on "Launch agent from browser". I guess it's the latest with this method ? In the agent.jar manifest file it's written 3.15 for the version...
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            My first guess is that Remoting receiver thread is not active due to whatever reason. Or maybe not, the HTTPs receiver should be working

            Please provide...
            1) Jenkins log, starting from the master startup
            2) Thread dump for the master

            If you could just generate a bundle using https://wiki.jenkins.io/display/JENKINS/Support+Core+Plugin, it would be great

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited My first guess is that Remoting receiver thread is not active due to whatever reason. Or maybe not, the HTTPs receiver should be working Please provide... 1) Jenkins log, starting from the master startup 2) Thread dump for the master If you could just generate a bundle using https://wiki.jenkins.io/display/JENKINS/Support+Core+Plugin , it would be great
            Hide
            blaoueille Bruno Laoueillé added a comment -

            Hi,

             

            I hope it's what you asked :

            support_2018-01-02_17.45.19.zip

            master-logs.txt

            Show
            blaoueille Bruno Laoueillé added a comment - Hi,   I hope it's what you asked : support_2018-01-02_17.45.19.zip master-logs.txt
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Some investigation while I am waiting for the info...

            1) No changes in JnlpAgentEndpoint since 2.95
            2) No changes in Engine#connect()
            3) AFAICT https://github.com/jenkinsci/remoting/pull/210 is the most plausible source of regressions, but I do not see the bogus logic so far.

            Please also provide agent startup arguments

            Show
            oleg_nenashev Oleg Nenashev added a comment - Some investigation while I am waiting for the info... 1) No changes in JnlpAgentEndpoint since 2.95 2) No changes in Engine#connect() 3) AFAICT https://github.com/jenkinsci/remoting/pull/210 is the most plausible source of regressions, but I do not see the bogus logic so far. Please also provide agent startup arguments
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            A full support bundle would help though. In particular I am interested whether the agent uses HTTPS and what are the certificate check flags

            Show
            oleg_nenashev Oleg Nenashev added a comment - A full support bundle would help though. In particular I am interested whether the agent uses HTTPS and what are the certificate check flags
            Hide
            blaoueille Bruno Laoueillé added a comment -
            Show
            blaoueille Bruno Laoueillé added a comment - Yes here :  support_2018-01-02_18.01.52.zip
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Yes, for JNLP agent I really need startup arguments.

            Other info:

            • 8081 is a TCP agent listener port
            • TCP Agent Listener thread is running
            • There is a proxy configured on the instance. Likely it's not related since there are other community votes
            "TCP agent listener port=8081" id=53 (0x35) state=RUNNABLE cpu=0% (running in native)
                at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
                at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
                at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
                at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:157)
            
            Show
            oleg_nenashev Oleg Nenashev added a comment - Yes, for JNLP agent I really need startup arguments. Other info: 8081 is a TCP agent listener port TCP Agent Listener thread is running There is a proxy configured on the instance. Likely it's not related since there are other community votes "TCP agent listener port=8081" id=53 (0x35) state=RUNNABLE cpu=0% (running in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:157)
            Hide
            blaoueille Bruno Laoueillé added a comment -

            OK, How can I retrieve agent startup arguments ?

             

            On startup page for the agent start through the browser I have

            java -jar agent.jar -jnlpUrl http://myserver:10004/computer/Windows%20Slave/slave-agent.jnlp -secret **************************************************************** -workDir "D:\Jenkins"

             

            (Note: I've replaced the real server name by myserver, and changed the secret code).

             

            Show
            blaoueille Bruno Laoueillé added a comment - OK, How can I retrieve agent startup arguments ?   On startup page for the agent start through the browser I have java -jar agent.jar -jnlpUrl http://myserver:10004/computer/Windows%20Slave/slave-agent.jnlp -secret **************************************************************** -workDir "D:\Jenkins"   (Note: I've replaced the real server name by myserver, and changed the secret code).  
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Yes, it's what I needed. Now it's a time to reproduce it

            Show
            oleg_nenashev Oleg Nenashev added a comment - Yes, it's what I needed. Now it's a time to reproduce it
            Hide
            panajev Goffredo Marocchi added a comment - - edited

            Seeing something very similar.

            In a job:

            First time build. Skipping changelog.
             No emails were triggered.
             FATAL: command execution failed
             Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from mdbuildserver01217.gamesys.corp/10.194.194.147:60268
             at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1696)
             at hudson.remoting.UserResponse.retrieve(UserRequest.java:313)
             at hudson.remoting.Channel.call(Channel.java:909)
             at hudson.Launcher$RemoteLauncher.launch(Launcher.java:1053)
             at hudson.Launcher$ProcStarter.start(Launcher.java:450)
             at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
             at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
             at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
             at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
             at hudson.model.Build$BuildExecution.build(Build.java:206)
             at hudson.model.Build$BuildExecution.doRun(Build.java:163)
             at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
             at hudson.model.Run.execute(Run.java:1727)
             at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
             at hudson.model.ResourceController.execute(ResourceController.java:97)
             at hudson.model.Executor.run(Executor.java:429)
             java.lang.NoSuchMethodError: hudson.Launcher$RemoteLaunchCallable.getOpenChannelOrFail()Lhudson/remoting/Channel;
             at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1292)
             at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1260)
             at hudson.remoting.UserRequest.perform(UserRequest.java:207)
             at hudson.remoting.UserRequest.perform(UserRequest.java:53)
             at hudson.remoting.Request$2.run(Request.java:358)
             at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
             at java.util.concurrent.FutureTask.run(FutureTask.java:266)
             at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
             at hudson.remoting.Engine$1$1.run(Engine.java:98)
             at java.lang.Thread.run(Thread.java:748)
             Caused: java.io.IOException: Remote call on JNLP4-connect connection from mdbuildserver01217.gamesys.corp/10.194.194.147:60268 failed
             at hudson.remoting.Channel.call(Channel.java:917)
             at hudson.Launcher$RemoteLauncher.launch(Launcher.java:1053)
             at hudson.Launcher$ProcStarter.start(Launcher.java:450)
             at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
             at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
             at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
             at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
             at hudson.model.Build$BuildExecution.build(Build.java:206)
             at hudson.model.Build$BuildExecution.doRun(Build.java:163)
             at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
             at hudson.model.Run.execute(Run.java:1727)
             at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
             at hudson.model.ResourceController.execute(ResourceController.java:97)
             at hudson.model.Executor.run(Executor.java:429)
             Build step 'Execute shell' marked build as failure

             

            Starting the agent up:

            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main createEngine
            INFO: Setting up slave: <SlaveName>
            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener <init>
            INFO: Jenkins agent is running in headless mode.
            Jan 02, 2018 7:55:08 PM hudson.remoting.Engine startEngine
            WARNING: No Working Directory. Using the legacy JAR Cache location: /Users/mdbuildserver/.jenkins/cache/jars
            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status
            INFO: Locating server among [https://<jenkins URL>/]
            Jan 02, 2018 7:55:08 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
            INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status
            INFO: Agent discovery successful
              Agent address: <jenkins URL>/
              Agent port:    42650
              Identity:      fb:f3:24:67:05:be:94:06:49:2a:aa:64:c2:83:32:c7
            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status
            INFO: Handshaking
            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status
            INFO: Connecting to nps-jenkins.gamesys.co.uk:42650
            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status
            INFO: Trying protocol: JNLP4-connect
            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status
            INFO: Remote identity confirmed: fb:f3:24:67:05:be:94:06:49:2a:aa:64:c2:83:32:c7
            Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status
            INFO: Connected
            Jan 02, 2018 7:55:08 PM hudson.remoting.UserRequest perform
            WARNING: LinkageError while performing UserRequest:hudson.slaves.ChannelPinger$SetUpRemotePing@2905
            java.lang.NoSuchMethodError: hudson.slaves.ChannelPinger$SetUpRemotePing.getOpenChannelOrFail()Lhudson/remoting/Channel;
                at hudson.slaves.ChannelPinger$SetUpRemotePing.call(ChannelPinger.java:137)
                at hudson.slaves.ChannelPinger$SetUpRemotePing.call(ChannelPinger.java:121)
                at hudson.remoting.UserRequest.perform(UserRequest.java:207)
                at hudson.remoting.UserRequest.perform(UserRequest.java:53)
                at hudson.remoting.Request$2.run(Request.java:358)
                at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at hudson.remoting.Engine$1$1.run(Engine.java:98)
                at java.lang.Thread.run(Thread.java:748)
            
            Jan 02, 2018 7:55:08 PM hudson.remoting.UserRequest perform
            WARNING: LinkageError while performing UserRequest:jenkins.slaves.StandardOutputSwapper$ChannelSwapper@3b16348f
            java.lang.NoSuchMethodError: jenkins.slaves.StandardOutputSwapper$ChannelSwapper.getOpenChannelOrFail()Lhudson/remoting/Channel;
                at jenkins.slaves.StandardOutputSwapper$ChannelSwapper.call(StandardOutputSwapper.java:42)
                at jenkins.slaves.StandardOutputSwapper$ChannelSwapper.call(StandardOutputSwapper.java:39)
                at hudson.remoting.UserRequest.perform(UserRequest.java:207)
                at hudson.remoting.UserRequest.perform(UserRequest.java:53)
                at hudson.remoting.Request$2.run(Request.java:358)
                at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at hudson.remoting.Engine$1$1.run(Engine.java:98)
                at java.lang.Thread.run(Thread.java:748)
            
            Jan 02, 2018 7:55:09 PM com.youdevise.hudson.slavestatus.SlaveListener call
            INFO: Slave-status listener starting
            Jan 02, 2018 7:55:09 PM com.youdevise.hudson.slavestatus.SocketHTTPListener waitForConnection
            INFO: Slave-status listener ready on port 3141
            

             

            Agent is launched like so:

            java -jar agent.jar -jnlpUrl https://<Jenkins URL>/computer/<AgentName>/slave-agent.jnlp -secret <Agent Secret>
            

             

            Show
            panajev Goffredo Marocchi added a comment - - edited Seeing something very similar. In a job: First time build. Skipping changelog. No emails were triggered. FATAL: command execution failed Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from mdbuildserver01217.gamesys.corp/10.194.194.147:60268 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1696) at hudson.remoting.UserResponse.retrieve(UserRequest.java:313) at hudson.remoting.Channel.call(Channel.java:909) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:1053) at hudson.Launcher$ProcStarter.start(Launcher.java:450) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.Build$BuildExecution.build(Build.java:206) at hudson.model.Build$BuildExecution.doRun(Build.java:163) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504) at hudson.model.Run.execute(Run.java:1727) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) java.lang.NoSuchMethodError: hudson.Launcher$RemoteLaunchCallable.getOpenChannelOrFail()Lhudson/remoting/Channel; at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1292) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1260) at hudson.remoting.UserRequest.perform(UserRequest.java:207) at hudson.remoting.UserRequest.perform(UserRequest.java:53) at hudson.remoting.Request$2.run(Request.java:358) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1$1.run(Engine.java:98) at java.lang. Thread .run( Thread .java:748) Caused: java.io.IOException: Remote call on JNLP4-connect connection from mdbuildserver01217.gamesys.corp/10.194.194.147:60268 failed at hudson.remoting.Channel.call(Channel.java:917) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:1053) at hudson.Launcher$ProcStarter.start(Launcher.java:450) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.Build$BuildExecution.build(Build.java:206) at hudson.model.Build$BuildExecution.doRun(Build.java:163) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504) at hudson.model.Run.execute(Run.java:1727) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Build step 'Execute shell' marked build as failure   Starting the agent up: Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main createEngine INFO: Setting up slave: <SlaveName> Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener <init> INFO: Jenkins agent is running in headless mode. Jan 02, 2018 7:55:08 PM hudson.remoting.Engine startEngine WARNING: No Working Directory. Using the legacy JAR Cache location: /Users/mdbuildserver/.jenkins/cache/jars Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [https: //<jenkins URL>/] Jan 02, 2018 7:55:08 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping] Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Agent discovery successful   Agent address: <jenkins URL>/   Agent port:    42650   Identity:      fb:f3:24:67:05:be:94:06:49:2a:aa:64:c2:83:32:c7 Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to nps-jenkins.gamesys.co.uk:42650 Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Trying protocol: JNLP4-connect Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Remote identity confirmed: fb:f3:24:67:05:be:94:06:49:2a:aa:64:c2:83:32:c7 Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connected Jan 02, 2018 7:55:08 PM hudson.remoting.UserRequest perform WARNING: LinkageError while performing UserRequest:hudson.slaves.ChannelPinger$SetUpRemotePing@2905 java.lang.NoSuchMethodError: hudson.slaves.ChannelPinger$SetUpRemotePing.getOpenChannelOrFail()Lhudson/remoting/Channel;     at hudson.slaves.ChannelPinger$SetUpRemotePing.call(ChannelPinger.java:137)     at hudson.slaves.ChannelPinger$SetUpRemotePing.call(ChannelPinger.java:121)     at hudson.remoting.UserRequest.perform(UserRequest.java:207)     at hudson.remoting.UserRequest.perform(UserRequest.java:53)     at hudson.remoting.Request$2.run(Request.java:358)     at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)     at java.util.concurrent.FutureTask.run(FutureTask.java:266)     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)     at hudson.remoting.Engine$1$1.run(Engine.java:98)     at java.lang. Thread .run( Thread .java:748) Jan 02, 2018 7:55:08 PM hudson.remoting.UserRequest perform WARNING: LinkageError while performing UserRequest:jenkins.slaves.StandardOutputSwapper$ChannelSwapper@3b16348f java.lang.NoSuchMethodError: jenkins.slaves.StandardOutputSwapper$ChannelSwapper.getOpenChannelOrFail()Lhudson/remoting/Channel;     at jenkins.slaves.StandardOutputSwapper$ChannelSwapper.call(StandardOutputSwapper.java:42)     at jenkins.slaves.StandardOutputSwapper$ChannelSwapper.call(StandardOutputSwapper.java:39)     at hudson.remoting.UserRequest.perform(UserRequest.java:207)     at hudson.remoting.UserRequest.perform(UserRequest.java:53)     at hudson.remoting.Request$2.run(Request.java:358)     at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)     at java.util.concurrent.FutureTask.run(FutureTask.java:266)     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)     at hudson.remoting.Engine$1$1.run(Engine.java:98)     at java.lang. Thread .run( Thread .java:748) Jan 02, 2018 7:55:09 PM com.youdevise.hudson.slavestatus.SlaveListener call INFO: Slave-status listener starting Jan 02, 2018 7:55:09 PM com.youdevise.hudson.slavestatus.SocketHTTPListener waitForConnection INFO: Slave-status listener ready on port 3141   Agent is launched like so: java -jar agent.jar -jnlpUrl https: //<Jenkins URL>/computer/<AgentName>/slave-agent.jnlp -secret <Agent Secret>  
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Goffredo Marocchi could you please create a separate ticket for now? I am not 100% sure the issues are same, but in the case of your stacktraces I know what to fix

            Show
            oleg_nenashev Oleg Nenashev added a comment - Goffredo Marocchi could you please create a separate ticket for now? I am not 100% sure the issues are same, but in the case of your stacktraces I know what to fix
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Regarding the issue from Goffredo Marocchi, updating agent to Remoting 3.15 is a workaround, working on a fix

            Show
            oleg_nenashev Oleg Nenashev added a comment - Regarding the issue from Goffredo Marocchi , updating agent to Remoting 3.15 is a workaround, working on a fix
            Hide
            panajev Goffredo Marocchi added a comment -

            Thanks Oleg Nenashev if you want me to create a separate ticket I will do so right away and link it here.

            Show
            panajev Goffredo Marocchi added a comment - Thanks Oleg Nenashev if you want me to create a separate ticket I will do so right away and link it here.
            Show
            panajev Goffredo Marocchi added a comment - Done: https://issues.jenkins-ci.org/browse/JENKINS-48761
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            There is a chance that my patch for JENKINS-48761 solves this issue as well

            Show
            oleg_nenashev Oleg Nenashev added a comment - There is a chance that my patch for JENKINS-48761 solves this issue as well
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Most likely it is fixed in Jenkins 2.100, the release is in progress. Retrospective thread: https://groups.google.com/forum/#!topic/jenkinsci-dev/-VQ5YU0yYhg

            Show
            oleg_nenashev Oleg Nenashev added a comment - Most likely it is fixed in Jenkins 2.100, the release is in progress. Retrospective thread: https://groups.google.com/forum/#!topic/jenkinsci-dev/-VQ5YU0yYhg
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            content/_data/changelogs/weekly.yml
            http://jenkins-ci.org/commit/jenkins.io/ac42d83bf51e56e826991839a5471eccdd767543
            Log:
            Changelog: noting JENKINS-48761/JENKINS-48754 in Jenkins 2.100.

            The PR also adds warning banners to 2.98/2.99

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: content/_data/changelogs/weekly.yml http://jenkins-ci.org/commit/jenkins.io/ac42d83bf51e56e826991839a5471eccdd767543 Log: Changelog: noting JENKINS-48761 / JENKINS-48754 in Jenkins 2.100. The PR also adds warning banners to 2.98/2.99
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: R. Tyler Croy
            Path:
            content/_data/changelogs/weekly.yml
            http://jenkins-ci.org/commit/jenkins.io/99fe6d097fd1cc1f6797ad7b816c98dec3261a37
            Log:
            Merge pull request #1305 from oleg-nenashev/changelog/2.100

            Changelog: noting JENKINS-48761/JENKINS-48754 in Jenkins 2.100.

            Compare: https://github.com/jenkins-infra/jenkins.io/compare/62362762ff55...99fe6d097fd1

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: content/_data/changelogs/weekly.yml http://jenkins-ci.org/commit/jenkins.io/99fe6d097fd1cc1f6797ad7b816c98dec3261a37 Log: Merge pull request #1305 from oleg-nenashev/changelog/2.100 Changelog: noting JENKINS-48761 / JENKINS-48754 in Jenkins 2.100. Compare: https://github.com/jenkins-infra/jenkins.io/compare/62362762ff55...99fe6d097fd1
            Hide
            blaoueille Bruno Laoueillé added a comment -

            Hi,

            Thank you for your actions. I have updated to the 2.100 version, restarted Jenkins, then go to my slave and refreshed my browser cache, finally I launch again the agent from the browser and unfortunally the result was identical. It doesn't work. I also tried to download the agent.jar and launch by command line, we have exactly the same thing. Same stack trace on the agent side...

            I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ?

             

            However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed). 

             

            Thank you in advance for your help. 

             

            Regards,

            Bruno.

            Show
            blaoueille Bruno Laoueillé added a comment - Hi, Thank you for your actions. I have updated to the 2.100 version, restarted Jenkins, then go to my slave and refreshed my browser cache, finally I launch again the agent from the browser and unfortunally the result was identical. It doesn't work. I also tried to download the agent.jar and launch by command line, we have exactly the same thing. Same stack trace on the agent side... I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ?   However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed).    Thank you in advance for your help.    Regards, Bruno.
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            Thanks for the quick reply, digging in.

            > I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ?

            It's not. The version should be 3.15, maybe it's just a typo. But yes, all changes in 2.100 were on the master side, Remoting has not been changed.

            > However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed).

            I am not sure the version rollback from Jenkins UI operates correctly in Docker.
            My recommendation would be to replace a container.

            Thanks for the pointers regarding Docker though. Do you use the standard Jenkins image or your custom one?

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited Thanks for the quick reply, digging in. > I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ? It's not. The version should be 3.15, maybe it's just a typo. But yes, all changes in 2.100 were on the master side, Remoting has not been changed. > However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed). I am not sure the version rollback from Jenkins UI operates correctly in Docker. My recommendation would be to replace a container. Thanks for the pointers regarding Docker though. Do you use the standard Jenkins image or your custom one?
            Hide
            blaoueille Bruno Laoueillé added a comment - - edited

            For the version 3.5 it's my bad, sorry, it's the 3.15 (bad typing).

             

            EDIT: For the Docker image, I dont know sorry, it's not me who managed it. I can ask to another person.

            Show
            blaoueille Bruno Laoueillé added a comment - - edited For the version 3.5 it's my bad, sorry, it's the 3.15 (bad typing).   EDIT: For the Docker image, I dont know sorry, it's not me who managed it. I can ask to another person.
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            OK, Docker... So far I can say the following:

            • Master runs on the 8082 port according to the bundle, but the web UI suggests the 10004 port. It is a whatever port mapping or external proxy, I'd guess
            • Agent fails to connect to TcpAgentListener port, the connection is refused. Probably it's just because the port is not exposed
            • TcpAgentListener runs with port set to 0. It means the it will be taking a random port every restart
            • Since you run the master in Docker, I am not sure how it is supposed to work. You would have to expose each port on the Docker host. I doubt you do that.

            Please check which port is exposed up for TcpAgentListener, and make sure the same port is configured in the security global config. Since you run in Docker, I would definitely recommend setting port flags so that the port won't be changeable from UI:

            -Djenkins.model.Jenkins.slaveAgentPort=${YOUR_EXPOSED_PORT} -Djenkins.model.Jenkins.slaveAgentPortEnforce=true
            

            Example for Docker

            "TCP agent listener port=0" id=86 (0x56) state=RUNNABLE cpu=0% (running in native)
                at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
                at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
                at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
                at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:157)
            
            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited OK, Docker... So far I can say the following: Master runs on the 8082 port according to the bundle, but the web UI suggests the 10004 port. It is a whatever port mapping or external proxy, I'd guess Agent fails to connect to TcpAgentListener port, the connection is refused. Probably it's just because the port is not exposed TcpAgentListener runs with port set to 0 . It means the it will be taking a random port every restart Since you run the master in Docker, I am not sure how it is supposed to work. You would have to expose each port on the Docker host. I doubt you do that. Please check which port is exposed up for TcpAgentListener, and make sure the same port is configured in the security global config. Since you run in Docker, I would definitely recommend setting port flags so that the port won't be changeable from UI: -Djenkins.model.Jenkins.slaveAgentPort=${YOUR_EXPOSED_PORT} -Djenkins.model.Jenkins.slaveAgentPortEnforce=true Example for Docker "TCP agent listener port=0" id=86 (0x56) state=RUNNABLE cpu=0% (running in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:157)
            Hide
            blaoueille Bruno Laoueillé added a comment -

            Yes it's ok for that, moreover the configuration has not changed on that. We have put the 8081 port for Tcp agents with only JNLP/4 and it is accessible and configured on our Docker. (It was working well before updates).

            Show
            blaoueille Bruno Laoueillé added a comment - Yes it's ok for that, moreover the configuration has not changed on that. We have put the 8081 port for Tcp agents with only JNLP/4 and it is accessible and configured on our Docker. (It was working well before updates).
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Back to this story...

            > We have put the 8081 port

            How did you do that? The threaddump definitely suggests you're running with a random port

            Show
            oleg_nenashev Oleg Nenashev added a comment - Back to this story... > We have put the 8081 port How did you do that? The threaddump definitely suggests you're running with a random port
            Hide
            blaoueille Bruno Laoueillé added a comment -

            Hi,

            Sorry for late answer. But you are right, the configuration of the connection through tuneling was not set for the slave. I forgot it after my second installation (it was set only in the global security part).

            So after correcting this part the new version has worked perfectly.

            Thank you for your support.

            Regards,

            Bruno.

            Show
            blaoueille Bruno Laoueillé added a comment - Hi, Sorry for late answer. But you are right, the configuration of the connection through tuneling was not set for the slave. I forgot it after my second installation (it was set only in the global security part). So after correcting this part the new version has worked perfectly. Thank you for your support. Regards, Bruno.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Thanks for the reply! So this is not a defect. I will close it as a duplicate of JENKINS-48761, because the most of the voters/watchers followed the summary of the issue && timing was very coincidental

            Show
            oleg_nenashev Oleg Nenashev added a comment - Thanks for the reply! So this is not a defect. I will close it as a duplicate of JENKINS-48761 , because the most of the voters/watchers followed the summary of the issue && timing was very coincidental

              People

              • Assignee:
                oleg_nenashev Oleg Nenashev
                Reporter:
                michaelmotteler Michael M.
              • Votes:
                4 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: