Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6257

Perforce plugin cannot determine host name of slave

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: p4-plugin
    • Labels:
      None
    • Environment:
      Hudson 1.354, Perforce plugin 1.0.28
    • Similar Issues:

      Description

      I get many warnings in log saying something like:

      ... PerforceSCM getEffectiveClientName
      WARNING: Could not get hostname for slave <named-slave>

      It obviously knows the name of the slave, but where's it supposed to get the hostname from? Is this not a Perforce issue, but a more general slave issue?
      It happens for both Linux and Windows based slaves.

        Attachments

          Issue Links

            Activity

            Hide
            rpetti Rob Petti added a comment -

            It gets the hostname from the slave itself by running java.net.InetAddress.getLocalHost().getHostName(). That warning indicates that for some reason the hostname is set to something that cannot actually be resolved using DNS. I don't know if there's another way to do it, but I'm all ears. The solution currently is to make sure the slave can resolve itself properly.

            Show
            rpetti Rob Petti added a comment - It gets the hostname from the slave itself by running java.net.InetAddress.getLocalHost().getHostName(). That warning indicates that for some reason the hostname is set to something that cannot actually be resolved using DNS. I don't know if there's another way to do it, but I'm all ears. The solution currently is to make sure the slave can resolve itself properly.
            Hide
            rpetti Rob Petti added a comment -

            By the way, it does this for automatic generation of the perforce workspace name. It used to use '-hostname' as a suffix, but now it's configurable, and the hostname can simply be used as a token replacement. See the "Client name format for slaves" option.

            Show
            rpetti Rob Petti added a comment - By the way, it does this for automatic generation of the perforce workspace name. It used to use '-hostname' as a suffix, but now it's configurable, and the hostname can simply be used as a token replacement. See the "Client name format for slaves" option.
            Hide
            torbent torbent added a comment -

            Of course the slaves have proper hostnames and know what they are
            The hostname command in shells give the correct name, as expected.
            One twist here could be that:

            • The Linux slave in question has 3 eth interfaces. The correct one, and the only one with a cable in, is eth3. eth0, in particular, has an IP address but no name associated.
            • The Windows slave (running ipconfig /all) shows two VMWare interfaces before the real one, and only the real one has a name associated with its address.
              So perhaps it's the resolver inside Java that has a problem? That sounds a bit "fantastic", so it's probably something a lot closer to Hudson.

            http://www.exampledepot.com/egs/java.net/Local.html suggests that your method does not work with virtual machines.
            http://cboard.cprogramming.com/tech-board/111960-how-do-i-get-hostname-java.html agrees with your method, but also discusses multiple network interfaces a bit.

            I don't work with Java, but how do you determine the "Could not get hostname" status? As far as I can read, the function getHostName() will return either the name or a string with the formatted IP address. Neither will look like a failure?

            Show
            torbent torbent added a comment - Of course the slaves have proper hostnames and know what they are The hostname command in shells give the correct name, as expected. One twist here could be that: The Linux slave in question has 3 eth interfaces. The correct one, and the only one with a cable in, is eth3. eth0, in particular, has an IP address but no name associated. The Windows slave (running ipconfig /all ) shows two VMWare interfaces before the real one, and only the real one has a name associated with its address. So perhaps it's the resolver inside Java that has a problem? That sounds a bit "fantastic", so it's probably something a lot closer to Hudson. http://www.exampledepot.com/egs/java.net/Local.html suggests that your method does not work with virtual machines. http://cboard.cprogramming.com/tech-board/111960-how-do-i-get-hostname-java.html agrees with your method, but also discusses multiple network interfaces a bit. I don't work with Java, but how do you determine the "Could not get hostname" status? As far as I can read, the function getHostName() will return either the name or a string with the formatted IP address. Neither will look like a failure?
            Hide
            rpetti Rob Petti added a comment -

            I think it's actually the getLocalHost() that fails. Possibly because of the multiple interfaces... I have heard of cases in which this method doesn't work, so it's not too outlandish.
            Does the following resolve the host properly if you run it on the linux slave?

            nslookup `hostname`
            
            Show
            rpetti Rob Petti added a comment - I think it's actually the getLocalHost() that fails. Possibly because of the multiple interfaces... I have heard of cases in which this method doesn't work, so it's not too outlandish. Does the following resolve the host properly if you run it on the linux slave? nslookup `hostname`
            Hide
            torbent torbent added a comment -

            Yes, nslookup `hostname` does the right thing.
            I tried making a small Java programme with the code that you use. On the Linux slave, getLocalHost().toString() returns a string of the form "hostname/ip-address" where ip-address is actually the wrong one (from eth0, not eth3). However, the hostname is the correct hostname. And getLocalHost().getHostName() returns the correct hostname of the slave.
            On my Windows slave, both hostname and ip-address are the correct ones.

            So I suspect that there's a problem either in your logic to determine whether you got a hostname or not, or something doesn't work properly when called from within Hudson?

            Show
            torbent torbent added a comment - Yes, nslookup `hostname` does the right thing. I tried making a small Java programme with the code that you use. On the Linux slave, getLocalHost().toString() returns a string of the form "hostname/ip-address" where ip-address is actually the wrong one (from eth0, not eth3). However, the hostname is the correct hostname. And getLocalHost().getHostName() returns the correct hostname of the slave. On my Windows slave, both hostname and ip-address are the correct ones. So I suspect that there's a problem either in your logic to determine whether you got a hostname or not, or something doesn't work properly when called from within Hudson?
            Hide
            rpetti Rob Petti added a comment -

            Gah, looks like it's not the getLocalHost().getHostname() that's failing, but rather the remote call. I'm adding some code to log the full stack trace. I imagine that this could happen if the slave is having connection issues, or the workspace has been deleted.

            Show
            rpetti Rob Petti added a comment - Gah, looks like it's not the getLocalHost().getHostname() that's failing, but rather the remote call. I'm adding some code to log the full stack trace. I imagine that this could happen if the slave is having connection issues, or the workspace has been deleted.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in hudson
            User: : rpetti
            Path:
            trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/PerforceSCM.java
            http://jenkins-ci.org/commit/30125
            Log:
            JENKINS-6257 adding some more logging in the event that the remote call for determining the slave hostname fails

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : rpetti Path: trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/PerforceSCM.java http://jenkins-ci.org/commit/30125 Log: JENKINS-6257 adding some more logging in the event that the remote call for determining the slave hostname fails
            Hide
            torbent torbent added a comment -

            I don't have the resources to build and test a plugin - I can only handle releases.
            Is there any other way I can assist? Like running commands in the scripting console or whatever its name is? (There's some way of executing Groovy scripts on the slaves, methinks)

            I doubt it's connection issues; the Windows slaves are actually running on the same computer as the Master, and the Linux slave is on the same local network. Mind you, they tend to crash once (randomly) after a Hudson upgrade ...

            Deleted workspaces? Maybe, but not likely. It's a consistent warning in the log. Occurs on every build of certain jobs (haven't tried to determine what they have in common).

            Show
            torbent torbent added a comment - I don't have the resources to build and test a plugin - I can only handle releases. Is there any other way I can assist? Like running commands in the scripting console or whatever its name is? (There's some way of executing Groovy scripts on the slaves, methinks) I doubt it's connection issues; the Windows slaves are actually running on the same computer as the Master, and the Linux slave is on the same local network. Mind you, they tend to crash once (randomly) after a Hudson upgrade ... Deleted workspaces? Maybe, but not likely. It's a consistent warning in the log. Occurs on every build of certain jobs (haven't tried to determine what they have in common).
            Hide
            rpetti Rob Petti added a comment -

            Yeah, I'm not asking you to build it yourself or anything, I'm just adding the logging so we can take a closer look at it after the next release.

            Show
            rpetti Rob Petti added a comment - Yeah, I'm not asking you to build it yourself or anything, I'm just adding the logging so we can take a closer look at it after the next release.
            Hide
            torbent torbent added a comment -

            I just tried the "Script Console" feature on our slaves. Very neat
            I used the same code as I did in my comment of 16/Apr/10 04:26 AM (hmmm, wonder if I can force the proper timezone?), pasted it into the script console, ran it, and it showed exactly the same output. So that code can figure out the hostname. Nice to know
            On the Linux slave I get a fully qualified hostname, on the Windows slave just the hostname.

            Of course this doesn't reveal anything about remote calls failing, but at least it demonstrates that we can get the hostname from within Hudson.

            Show
            torbent torbent added a comment - I just tried the "Script Console" feature on our slaves. Very neat I used the same code as I did in my comment of 16/Apr/10 04:26 AM (hmmm, wonder if I can force the proper timezone?), pasted it into the script console, ran it, and it showed exactly the same output. So that code can figure out the hostname. Nice to know On the Linux slave I get a fully qualified hostname, on the Windows slave just the hostname. Of course this doesn't reveal anything about remote calls failing, but at least it demonstrates that we can get the hostname from within Hudson.
            Hide
            rpetti Rob Petti added a comment -

            I figured as much. The plugin release system is messed up right now because of the Oracle takeover, so the release I pushed out on Monday still hasn't made it through yet. Kohsuke is still working on it last I checked.

            Show
            rpetti Rob Petti added a comment - I figured as much. The plugin release system is messed up right now because of the Oracle takeover, so the release I pushed out on Monday still hasn't made it through yet. Kohsuke is still working on it last I checked.
            Hide
            torbent torbent added a comment -

            I've now upgraded to 1.0.29, and ... well ... there's more in the logs, but I don't like it:

            10-05-2010 11:43:36 hudson.plugins.perforce.PerforceSCM getEffectiveClientName
            INFO: java.lang.NullPointerException
            	at hudson.plugins.perforce.PerforceSCM.getEffectiveClientName(PerforceSCM.java:972)
            	at hudson.plugins.perforce.PerforceSCM.getEffectiveClientName(PerforceSCM.java:954)
            	at hudson.plugins.perforce.PerforceSCM.buildEnvVars(PerforceSCM.java:368)
            	at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:698)
            	at hudson.model.FreeStyleBuild$RunnerImpl.decideWorkspace(FreeStyleBuild.java:55)
            	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:405)
            	at hudson.model.Run.run(Run.java:1241)
            	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
            	at hudson.model.ResourceController.execute(ResourceController.java:88)
            	at hudson.model.Executor.run(Executor.java:122)
            
            Show
            torbent torbent added a comment - I've now upgraded to 1.0.29, and ... well ... there's more in the logs, but I don't like it: 10-05-2010 11:43:36 hudson.plugins.perforce.PerforceSCM getEffectiveClientName INFO: java.lang.NullPointerException at hudson.plugins.perforce.PerforceSCM.getEffectiveClientName(PerforceSCM.java:972) at hudson.plugins.perforce.PerforceSCM.getEffectiveClientName(PerforceSCM.java:954) at hudson.plugins.perforce.PerforceSCM.buildEnvVars(PerforceSCM.java:368) at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:698) at hudson.model.FreeStyleBuild$RunnerImpl.decideWorkspace(FreeStyleBuild.java:55) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:405) at hudson.model.Run.run(Run.java:1241) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:122)
            Hide
            rpetti Rob Petti added a comment -

            I like it! That gives a very clear indication about what the cause of the issue is. Somehow, you are running builds without a hudson workspace, so when it tries to perform the remote call to get the hostname, it fails. I have no idea how this is possible, but it might be related to your other problem in JENKINS-6254. If your workspaces are getting confused in hudson, then it's possible that they may even become null sometimes, which would cause this error.

            Show
            rpetti Rob Petti added a comment - I like it! That gives a very clear indication about what the cause of the issue is. Somehow, you are running builds without a hudson workspace, so when it tries to perform the remote call to get the hostname, it fails. I have no idea how this is possible, but it might be related to your other problem in JENKINS-6254 . If your workspaces are getting confused in hudson, then it's possible that they may even become null sometimes, which would cause this error.
            Hide
            torbent torbent added a comment -

            Running builds without a workspace? Sounds strange ...
            The symptoms appear, as far as I can tell, when the Perforce plugin does its work at the beginning of a build. Is it in any way possible that the workspace could somehow end up not being created yet, as opposed to just being the wrong one?
            Other than that, I agree that it sounds somewhat consistent with "my other problem", where it clearly uses (or is asked to use) a wrong workspace.

            Show
            torbent torbent added a comment - Running builds without a workspace? Sounds strange ... The symptoms appear, as far as I can tell, when the Perforce plugin does its work at the beginning of a build. Is it in any way possible that the workspace could somehow end up not being created yet, as opposed to just being the wrong one? Other than that, I agree that it sounds somewhat consistent with "my other problem", where it clearly uses (or is asked to use) a wrong workspace.
            Hide
            torbent torbent added a comment -

            I'm still getting these errors for every build that takes place on a slave. I find it hard to believe that Hudson chooses a bad workspace every time? There are rather many p4 commands being logged on the Hudson console now - I assume logging has been increased with one of the more recent releases of the plugin?

            Show
            torbent torbent added a comment - I'm still getting these errors for every build that takes place on a slave. I find it hard to believe that Hudson chooses a bad workspace every time? There are rather many p4 commands being logged on the Hudson console now - I assume logging has been increased with one of the more recent releases of the plugin?
            Hide
            rpetti Rob Petti added a comment -

            Must be some corner case that you are hitting in your configuration, unfortunately I have no idea what it could be. For whatever reason, hudson can't run the command in your job's workspace directory. If you aren't using ${hostname} in your slave client name, then it's safe for you to ignore it.

            Show
            rpetti Rob Petti added a comment - Must be some corner case that you are hitting in your configuration, unfortunately I have no idea what it could be. For whatever reason, hudson can't run the command in your job's workspace directory. If you aren't using ${hostname} in your slave client name, then it's safe for you to ignore it.
            Hide
            torbent torbent added a comment -

            I believe most of my jobs use the default (blank) which I'm not really sure what means ... Use the given clientname?
            Anyway, all jobs appear to be working fine - it just bugs me that there's this error message showing up as often ...

            Show
            torbent torbent added a comment - I believe most of my jobs use the default (blank) which I'm not really sure what means ... Use the given clientname? Anyway, all jobs appear to be working fine - it just bugs me that there's this error message showing up as often ...
            Hide
            rpetti Rob Petti added a comment -

            That's odd. It should never be blank. If it is somehow set to be blank, then it will set it to one of the defaults based on the old settings if they are available.

            One of my remote slaves that I control in another office seems to be exhibiting the same behavior... but I can't be sure if that's because it has a bogus hostname or not. In any case, I might change the plugin so it won't attempt to get the hostname if it doesn't have to.

            Show
            rpetti Rob Petti added a comment - That's odd. It should never be blank. If it is somehow set to be blank, then it will set it to one of the defaults based on the old settings if they are available. One of my remote slaves that I control in another office seems to be exhibiting the same behavior... but I can't be sure if that's because it has a bogus hostname or not. In any case, I might change the plugin so it won't attempt to get the hostname if it doesn't have to.
            Hide
            torbent torbent added a comment -

            I'm talking about the client name in the "Advanced" section, which can e.g. be ${clientname}. I believe most of my jobs were created before that field was created, so they have a blank value for it. That is documented to mean something, which of course I cannot remember right now. Is this (part of) the problem?
            The recommended default, AFAIR, is ${clientname}-${hash} which apparently does not stop the master from polling.

            Show
            torbent torbent added a comment - I'm talking about the client name in the "Advanced" section, which can e.g. be ${clientname}. I believe most of my jobs were created before that field was created, so they have a blank value for it. That is documented to mean something, which of course I cannot remember right now. Is this (part of) the problem? The recommended default, AFAIR, is ${clientname}-${hash} which apparently does not stop the master from polling.
            Hide
            rpetti Rob Petti added a comment -

            I'm also referring to that client name. Any job that was created before this option was added will either use the default of ${basename}${hash}, ${basename} or ${basename}${hostname} based on what the old settings were. It should NEVER appear to be blank... This value is used to determine which client to use for the slaves, and the hostname of the slave can be used in the substitution. While I don't believe this is causing your problem directly, if the field appears to be blank then I suspect there may be something wrong elsewhere in your setup.

            Show
            rpetti Rob Petti added a comment - I'm also referring to that client name. Any job that was created before this option was added will either use the default of ${basename} ${hash}, ${basename} or ${basename} ${hostname} based on what the old settings were. It should NEVER appear to be blank... This value is used to determine which client to use for the slaves, and the hostname of the slave can be used in the substitution. While I don't believe this is causing your problem directly, if the field appears to be blank then I suspect there may be something wrong elsewhere in your setup.
            Hide
            torbent torbent added a comment -

            Alright, I'll check my settings. If I find blanks, what kind of "something wrong" should I be looking for?

            If I submit a new job with a blank value, what will it become? ${basename}, I presume?

            Show
            torbent torbent added a comment - Alright, I'll check my settings. If I find blanks, what kind of "something wrong" should I be looking for? If I submit a new job with a blank value, what will it become? ${basename}, I presume?
            Hide
            rpetti Rob Petti added a comment -

            Corrupt class files in the plugin installation directory, or even hudson itself are the only things I can think of. There could possibly be something amiss in the code, but it's pretty simple: If it's blank or null, return a sensible default instead. In any case, this problem itself is probably not causing the issue, but may be indicative of a deeper problem with your installation.

            As I mentioned, the default depends on the settings the job had before you upgraded. ${basename} if "Don't Rename Client" was checked, ${basename}${hostname} if "Use Old Client Name" was checked, and ${basename}${hash} otherwise.

            Show
            rpetti Rob Petti added a comment - Corrupt class files in the plugin installation directory, or even hudson itself are the only things I can think of. There could possibly be something amiss in the code, but it's pretty simple: If it's blank or null, return a sensible default instead. In any case, this problem itself is probably not causing the issue, but may be indicative of a deeper problem with your installation. As I mentioned, the default depends on the settings the job had before you upgraded. ${basename} if "Don't Rename Client" was checked, ${basename} ${hostname} if "Use Old Client Name" was checked, and ${basename} ${hash} otherwise.
            Hide
            torbent torbent added a comment -

            I've just upgraded to 1.360 and p4 plugin 1.1.0. As a part of the upgrade I deleted all .hpi files and the extracted subdirectories in 'plugin', as well as the entire 'war' directory of hudson itself. Should be a clean good installation now.
            But I still have the same problem. In fact, it appears to be logged for ALL polling activity (noticed that a couple of days ago, so not just 1.1.0) on slaves.
            I checked all jobs for one of the slaves that it cannot determine the name of. All jobs had ${basename} as their slave client name.
            I'll attach a piece of the log.

            Show
            torbent torbent added a comment - I've just upgraded to 1.360 and p4 plugin 1.1.0. As a part of the upgrade I deleted all .hpi files and the extracted subdirectories in 'plugin', as well as the entire 'war' directory of hudson itself. Should be a clean good installation now. But I still have the same problem. In fact, it appears to be logged for ALL polling activity (noticed that a couple of days ago, so not just 1.1.0) on slaves. I checked all jobs for one of the slaves that it cannot determine the name of. All jobs had ${basename } as their slave client name. I'll attach a piece of the log.
            Hide
            torbent torbent added a comment -

            Roughly two minutes of log (383 lines), showing Perforce problems with determining slave hostnames when polling. Actual slave names have been deleted; it happened on 2 or 3 different slaves. I believe a couple of builds were triggered in this period - they all work fine wrt Perforce.

            Show
            torbent torbent added a comment - Roughly two minutes of log (383 lines), showing Perforce problems with determining slave hostnames when polling. Actual slave names have been deleted; it happened on 2 or 3 different slaves. I believe a couple of builds were triggered in this period - they all work fine wrt Perforce.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in hudson
            User: : rpetti
            Path:
            trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/PerforceSCM.java
            http://jenkins-ci.org/commit/31632
            Log:
            [FIXED JENKINS-6257] create a workspace filepath for running GetHostname if none is provided during polling

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : rpetti Path: trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/PerforceSCM.java http://jenkins-ci.org/commit/31632 Log: [FIXED JENKINS-6257] create a workspace filepath for running GetHostname if none is provided during polling

              People

              • Assignee:
                rpetti Rob Petti
                Reporter:
                torbent torbent
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: