Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13032

Updating cvs plugin to version 2.0 causes "java.lang.RuntimeException: CVS authentication failure while running rlog command"

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Blocker
    • Resolution: Fixed
    • Component/s: cvs-plugin
    • Labels:
    • Environment:
      CentOS release 5.4 64-bit, Tomcat 6.0.20, Jenkins 1.454
    • Similar Issues:

      Description

      On version 1.6 everything was ok. Now every job using cvs fails.

      After updating cvs plugin from version 1.6 to 2.0 I get:

      FATAL: CVS authentication failure while running rlog command
      java.lang.RuntimeException: CVS authentication failure while running rlog command
      at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:529)
      at hudson.scm.CVSSCM.calculateChangeLog(CVSSCM.java:409)
      at hudson.scm.CVSSCM.checkout(CVSSCM.java:829)
      at hudson.model.AbstractProject.checkout(AbstractProject.java:1195)
      at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:579)
      at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:468)
      at hudson.model.Run.run(Run.java:1408)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:238)
      Caused by: org.netbeans.lib.cvsclient.connection.AuthenticationException: Timeout, no response from server.
      at org.netbeans.lib.cvsclient.Client.ensureConnection(Client.java:418)
      at org.netbeans.lib.cvsclient.command.log.RlogCommand.execute(RlogCommand.java:357)
      at org.netbeans.lib.cvsclient.Client.executeCommand(Client.java:710)
      at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:521)
      ... 9 more

        Attachments

          Activity

          jacekw jacekw created issue -
          Hide
          jacekw jacekw added a comment -

          It's strange behaviour - series of "cvs checkout" and "cvs rlog" is succesfully made and this exception is thrown.

          Job's log:

          ...
          cvs checkout: Updating [dir name]
          cvs checkout: Updating [dir name]
          ...
          cvs rlog: Logging [dir/file name]
          cvs rlog: Logging [dir/file name]
          ...
          cvs rlog: Logging [dir/file name]
          FATAL: CVS authentication failure while running rlog command
          java.lang.RuntimeException: CVS authentication failure while running rlog command
          at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:529)
          at hudson.scm.CVSSCM.calculateChangeLog(CVSSCM.java:409)
          at hudson.scm.CVSSCM.checkout(CVSSCM.java:829)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1195)
          at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:579)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:468)
          at hudson.model.Run.run(Run.java:1408)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:238)
          Caused by: org.netbeans.lib.cvsclient.connection.AuthenticationException: Timeout, no response from server.
          at org.netbeans.lib.cvsclient.Client.ensureConnection(Client.java:418)
          at org.netbeans.lib.cvsclient.command.log.RlogCommand.execute(RlogCommand.java:357)
          at org.netbeans.lib.cvsclient.Client.executeCommand(Client.java:710)
          at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:521)
          ... 9 more

          Show
          jacekw jacekw added a comment - It's strange behaviour - series of "cvs checkout" and "cvs rlog" is succesfully made and this exception is thrown. Job's log: ... cvs checkout: Updating [dir name] cvs checkout: Updating [dir name] ... cvs rlog: Logging [dir/file name] cvs rlog: Logging [dir/file name] ... cvs rlog: Logging [dir/file name] FATAL: CVS authentication failure while running rlog command java.lang.RuntimeException: CVS authentication failure while running rlog command at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:529) at hudson.scm.CVSSCM.calculateChangeLog(CVSSCM.java:409) at hudson.scm.CVSSCM.checkout(CVSSCM.java:829) at hudson.model.AbstractProject.checkout(AbstractProject.java:1195) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:579) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:468) at hudson.model.Run.run(Run.java:1408) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:238) Caused by: org.netbeans.lib.cvsclient.connection.AuthenticationException: Timeout, no response from server. at org.netbeans.lib.cvsclient.Client.ensureConnection(Client.java:418) at org.netbeans.lib.cvsclient.command.log.RlogCommand.execute(RlogCommand.java:357) at org.netbeans.lib.cvsclient.Client.executeCommand(Client.java:710) at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:521) ... 9 more
          Hide
          mc1arke Michael Clarke added a comment -

          The message further down the stack trace shows the autentication failure as being caused by the server connection timing out. This therefore isn't something that can be fixed with code changes so I'm closing this issue.

          Show
          mc1arke Michael Clarke added a comment - The message further down the stack trace shows the autentication failure as being caused by the server connection timing out. This therefore isn't something that can be fixed with code changes so I'm closing this issue.
          mc1arke Michael Clarke made changes -
          Field Original Value New Value
          Status Open [ 1 ] Closed [ 6 ]
          Assignee Michael Clarke [ mc1arke ]
          Fix Version/s current [ 10162 ]
          Resolution Not A Defect [ 7 ]
          Hide
          jacekw jacekw added a comment -

          That's not as straightforward as it seems - after cvs plugin downgrade to 1.6 and reverting changes in job's configs everything is working fine. So I'm sure that's not problem of timeouts. Eventually this problem may be caused by cvs version we're using.

          Show
          jacekw jacekw added a comment - That's not as straightforward as it seems - after cvs plugin downgrade to 1.6 and reverting changes in job's configs everything is working fine. So I'm sure that's not problem of timeouts. Eventually this problem may be caused by cvs version we're using.
          Hide
          jroyer Joël Royer added a comment - - edited

          I've got the same problem since version 2.0 of the plugin.
          I've got timeouts randomly. However, the CVS server is available, I can ping it from the jenkins server.

          Windows Server 2003 / Glassfish 3.1.2.2 / Oracle SDK 1.7.0_07 / Jenkins 1.486 / CVS Plugin 2.0

          Here is the log in the console:

          Building in workspace C:\PIC\work\jenkins-work\jobs\OX1_V011703 - etape 02 - TI - CAP\workspace
          Using locally configured password for connection to :pserver:hudson@srvcvs1:/cvsroot
          cvs checkout -P -r OX1_V011703 -d web src/web
          ERROR: CVS Authentication failed: Timeout, no response from server.
          org.netbeans.lib.cvsclient.connection.AuthenticationException: Timeout, no response from server.
          at org.netbeans.lib.cvsclient.Client.ensureConnection(Client.java:418)
          at org.netbeans.lib.cvsclient.command.checkout.CheckoutCommand.execute(CheckoutCommand.java:299)
          at org.netbeans.lib.cvsclient.Client.executeCommand(Client.java:710)
          at hudson.scm.AbstractCvs$1.invoke(AbstractCvs.java:243)
          at hudson.scm.AbstractCvs$1.invoke(AbstractCvs.java:223)
          at hudson.FilePath.act(FilePath.java:851)
          at hudson.FilePath.act(FilePath.java:824)
          at hudson.scm.AbstractCvs.perform(AbstractCvs.java:223)
          at hudson.scm.AbstractCvs.checkout(AbstractCvs.java:189)
          at hudson.scm.CVSSCM.checkout(CVSSCM.java:318)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1308)
          at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:679)
          at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:584)
          at hudson.model.Run.execute(Run.java:1516)
          at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:236)
          ERROR: Cvs task failed
          Archiving artifacts
          Finished: FAILURE

          Show
          jroyer Joël Royer added a comment - - edited I've got the same problem since version 2.0 of the plugin. I've got timeouts randomly. However, the CVS server is available, I can ping it from the jenkins server. Windows Server 2003 / Glassfish 3.1.2.2 / Oracle SDK 1.7.0_07 / Jenkins 1.486 / CVS Plugin 2.0 Here is the log in the console: Building in workspace C:\PIC\work\jenkins-work\jobs\OX1_V011703 - etape 02 - TI - CAP\workspace Using locally configured password for connection to :pserver:hudson@srvcvs1:/cvsroot cvs checkout -P -r OX1_V011703 -d web src/web ERROR: CVS Authentication failed: Timeout, no response from server. org.netbeans.lib.cvsclient.connection.AuthenticationException: Timeout, no response from server. at org.netbeans.lib.cvsclient.Client.ensureConnection(Client.java:418) at org.netbeans.lib.cvsclient.command.checkout.CheckoutCommand.execute(CheckoutCommand.java:299) at org.netbeans.lib.cvsclient.Client.executeCommand(Client.java:710) at hudson.scm.AbstractCvs$1.invoke(AbstractCvs.java:243) at hudson.scm.AbstractCvs$1.invoke(AbstractCvs.java:223) at hudson.FilePath.act(FilePath.java:851) at hudson.FilePath.act(FilePath.java:824) at hudson.scm.AbstractCvs.perform(AbstractCvs.java:223) at hudson.scm.AbstractCvs.checkout(AbstractCvs.java:189) at hudson.scm.CVSSCM.checkout(CVSSCM.java:318) at hudson.model.AbstractProject.checkout(AbstractProject.java:1308) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:679) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:584) at hudson.model.Run.execute(Run.java:1516) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236) ERROR: Cvs task failed Archiving artifacts Finished: FAILURE
          jroyer Joël Royer made changes -
          Resolution Not A Defect [ 7 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          Hide
          mc1arke Michael Clarke added a comment -

          See https://github.com/jenkinsci/cvsclient/blob/master/src/main/java/org/netbeans/lib/cvsclient/Client.java#L361

          This is the method that's causing the error you're seeing whilst it's trying to do it open a connection to the server, with a timeout of 1 minute before failing the connection. If the server fails to accept the connection within this time then the exception you see is thrown. This method doesn't issue any CVS commands, only looks for a network port connection so would be impacted by proxy servers and/or firewall restrictions. Do you have either of these between your Jenkins server and your CVS server?

          Note: Pinging your server only shows the network stack is responding, CVS Server sits ontop of this and could be failing to respond whilst the ping command shows the server as being available.

          Show
          mc1arke Michael Clarke added a comment - See https://github.com/jenkinsci/cvsclient/blob/master/src/main/java/org/netbeans/lib/cvsclient/Client.java#L361 This is the method that's causing the error you're seeing whilst it's trying to do it open a connection to the server, with a timeout of 1 minute before failing the connection. If the server fails to accept the connection within this time then the exception you see is thrown. This method doesn't issue any CVS commands, only looks for a network port connection so would be impacted by proxy servers and/or firewall restrictions. Do you have either of these between your Jenkins server and your CVS server? Note: Pinging your server only shows the network stack is responding, CVS Server sits ontop of this and could be failing to respond whilst the ping command shows the server as being available.
          Hide
          jroyer Joël Royer added a comment -

          Yes, we have a proxy. I setted up Jenkins to use this proxy in the plugin management, but I added the CVS server hostname in exception list ("No proxy host"). Is the plugin using those settings?

          Moreover, we have a strange behavior. When we got this error, if we restart the application server, CVS server is reachable.

          Show
          jroyer Joël Royer added a comment - Yes, we have a proxy. I setted up Jenkins to use this proxy in the plugin management, but I added the CVS server hostname in exception list ("No proxy host"). Is the plugin using those settings? Moreover, we have a strange behavior. When we got this error, if we restart the application server, CVS server is reachable.
          Hide
          mc1arke Michael Clarke added a comment -

          CVS plugin doesn't use Jenkins network settings and will only ever attempt a direct connection unless you set values in the CVSROOT entry (CVSROOT supports proxy settings from what I remember but this would be difficult to create automatically from the Jenkins configuration given exclude rules etc).

          By 'application server', is this the app server hosting Jenkins? At what point does your issue re-occur (is this random or do you see other issues before hand (hanging jobs, failing jobs etc)? Once the issue does re-occur, do your jobs keep failing until restart or do they recover themselves and, if so, is there a pattern to the recovery (a set time between fail and first recovery, killing selected process on the Jenkins server etc)?

          Show
          mc1arke Michael Clarke added a comment - CVS plugin doesn't use Jenkins network settings and will only ever attempt a direct connection unless you set values in the CVSROOT entry (CVSROOT supports proxy settings from what I remember but this would be difficult to create automatically from the Jenkins configuration given exclude rules etc). By 'application server', is this the app server hosting Jenkins? At what point does your issue re-occur (is this random or do you see other issues before hand (hanging jobs, failing jobs etc)? Once the issue does re-occur, do your jobs keep failing until restart or do they recover themselves and, if so, is there a pattern to the recovery (a set time between fail and first recovery, killing selected process on the Jenkins server etc)?
          Hide
          jroyer Joël Royer added a comment -

          I'm using GlassFish App Server v3.1.2.2.
          The issue re-occurs from time to time but when it reoccurs, the only solution we've found for the moment is to stop et start GlassFish. There is no automatic recover.

          Show
          jroyer Joël Royer added a comment - I'm using GlassFish App Server v3.1.2.2. The issue re-occurs from time to time but when it reoccurs, the only solution we've found for the moment is to stop et start GlassFish. There is no automatic recover.
          Hide
          mc1arke Michael Clarke added a comment -

          Next time you have the failure, could you go to /threadDump on your Jenkins host (e.g. http://localhost:8080/threadDump) before and after your restart so I can see if there's any rogue threads hanging around.

          Show
          mc1arke Michael Clarke added a comment - Next time you have the failure, could you go to /threadDump on your Jenkins host (e.g. http://localhost:8080/threadDump ) before and after your restart so I can see if there's any rogue threads hanging around.
          Hide
          jroyer Joël Royer added a comment -

          The bug re-occurs today.
          I joined a zip file which contains thread dumps before and after server reboot.

          Show
          jroyer Joël Royer added a comment - The bug re-occurs today. I joined a zip file which contains thread dumps before and after server reboot.
          Hide
          jroyer Joël Royer added a comment - - edited

          The bug re-occurs today.
          Here is a zip file which contains thread dumps: 1 before restart and 1 after restart

          (See joined file threadDumps.zip)

          Show
          jroyer Joël Royer added a comment - - edited The bug re-occurs today. Here is a zip file which contains thread dumps: 1 before restart and 1 after restart (See joined file threadDumps.zip)
          jroyer Joël Royer made changes -
          Attachment threadDumps.zip [ 22745 ]
          jroyer Joël Royer made changes -
          Attachment threadDumps.zip [ 22748 ]
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Michael Clarke
          Path:
          src/main/java/org/netbeans/lib/cvsclient/connection/PServerConnection.java
          http://jenkins-ci.org/commit/cvsclient/44ec053fafb65c44fc6fd4d88ce1d3e13e0da5d9
          Log:
          JENKINS-13032: Force timeout on initial connection to CVS Server

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michael Clarke Path: src/main/java/org/netbeans/lib/cvsclient/connection/PServerConnection.java http://jenkins-ci.org/commit/cvsclient/44ec053fafb65c44fc6fd4d88ce1d3e13e0da5d9 Log: JENKINS-13032 : Force timeout on initial connection to CVS Server
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Michael Clarke
          Path:
          pom.xml
          http://jenkins-ci.org/commit/cvs-plugin/443d3a1d640432d530cca78f6b777f966f2197a0
          Log:
          [FIXED JENKINS-13032] Upgrade to latest CVS Client with socket timeout

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michael Clarke Path: pom.xml http://jenkins-ci.org/commit/cvs-plugin/443d3a1d640432d530cca78f6b777f966f2197a0 Log: [FIXED JENKINS-13032] Upgrade to latest CVS Client with socket timeout
          scm_issue_link SCM/JIRA link daemon made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 143492 ] JNJira + In-Review [ 190583 ]

            People

            • Assignee:
              mc1arke Michael Clarke
              Reporter:
              jacekw jacekw
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: