Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3080

scp plugin doesn't work for multi configuration project

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: scp-plugin
    • Labels:
      None
    • Environment:
      Platform: All, OS: All
    • Similar Issues:

      Description

      I am trying to use scp plugin under multi configuration project without much
      success. First time when I start the build, it never finish uploading even after
      5mins and the project is just blinking. I decided to kill the project.

      During my 2nd attempt at least I got this error in my log. BTW single
      configuration seem to be working just fine.

      started
      Building remotely on XXXXXXXXXXXXXXXX.XXXXXX.XXX
      [windows] $ bash -xe C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\hudson8584819837934197381.sh
      + '[' x '!=' x ']'
      + which scp
      /usr/bin/scp
      + echo windows
      windows
      + echo windows
      + echo 2009-02-18_22-15-23
      + cat windows.txt
      windows
      2009-02-18_22-15-23
      [SCP] Connecting to XXXXXXXXXXXXXXXX.XXXXXX.XXX
      [SCP] remote folderPath test,local file:windows.txt
      ERROR: Failed to upload files
      4:
      at com.jcraft.jsch.ChannelSftp.stat(Unknown Source)
      at be.certipost.hudson.plugin.SCPSite.upload(SCPSite.java:178)
      at
      be.certipost.hudson.plugin.SCPRepositoryPublisher.perform(SCPRepositoryPublisher.java:121)
      at
      hudson.model.AbstractBuild$AbstractRunner.performAllBuildStep(AbstractBuild.java:365)
      at
      hudson.model.AbstractBuild$AbstractRunner.performAllBuildStep(AbstractBuild.java:353)
      at hudson.model.Build$RunnerImpl.post2(Build.java:169)
      at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:338)
      at hudson.model.Run.run(Run.java:914)
      at hudson.model.Build.run(Build.java:112)
      at hudson.model.ResourceController.execute(ResourceController.java:93)
      at hudson.model.Executor.run(Executor.java:119)
      finished: UNSTABLE

        Attachments

          Activity

          Hide
          ramazanyich2 ramazanyich2 added a comment -

          When you did a global configuration for SCP site, white did you provide as
          rootRepositoryPath parameter ?
          does directory which you enter there exists on SCP server ?

          Show
          ramazanyich2 ramazanyich2 added a comment - When you did a global configuration for SCP site, white did you provide as rootRepositoryPath parameter ? does directory which you enter there exists on SCP server ?
          Hide
          vijayj vijayj added a comment -

          > When you did a global configuration for SCP site,

          Prior to start of the build I had completed the Global configuration.

          > white did you provide as rootRepositoryPath parameter ?

          Host: hostname.domainname.com
          Port: 22
          Root Repository Path: /home/builds/
          Username : root
          Password : XXXXXXXXXXXXX
          keyfile :

          > does directory which you enter there exists on SCP server ?

          Yes.

          These are my observations.

          o Remember there is no issues with single configuration. I am using the same
          global configuration. All the files are getting copied into the same directory.
          For Eg:
          ws/make/Makefile
          ws/src/foo.java

          Both the files are getting copied under $ROOTDIR/$DEST/*

          o Multi configuration project is also trying to copy everything into the same
          location. I think this could be the issue. For a multi configuration project,
          somehow they all need to be copied into a different directory. There may be a
          race condition here. You may have to find out the label or hostname, then copy
          them under $ROOTDIR/$DEST/$LABEL/*

          o Also it would be so nice if you can provide option for maintaining the
          directory structure instead of dumping everything into the same location. Some
          like scp –r would normally preserve the directory structure.

          Thanks a lot for your quick response. This would really save lot of workaround
          that I have been thinking about doing for this issue.

          Show
          vijayj vijayj added a comment - > When you did a global configuration for SCP site, Prior to start of the build I had completed the Global configuration. > white did you provide as rootRepositoryPath parameter ? Host: hostname.domainname.com Port: 22 Root Repository Path: /home/builds/ Username : root Password : XXXXXXXXXXXXX keyfile : > does directory which you enter there exists on SCP server ? Yes. These are my observations. o Remember there is no issues with single configuration. I am using the same global configuration. All the files are getting copied into the same directory. For Eg: ws/make/Makefile ws/src/foo.java Both the files are getting copied under $ROOTDIR/$DEST/* o Multi configuration project is also trying to copy everything into the same location. I think this could be the issue. For a multi configuration project, somehow they all need to be copied into a different directory. There may be a race condition here. You may have to find out the label or hostname, then copy them under $ROOTDIR/$DEST/$LABEL/* o Also it would be so nice if you can provide option for maintaining the directory structure instead of dumping everything into the same location. Some like scp –r would normally preserve the directory structure. Thanks a lot for your quick response. This would really save lot of workaround that I have been thinking about doing for this issue.
          Hide
          vijayj vijayj added a comment -

          BTW I just created multi configuration project with only one configuration. I am
          using $BUILD_NUMBER/$JOB_NAME as my DEST.

          Even a simple txt file transfer is struggling.
          <LOG>
          ....
          .....
          + /usr/bin/wget -q
          http://hostname:8080/hudson//job/matrix/label=linux/33/consoleText
          + mv consoleText consoleText.txt
          [SCP] Connecting to vjayaraman-dl.datadomain.com
          [SCP] remote folderPath 33/matrix/label=linux,local file:consoleText.txt
          [SCP] Trying to create /home/builds/33
          [SCP] Trying to create /home/builds/33/matrix
          [SCP] Trying to create /home/builds/33/matrix/label=linux

          </LOG>

          Build is simply running for the last 5mins without much action. Also I have
          checked the scp server and it created an empty file but there is no contents.

          Show
          vijayj vijayj added a comment - BTW I just created multi configuration project with only one configuration. I am using $BUILD_NUMBER/$JOB_NAME as my DEST. Even a simple txt file transfer is struggling. <LOG> .... ..... + /usr/bin/wget -q http://hostname:8080/hudson//job/matrix/label=linux/33/consoleText + mv consoleText consoleText.txt [SCP] Connecting to vjayaraman-dl.datadomain.com [SCP] remote folderPath 33/matrix/label=linux,local file:consoleText.txt [SCP] Trying to create /home/builds/33 [SCP] Trying to create /home/builds/33/matrix [SCP] Trying to create /home/builds/33/matrix/label=linux </LOG> Build is simply running for the last 5mins without much action. Also I have checked the scp server and it created an empty file but there is no contents.
          Hide
          vijayj vijayj added a comment -

          Ha ha even single configuration project is having problem.

          If the project is getting executed from the master node then there is no
          issues with transfer. If i choose to tie the single configuration project to
          some specific slave node then it is having the same issue as multi configuration.

          Show
          vijayj vijayj added a comment - Ha ha even single configuration project is having problem. If the project is getting executed from the master node then there is no issues with transfer. If i choose to tie the single configuration project to some specific slave node then it is having the same issue as multi configuration.
          Hide
          ramazanyich2 ramazanyich2 added a comment -

          I remember I had once issue with hanging builds on slave due to non synchronized
          clock on master and slave. Even difference in some seconds could be an issue.
          I suggest that you check that clocks of master and slaves are synchronized using
          some NTP server.

          Show
          ramazanyich2 ramazanyich2 added a comment - I remember I had once issue with hanging builds on slave due to non synchronized clock on master and slave. Even difference in some seconds could be an issue. I suggest that you check that clocks of master and slaves are synchronized using some NTP server.
          Hide
          vijayj vijayj added a comment -

          I have three slave machines (solaris 9, linux [fedora core 4] and windows 2003)
          attached with my master hudson (running at fedora core4). Time isn't synced
          for windows and solaris but both master and slave linux machine time is synced.

          I can confirm that single configuration project running on my slave linux
          machine isn't working either. Not sure solaris and windows being out of sync has
          any issues here, but I would be surprised.

          Show
          vijayj vijayj added a comment - I have three slave machines (solaris 9, linux [fedora core 4] and windows 2003) attached with my master hudson (running at fedora core4). Time isn't synced for windows and solaris but both master and slave linux machine time is synced. I can confirm that single configuration project running on my slave linux machine isn't working either. Not sure solaris and windows being out of sync has any issues here, but I would be surprised.
          Hide
          vijayj vijayj added a comment -

          All three of my slave machines are in sync with master now. SCP plugin doesn't
          work when executed through one of the slave machine(s) either through single
          configuration or multi configuration.

          Show
          vijayj vijayj added a comment - All three of my slave machines are in sync with master now. SCP plugin doesn't work when executed through one of the slave machine(s) either through single configuration or multi configuration.

            People

            • Assignee:
              ramazanyich2 ramazanyich2
              Reporter:
              vijayj vijayj
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated: