Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3949

Subversion checkouts creates files for symlinks

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: subversion-plugin
    • Labels:
      None
    • Environment:
      Platform: All, OS: Linux

      Description

      The check out of project containing checked in symlinks creates a workspace
      with files instead of symlinks. E.g., creating a repository containing
      the elements:

      $ svn co file://$PWD/test/trunk/data
      A data/file.link
      A data/file.txt
      Checked out revision 3.
      $ ls -l data
      total 12
      lrwxrwxrwx 1 mrohan mrohan 8 Jun 28 17:55 file.link -> file.txt
      rw-rw-r- 1 mrohan mrohan 16 Jun 28 17:55 file.txt

      Within Hudson, the workspace contains the elements:

      $ ls -l data
      total 16
      rw-rw-r- 1 hudson hudson 13 Jun 28 17:53 file.link
      rw-rw-r- 1 hudson hudson 16 Jun 28 17:53 file.txt
      $ cat data/file.link
      link file.txt

      There is no newline in the file created.

      1. output.1.310.txt
        0.5 kB
        mkrohan
      2. output.1.311.txt
        0.5 kB
        mkrohan
      3. svnkit.w-def.0.log
        12 kB
        mkrohan
      4. svnkit.wo-def.0.log
        12 kB
        mkrohan

        Issue Links

          Activity

          Hide
          abarbieri Andrea Barbieri added a comment -

          Hudson v1.313 (latest to-day) contains the Subversion plugin which in turns uses
          SVNKit 1.3.0 and this in turns supports SVN 1.6.x and hence SVN externals
          pointing to files as well as directories.

          Show
          abarbieri Andrea Barbieri added a comment - Hudson v1.313 (latest to-day) contains the Subversion plugin which in turns uses SVNKit 1.3.0 and this in turns supports SVN 1.6.x and hence SVN externals pointing to files as well as directories.
          Hide
          abarbieri Andrea Barbieri added a comment -

          duplicate of Issue 3673 as well, I think...

          Show
          abarbieri Andrea Barbieri added a comment - duplicate of Issue 3673 as well, I think...
          Hide
          mkrohan mkrohan added a comment -

          This issue is not related to Subversion externals. The underlying problem is
          with Unix symbolic links checked into Subversion. Upto version 1.310, Hudson
          correctly created symbolic links on checking out from Subversion. Starting
          with version 1.311, it now creates files instead.

          From your comments, I suspect you work primarily on Windows. The following
          "how to repeat" requires Unix (e.g., Linux). I'm using my home directory in
          the following commands (/home/mrohan), substitute where appropriate:

          1) Create a Subversion repository:
          $ svnadmin create /home/mrohan/repos

          2) Create a directory in Subversion:
          $ svn mkdir -mExample file:///home/mrohan/repos/xmpl

          Committed revision 1.

          3) Checkout the directory:
          $ svn co file:///home/mrohan/repos/xmpl
          Checked out revision 1.

          4) Populate with a file and symbolic link and check in:
          $ cd xmpl
          $ echo "This is a file" > file.txt
          $ ln -s file.txt file.link
          $ ls -l
          total 4
          lrwxrwxrwx 1 mrohan mrohan 8 2009-06-30 09:32 file.link -> file.txt
          rw-rw-r- 1 mrohan mrohan 15 2009-06-30 09:32 file.txt
          $ svn add *
          A file.link
          A file.txt
          $ svn commit -mExample
          Adding file.link
          Adding file.txt
          Transmitting file data ..
          Committed revision 2.

          5) Start Hudson and define a job "Symlink Test" with
          5.1) Source Code in Subversion:
          Repository: file:///home/mrohan/repos/xmpl
          Local module: xmpl
          5.2) Add an "Execute shell" build step containing
          ls -l
          ls -l xmpl

          6) Run this job under v1.310 and v1.311, v1.312 or v1.312.

          The output for v1.310 shows a symlink in the workspace. The output
          for any later version show a file of 13 bytes. I'll attach my output.

          This is probably a bug in SVNKit but it is a regression for Hudson.

          Show
          mkrohan mkrohan added a comment - This issue is not related to Subversion externals. The underlying problem is with Unix symbolic links checked into Subversion. Upto version 1.310, Hudson correctly created symbolic links on checking out from Subversion. Starting with version 1.311, it now creates files instead. From your comments, I suspect you work primarily on Windows. The following "how to repeat" requires Unix (e.g., Linux). I'm using my home directory in the following commands (/home/mrohan), substitute where appropriate: 1) Create a Subversion repository: $ svnadmin create /home/mrohan/repos 2) Create a directory in Subversion: $ svn mkdir -mExample file:///home/mrohan/repos/xmpl Committed revision 1. 3) Checkout the directory: $ svn co file:///home/mrohan/repos/xmpl Checked out revision 1. 4) Populate with a file and symbolic link and check in: $ cd xmpl $ echo "This is a file" > file.txt $ ln -s file.txt file.link $ ls -l total 4 lrwxrwxrwx 1 mrohan mrohan 8 2009-06-30 09:32 file.link -> file.txt rw-rw-r - 1 mrohan mrohan 15 2009-06-30 09:32 file.txt $ svn add * A file.link A file.txt $ svn commit -mExample Adding file.link Adding file.txt Transmitting file data .. Committed revision 2. 5) Start Hudson and define a job "Symlink Test" with 5.1) Source Code in Subversion: Repository: file:///home/mrohan/repos/xmpl Local module: xmpl 5.2) Add an "Execute shell" build step containing ls -l ls -l xmpl 6) Run this job under v1.310 and v1.311, v1.312 or v1.312. The output for v1.310 shows a symlink in the workspace. The output for any later version show a file of 13 bytes. I'll attach my output. This is probably a bug in SVNKit but it is a regression for Hudson.
          Hide
          mkrohan mkrohan added a comment -

          Created an attachment (id=760)
          Subversion check-out listing for v1.310

          Show
          mkrohan mkrohan added a comment - Created an attachment (id=760) Subversion check-out listing for v1.310
          Hide
          mkrohan mkrohan added a comment -

          Created an attachment (id=761)
          Subversion check-out listing for v1.311

          Show
          mkrohan mkrohan added a comment - Created an attachment (id=761) Subversion check-out listing for v1.311
          Hide
          mkrohan mkrohan added a comment -

          Downloaded SVNKit 1.3.0.5847, made the example repository available via
          http (not sure what the exact file: syntax is on Windows). jsvn successfully
          checks out and creates a symlink on Unix. On Windows, since symlinks are
          not supported on this platform, it instead creates a file containing
          "link file.txt" (13 bytes).

          It appears Hudson might be configuring SVNKit for Windows on Unix systems.

          jsvn on Unix:
          $ jsvn co http://10.20.96.133/repos/xmpl
          A xmpl/file.link
          A xmpl/file.txt
          Checked out revision 2.
          $ ls -l xmpl
          total 4
          lrwxrwxrwx 1 mrohan mrohan 8 Jun 30 14:26 file.link -> file.txt
          rw-rw-r- 1 mrohan mrohan 15 Jun 30 14:26 file.txt

          jsvn on Windows:
          C:\mrohan\svnkit>jsvn co http://10.20.96.133/repos/xmpl
          A xmpl\file.link
          A xmpl\file.txt
          Checked out revision 2.

          C:\mrohan\svnkit>dir xmpl
          Volume in drive C is MROHAN-DEV3
          Volume Serial Number is 707B-3181

          Directory of C:\mrohan\svnkit\xmpl

          30-Jun-09 14:26 <DIR> .
          30-Jun-09 14:26 <DIR> ..
          30-Jun-09 14:26 13 file.link
          30-Jun-09 14:26 15 file.txt
          2 File(s) 28 bytes
          2 Dir(s) 14,278,062,080 bytes free

          C:\mrohan\svnkit>type xmpl\file.link
          link file.txt

          Show
          mkrohan mkrohan added a comment - Downloaded SVNKit 1.3.0.5847, made the example repository available via http (not sure what the exact file: syntax is on Windows). jsvn successfully checks out and creates a symlink on Unix. On Windows, since symlinks are not supported on this platform, it instead creates a file containing "link file.txt" (13 bytes). It appears Hudson might be configuring SVNKit for Windows on Unix systems. jsvn on Unix: $ jsvn co http://10.20.96.133/repos/xmpl A xmpl/file.link A xmpl/file.txt Checked out revision 2. $ ls -l xmpl total 4 lrwxrwxrwx 1 mrohan mrohan 8 Jun 30 14:26 file.link -> file.txt rw-rw-r - 1 mrohan mrohan 15 Jun 30 14:26 file.txt jsvn on Windows: C:\mrohan\svnkit>jsvn co http://10.20.96.133/repos/xmpl A xmpl\file.link A xmpl\file.txt Checked out revision 2. C:\mrohan\svnkit>dir xmpl Volume in drive C is MROHAN-DEV3 Volume Serial Number is 707B-3181 Directory of C:\mrohan\svnkit\xmpl 30-Jun-09 14:26 <DIR> . 30-Jun-09 14:26 <DIR> .. 30-Jun-09 14:26 13 file.link 30-Jun-09 14:26 15 file.txt 2 File(s) 28 bytes 2 Dir(s) 14,278,062,080 bytes free C:\mrohan\svnkit>type xmpl\file.link link file.txt
          Hide
          mkrohan mkrohan added a comment -

          This is a Unix (Linux) specific issue.

          Show
          mkrohan mkrohan added a comment - This is a Unix (Linux) specific issue.
          Hide
          akosmaroy akosmaroy added a comment -

          is there a workaround for this issue, until it is fixed?

          Show
          akosmaroy akosmaroy added a comment - is there a workaround for this issue, until it is fixed?
          Hide
          mkrohan mkrohan added a comment -

          I have not been able to find a workaround. I simply reverted my instance back
          to v1.309 (don't update to every version).

          Show
          mkrohan mkrohan added a comment - I have not been able to find a workaround. I simply reverted my instance back to v1.309 (don't update to every version).
          Hide
          akosmaroy akosmaroy added a comment -

          oh, it works in 1.309 - excellent...

          Show
          akosmaroy akosmaroy added a comment - oh, it works in 1.309 - excellent...
          Hide
          mkrohan mkrohan added a comment -

          It works up v1.310 and fails starting with v1.311

          Show
          mkrohan mkrohan added a comment - It works up v1.310 and fails starting with v1.311
          Hide
          thoth39 thoth39 added a comment -

          This bug is blocking deployment of Hudson on my project.
          Please raise it's priority.

          Show
          thoth39 thoth39 added a comment - This bug is blocking deployment of Hudson on my project. Please raise it's priority.
          Hide
          smworley smworley added a comment -

          I would also like to see this fixed ...

          As it stands, I am able to work around this issue by

          • manually removing the work area (jobs/<project>/workspace/trunk)
          • manually checking out a new trunk (svn co http://.. ../trunk)
          • updating my <project> configuration to USE UPDATE
          • Under "Subversion", select "Use Update" and save

          From this I am able to release using the Hudson Maven Release Plugin. It would
          appear that the plug-in is not directly impacted by this change ..??.. E.g. It
          creates a new "checkout" directory under target which has working symlinks.

          Show
          smworley smworley added a comment - I would also like to see this fixed ... As it stands, I am able to work around this issue by manually removing the work area (jobs/<project>/workspace/trunk) manually checking out a new trunk (svn co http:// .. ../trunk) updating my <project> configuration to USE UPDATE Under "Subversion", select "Use Update" and save From this I am able to release using the Hudson Maven Release Plugin. It would appear that the plug-in is not directly impacted by this change ..??.. E.g. It creates a new "checkout" directory under target which has working symlinks.
          Hide
          mkrohan mkrohan added a comment -

          Issue #3968 appears to be a duplicate of this issue.

          Show
          mkrohan mkrohan added a comment - Issue #3968 appears to be a duplicate of this issue.
          Hide
          mindless Alan Harder added a comment -
              • Issue 3968 has been marked as a duplicate of this issue. ***
          Show
          mindless Alan Harder added a comment - Issue 3968 has been marked as a duplicate of this issue. ***
          Hide
          mindless Alan Harder added a comment -
              • Issue 3904 has been marked as a duplicate of this issue. ***
          Show
          mindless Alan Harder added a comment - Issue 3904 has been marked as a duplicate of this issue. ***
          Hide
          mindless Alan Harder added a comment -

          set P2

          Show
          mindless Alan Harder added a comment - set P2
          Hide
          abarbieri Andrea Barbieri added a comment -

          in SubversionSCM.java lines 1656-1658:
          // work around for
          http://www.nabble.com/Slow-SVN-Checkout-tf4486786.html
          if(System.getProperty("svnkit.symlinks")==null)
          System.setProperty("svnkit.symlinks","false");

          could this be the cause of the problem on Hudson Unix nodes?

          the SVNKit FAQ suggests to disable symlinks only for 'older' versions of SVNKit,
          whilst for the latest one with proper JNA support (as in svnkit/1.3.0-hudson-1)
          symlinks feature should remain enabled.

          Show
          abarbieri Andrea Barbieri added a comment - in SubversionSCM.java lines 1656-1658: // work around for http://www.nabble.com/Slow-SVN-Checkout-tf4486786.html if(System.getProperty("svnkit.symlinks")==null) System.setProperty("svnkit.symlinks","false"); could this be the cause of the problem on Hudson Unix nodes? the SVNKit FAQ suggests to disable symlinks only for 'older' versions of SVNKit, whilst for the latest one with proper JNA support (as in svnkit/1.3.0-hudson-1) symlinks feature should remain enabled.
          Hide
          mkrohan mkrohan added a comment -

          I'm not familiar with the Hudson code base, but the snippet included
          suggested running Hudson with "java -Dsvnkit.symlinks=true -jar hudson.jar"
          would enable symlinks. I tried this for v1.314 and I'm afraid I'm still
          seeing a file containing "link file.txt". This does not invalidate the
          coding suggestion, just additional info.

          Show
          mkrohan mkrohan added a comment - I'm not familiar with the Hudson code base, but the snippet included suggested running Hudson with "java -Dsvnkit.symlinks=true -jar hudson.jar" would enable symlinks. I tried this for v1.314 and I'm afraid I'm still seeing a file containing "link file.txt". This does not invalidate the coding suggestion, just additional info.
          Hide
          abarbieri Andrea Barbieri added a comment -

          Have you tried turning on some debugging?
          https://wiki.svnkit.com/Troubleshooting

          with the file logging.properties:
          svnkit.level=FINEST
          svnkit-network.level=FINEST
          svnkit-fsfs.level=FINEST
          svnkit-wc.level=FINEST
          svnkit-cli.level=FINEST
          #
          handlers = java.util.logging.FileHandler

          #java.util.logging.FileHandler.pattern = %h/svnkit.%u.log
          java.util.logging.FileHandler.pattern = svnkit.%u.log
          java.util.logging.FileHandler.limit = 0
          java.util.logging.FileHandler.count = 1
          java.util.logging.FileHandler.append = false
          java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter

          turning on some debugging for Hudson would help especially for
          hudson.scm.Initializer to check if somehow svnkit.symlinks is set to false

          Show
          abarbieri Andrea Barbieri added a comment - Have you tried turning on some debugging? https://wiki.svnkit.com/Troubleshooting with the file logging.properties: svnkit.level=FINEST svnkit-network.level=FINEST svnkit-fsfs.level=FINEST svnkit-wc.level=FINEST svnkit-cli.level=FINEST # handlers = java.util.logging.FileHandler #java.util.logging.FileHandler.pattern = %h/svnkit.%u.log java.util.logging.FileHandler.pattern = svnkit.%u.log java.util.logging.FileHandler.limit = 0 java.util.logging.FileHandler.count = 1 java.util.logging.FileHandler.append = false java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter turning on some debugging for Hudson would help especially for hudson.scm.Initializer to check if somehow svnkit.symlinks is set to false
          Hide
          mkrohan mkrohan added a comment -

          This logging suggestion worked: my initial test of "svnkit.symlinks"
          was not valid as the test job had "Use update" enabled. Disabling this,
          i.e., forcing a clean checkout, with "-Dsvnkit.symlinks=true" on the
          java command line generated a workspace with the expected symlinks
          instead of files.

          For your reference, I'm attaching the svnkit logs for the runs without
          the svnkit.symlinks defn, and with the definitions.

          Show
          mkrohan mkrohan added a comment - This logging suggestion worked: my initial test of "svnkit.symlinks" was not valid as the test job had "Use update" enabled. Disabling this, i.e., forcing a clean checkout, with "-Dsvnkit.symlinks=true" on the java command line generated a workspace with the expected symlinks instead of files. For your reference, I'm attaching the svnkit logs for the runs without the svnkit.symlinks defn, and with the definitions.
          Hide
          mkrohan mkrohan added a comment -

          Created an attachment (id=769)
          SVNKit log file without -Dsvnkit.symlinks=true on cmd line

          Show
          mkrohan mkrohan added a comment - Created an attachment (id=769) SVNKit log file without -Dsvnkit.symlinks=true on cmd line
          Hide
          mkrohan mkrohan added a comment -

          Created an attachment (id=770)
          SVNKit log file with -Dsvnkit.symlinks=true on cmd line

          Show
          mkrohan mkrohan added a comment - Created an attachment (id=770) SVNKit log file with -Dsvnkit.symlinks=true on cmd line
          Hide
          dodok1 dodok1 added a comment -

          I wuold like to point out that there is simple test case in
          SubversionSCMTest::testSymbolicLinkCheckout() to check this problem.

          I think that this problem has something to do with JNA.

          Show
          dodok1 dodok1 added a comment - I wuold like to point out that there is simple test case in SubversionSCMTest::testSymbolicLinkCheckout() to check this problem. I think that this problem has something to do with JNA.
          Hide
          dodok1 dodok1 added a comment -

          After correcting my test case , I approve that solution is to remove those
          lines setting svnkit.symlinks to true.
          The issue with slow checkouts was solved in svnkit 1.3.0 using JNA.

          As soon as I will test in on the real deployment, I will commit that change.

          Show
          dodok1 dodok1 added a comment - After correcting my test case , I approve that solution is to remove those lines setting svnkit.symlinks to true. The issue with slow checkouts was solved in svnkit 1.3.0 using JNA. As soon as I will test in on the real deployment, I will commit that change.
          Hide
          dodok1 dodok1 added a comment -

          solved in trunk, removed obsolete workaround

          Show
          dodok1 dodok1 added a comment - solved in trunk, removed obsolete workaround
          Hide
          tommythekid Tommy McNeely added a comment -

          If anyone else thinks they are being bit by this very old bug, please check to see if you are actually affected by https://issues.jenkins-ci.org/browse/JENKINS-14217 ... which is what was getting us.

          ~tommy

          Show
          tommythekid Tommy McNeely added a comment - If anyone else thinks they are being bit by this very old bug, please check to see if you are actually affected by https://issues.jenkins-ci.org/browse/JENKINS-14217 ... which is what was getting us. ~tommy

            People

            • Assignee:
              Unassigned
              Reporter:
              mkrohan mkrohan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: