Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13844

"Emulate clean checkout by..." does not work with 1.7 subversion repositories

    Details

    • Similar Issues:

      Description

      The update strategy "Emulate clean checkout by..." does not delete files which are not part of the repository although the log reports that a cleanup is performed.

        Attachments

          Issue Links

            Activity

            Hide
            sogabe sogabe added a comment -

            Subversion Plugin DOES NOT support 1.7.

            Show
            sogabe sogabe added a comment - Subversion Plugin DOES NOT support 1.7.
            Hide
            simabeis Marcel Beister added a comment -

            To clarify this: The error happens when I chose "1.7" as format for the working copy, the server runs subversion "1.6".

            Show
            simabeis Marcel Beister added a comment - To clarify this: The error happens when I chose "1.7" as format for the working copy, the server runs subversion "1.6".
            Hide
            simabeis Marcel Beister added a comment -

            See Issue #2.

            Show
            simabeis Marcel Beister added a comment - See Issue #2.
            Hide
            depe Denis Eperonnier added a comment -

            Same for me using SVN server 1.5, Jenkins 1.472 + SVN 1.42 plugin, SVN WC 1.7
            Issue occurs on a WinXP 32bit slave

            Show
            depe Denis Eperonnier added a comment - Same for me using SVN server 1.5, Jenkins 1.472 + SVN 1.42 plugin, SVN WC 1.7 Issue occurs on a WinXP 32bit slave
            Hide
            martan Martin Adam added a comment -

            same here, environmen same as in comment above me, only I have jenkins 1.475 and it's a master building node

            Show
            martan Martin Adam added a comment - same here, environmen same as in comment above me, only I have jenkins 1.475 and it's a master building node
            Hide
            chris_lee Chris Lee added a comment -

            This is caused by changes in SVN 1.7 relating to ignored files - they are no longer reported by default in 'svn status' (the SVNKit API used by Jenkins mirrors this behaviour).

            In the workspace for a Jenkins project (after the job has run), no changes are shown:

            [root@devemr01 UTIL_TRUNK]# svn status
            

            However, the below command shows files/directories that are marked as ignored:

            [root@devemr01 UTIL_TRUNK]# svn status --no-ignore
            I       .gradle
            I       build
            

            ...and creating a new file that isn't ignored shows up in 'svn status':

            [root@devemr01 UTIL_TRUNK]# touch foo
            [root@devemr01 UTIL_TRUNK]# svn status
            ?       foo
            

            UpdateWithCleanUpdater.java needs to explicitly request that SVNKit include 'ignored' files such that they can be deleted.

            Show
            chris_lee Chris Lee added a comment - This is caused by changes in SVN 1.7 relating to ignored files - they are no longer reported by default in 'svn status' (the SVNKit API used by Jenkins mirrors this behaviour). In the workspace for a Jenkins project (after the job has run), no changes are shown: [root@devemr01 UTIL_TRUNK]# svn status However, the below command shows files/directories that are marked as ignored: [root@devemr01 UTIL_TRUNK]# svn status --no-ignore I .gradle I build ...and creating a new file that isn't ignored shows up in 'svn status': [root@devemr01 UTIL_TRUNK]# touch foo [root@devemr01 UTIL_TRUNK]# svn status ? foo UpdateWithCleanUpdater.java needs to explicitly request that SVNKit include 'ignored' files such that they can be deleted.
            Hide
            chris_lee Chris Lee added a comment -

            Code from UpdateWithCleanUpdater.java:

              clientManager.getStatusClient().doStatus(local, null, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler() {
                            public void handleStatus(SVNStatus status) throws SVNException {
                                SVNStatusType s = status.getContentsStatus();
                                if (s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED) {
                                    listener.getLogger().println("Deleting "+status.getFile());
                                    try {
                                        File f = status.getFile();
                                        if (f.isDirectory())
                                            hudson.Util.deleteRecursive(f);
                                        else
                                            f.delete();
                                    } catch (IOException e) {
                                        throw new SVNException(SVNErrorMessage.create(SVNErrorCode.UNKNOWN, e));
                                    }
                                }
                            }
                        }, null);
            

            ...the fifth parameter is 'includeIgnored' which is strangely set to false (and always has been), although the subsequent code expects ignored elements. Guessing that the inclusion of 'ignored' files was a bug with previous SVN (and SVNKit) versions that Jenkins inadvertently relied on.

            SVNKit API: http://svnkit.com/javadoc/org/tmatesoft/svn/core/wc/SVNStatusClient.html#doStatus(java.io.File, org.tmatesoft.svn.core.wc.SVNRevision, org.tmatesoft.svn.core.SVNDepth, boolean, boolean, boolean, boolean, org.tmatesoft.svn.core.wc.ISVNStatusHandler, java.util.Collection)

            Show
            chris_lee Chris Lee added a comment - Code from UpdateWithCleanUpdater.java: clientManager.getStatusClient().doStatus(local, null , SVNDepth.INFINITY, false , false , true , false , new ISVNStatusHandler() { public void handleStatus(SVNStatus status) throws SVNException { SVNStatusType s = status.getContentsStatus(); if (s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED) { listener.getLogger().println( "Deleting " +status.getFile()); try { File f = status.getFile(); if (f.isDirectory()) hudson.Util.deleteRecursive(f); else f.delete(); } catch (IOException e) { throw new SVNException(SVNErrorMessage.create(SVNErrorCode.UNKNOWN, e)); } } } }, null ); ...the fifth parameter is 'includeIgnored' which is strangely set to false (and always has been), although the subsequent code expects ignored elements. Guessing that the inclusion of 'ignored' files was a bug with previous SVN (and SVNKit) versions that Jenkins inadvertently relied on. SVNKit API: http://svnkit.com/javadoc/org/tmatesoft/svn/core/wc/SVNStatusClient.html#doStatus(java.io.File , org.tmatesoft.svn.core.wc.SVNRevision, org.tmatesoft.svn.core.SVNDepth, boolean, boolean, boolean, boolean, org.tmatesoft.svn.core.wc.ISVNStatusHandler, java.util.Collection)
            Hide
            martan Martin Adam added a comment -

            Does this explain not deleting unversioned files as well? Because you're mentioning only ignored files...

            Show
            martan Martin Adam added a comment - Does this explain not deleting unversioned files as well? Because you're mentioning only ignored files...
            Hide
            chris_lee Chris Lee added a comment -

            No. There is more to this issue than just the ignored files.

            Here's a test case from a simple Jenkins / SVN (Jenkins 1.483, SVN 1.7):

            After wiping out the workspace and running an initial build:

            [root@devemr01 UTIL_TRUNK]# ll
            total 24
            drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build
            -rw-rw-r--  1 jenkins jenkins  376 Sep 30 00:06 build.gradle
            drwxrwxr-x  2 jenkins jenkins 4096 Sep 30 00:06 lib
            drwxrwxr-x  2 jenkins jenkins 4096 Sep 30 00:06 releng
            -rw-rw-r--  1 jenkins jenkins   37 Sep 30 00:06 settings.gradle
            drwxrwxr-x  4 jenkins jenkins 4096 Sep 30 00:06 src
            

            ...adding an unversioned file:

            [root@devemr01 UTIL_TRUNK]# touch foo
            [root@devemr01 UTIL_TRUNK]# svn status
            ?       foo
            

            Running the job (note that the unversioned file wasn't detected or removed):

            00:07:59 Cleaning up /home/jenkins/workspace/UTIL_TRUNK/.
            00:07:59 Updating svn://svn.ma.net/util/trunk to revision '2012-09-30T00:07:58.927 -0700'
            00:07:59 At revision 38394
            

            ...and the unversioned file, and the ignored dir 'build', are still present:

            [root@devemr01 UTIL_TRUNK]# ll
            total 24
            drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build
            -rw-rw-r--  1 jenkins jenkins  376 Sep 30 00:06 build.gradle
            -rw-r--r--  1 root    root       0 Sep 30 00:07 foo
            drwxrwxr-x  2 jenkins jenkins 4096 Sep 30 00:06 lib
            drwxrwxr-x  2 jenkins jenkins 4096 Sep 30 00:06 releng
            -rw-rw-r--  1 jenkins jenkins   37 Sep 30 00:06 settings.gradle
            drwxrwxr-x  4 jenkins jenkins 4096 Sep 30 00:06 src
            [root@devemr01 UTIL_TRUNK]# 
            

            It appears that the SVNKit 'status' API call is working differently (and possibly incorrectly) for SVN 1.7 repositories.

            Show
            chris_lee Chris Lee added a comment - No. There is more to this issue than just the ignored files. Here's a test case from a simple Jenkins / SVN (Jenkins 1.483, SVN 1.7): After wiping out the workspace and running an initial build: [root@devemr01 UTIL_TRUNK]# ll total 24 drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build -rw-rw-r-- 1 jenkins jenkins 376 Sep 30 00:06 build.gradle drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 lib drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 releng -rw-rw-r-- 1 jenkins jenkins 37 Sep 30 00:06 settings.gradle drwxrwxr-x 4 jenkins jenkins 4096 Sep 30 00:06 src ...adding an unversioned file: [root@devemr01 UTIL_TRUNK]# touch foo [root@devemr01 UTIL_TRUNK]# svn status ? foo Running the job (note that the unversioned file wasn't detected or removed): 00:07:59 Cleaning up /home/jenkins/workspace/UTIL_TRUNK/. 00:07:59 Updating svn: //svn.ma.net/util/trunk to revision '2012-09-30T00:07:58.927 -0700' 00:07:59 At revision 38394 ...and the unversioned file, and the ignored dir 'build', are still present: [root@devemr01 UTIL_TRUNK]# ll total 24 drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build -rw-rw-r-- 1 jenkins jenkins 376 Sep 30 00:06 build.gradle -rw-r--r-- 1 root root 0 Sep 30 00:07 foo drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 lib drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 releng -rw-rw-r-- 1 jenkins jenkins 37 Sep 30 00:06 settings.gradle drwxrwxr-x 4 jenkins jenkins 4096 Sep 30 00:06 src [root@devemr01 UTIL_TRUNK]# It appears that the SVNKit 'status' API call is working differently (and possibly incorrectly) for SVN 1.7 repositories.
            Hide
            chris_lee Chris Lee added a comment -

            Found the issue; the status objects returned by SVNKit have different values when used against a SVN 1.7 working copy.

            Using the original snippet of code, status objects are returned for the correct files, but their status is STATUS_NONE, and hence the subsequent logic for removing files of specific statuses is not triggered:

             
            /home/jenkins/workspace/UTIL_TRUNK/.gradle    none 
            /home/jenkins/workspace/UTIL_TRUNK/foo        none 
            /home/jenkins/workspace/UTIL_TRUNK/build      none 
            

            Modifying the code to combine the node & contents status yields the correct results:

            /home/jenkins/workspace/UTIL_TRUNK/.gradle                   ignored
            Deleting /home/jenkins/workspace/UTIL_TRUNK/.gradle
            /home/jenkins/workspace/UTIL_TRUNK/foo                       unversioned
            Deleting /home/jenkins/workspace/UTIL_TRUNK/foo
            /home/jenkins/workspace/UTIL_TRUNK/build                     ignored
            

            The relevant change is:

             
                 // for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.) 
                 SVNStatusType s = status.getCombinedNodeAndContentsStatus(); 
                 //SVNStatusType s = status.getContentsStatus(); 
            

            The complete test harness is:

            import java.io.File;
            
            import java.util.*;
            import org.tmatesoft.svn.core.SVNDepth;
            import org.tmatesoft.svn.core.SVNException;
            import org.tmatesoft.svn.core.wc.ISVNStatusHandler;
            import org.tmatesoft.svn.core.wc.SVNClientManager;
            import org.tmatesoft.svn.core.wc.SVNStatus;
            import org.tmatesoft.svn.core.wc.SVNStatusType;
            import org.tmatesoft.svn.core.wc.*;
            import org.tmatesoft.svn.core.internal.wc.*;
            
            public class Test
            {
                public static void main( String[] args ) throws SVNException
                {
                    SVNClientManager clientManager = SVNClientManager.newInstance();
            
                    Collection<String> changeLists = Collections.emptyList();
                    //clientManager.getStatusClient().doStatus( new File( args[ 0 ] ).getAbsoluteFile(), SVNRevision.WORKING, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler()
                    clientManager.getStatusClient().doStatus( new File( args[ 0 ] ), null, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler()
                    {
                        @Override
                        public void handleStatus( SVNStatus status ) throws SVNException
                        {
                            // for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.)
                            SVNStatusType s = status.getCombinedNodeAndContentsStatus();
                            //SVNStatusType s = status.getContentsStatus();
            
                            System.out.printf( "%-60s %s\n", status.getFile(), s );
                            if( s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED )
                            {
                                System.out.println( "Deleting " + status.getFile() );
                            }
                        }
                    }, null );
                }
            }
            
            Show
            chris_lee Chris Lee added a comment - Found the issue; the status objects returned by SVNKit have different values when used against a SVN 1.7 working copy. Using the original snippet of code, status objects are returned for the correct files, but their status is STATUS_NONE, and hence the subsequent logic for removing files of specific statuses is not triggered: /home/jenkins/workspace/UTIL_TRUNK/.gradle none /home/jenkins/workspace/UTIL_TRUNK/foo none /home/jenkins/workspace/UTIL_TRUNK/build none Modifying the code to combine the node & contents status yields the correct results: /home/jenkins/workspace/UTIL_TRUNK/.gradle ignored Deleting /home/jenkins/workspace/UTIL_TRUNK/.gradle /home/jenkins/workspace/UTIL_TRUNK/foo unversioned Deleting /home/jenkins/workspace/UTIL_TRUNK/foo /home/jenkins/workspace/UTIL_TRUNK/build ignored The relevant change is: // for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.) SVNStatusType s = status.getCombinedNodeAndContentsStatus(); //SVNStatusType s = status.getContentsStatus(); The complete test harness is: import java.io.File; import java.util.*; import org.tmatesoft.svn.core.SVNDepth; import org.tmatesoft.svn.core.SVNException; import org.tmatesoft.svn.core.wc.ISVNStatusHandler; import org.tmatesoft.svn.core.wc.SVNClientManager; import org.tmatesoft.svn.core.wc.SVNStatus; import org.tmatesoft.svn.core.wc.SVNStatusType; import org.tmatesoft.svn.core.wc.*; import org.tmatesoft.svn.core.internal.wc.*; public class Test { public static void main( String [] args ) throws SVNException { SVNClientManager clientManager = SVNClientManager.newInstance(); Collection< String > changeLists = Collections.emptyList(); //clientManager.getStatusClient().doStatus( new File( args[ 0 ] ).getAbsoluteFile(), SVNRevision.WORKING, SVNDepth.INFINITY, false , false , true , false , new ISVNStatusHandler() clientManager.getStatusClient().doStatus( new File( args[ 0 ] ), null , SVNDepth.INFINITY, false , false , true , false , new ISVNStatusHandler() { @Override public void handleStatus( SVNStatus status ) throws SVNException { // for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.) SVNStatusType s = status.getCombinedNodeAndContentsStatus(); //SVNStatusType s = status.getContentsStatus(); System .out.printf( "%-60s %s\n" , status.getFile(), s ); if ( s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED ) { System .out.println( "Deleting " + status.getFile() ); } } }, null ); } }
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Nicolas De loof
            Path:
            src/main/java/hudson/scm/subversion/UpdateWithCleanUpdater.java
            http://jenkins-ci.org/commit/subversion-plugin/82377581f16cccaed8c4a90169d7d4c11d3f9845
            Log:
            Merge pull request #22 from forderud/master

            JENKINS-13844: "Emulate clean checkout by..." does not work with 1.7 subversion repositories

            Compare: https://github.com/jenkinsci/subversion-plugin/compare/3f18badadf37...82377581f16c

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De loof Path: src/main/java/hudson/scm/subversion/UpdateWithCleanUpdater.java http://jenkins-ci.org/commit/subversion-plugin/82377581f16cccaed8c4a90169d7d4c11d3f9845 Log: Merge pull request #22 from forderud/master JENKINS-13844 : "Emulate clean checkout by..." does not work with 1.7 subversion repositories Compare: https://github.com/jenkinsci/subversion-plugin/compare/3f18badadf37...82377581f16c
            Hide
            forderud Fredrik Orderud added a comment -

            Unable to reproduce problem anymore after commit of pull request #22. Therefore resolving.

            Show
            forderud Fredrik Orderud added a comment - Unable to reproduce problem anymore after commit of pull request #22. Therefore resolving.
            Hide
            hx_unbanned Linards L added a comment -

            Still happens on 1.492 with svn 1.43.

            System:
            java.vm.version 22.1-b02
            line.separator
            mail.smtp.sendpartial true
            mail.smtps.sendpartial true
            os.arch amd64
            os.name Windows Server 2008 R2 (Datacenter)

            Show
            hx_unbanned Linards L added a comment - Still happens on 1.492 with svn 1.43. System: java.vm.version 22.1-b02 line.separator mail.smtp.sendpartial true mail.smtps.sendpartial true os.arch amd64 os.name Windows Server 2008 R2 (Datacenter)
            Hide
            forderud Fredrik Orderud added a comment - - edited

            The fix has now been released with Subversion plugin 1.44.

            Linards L: Could you please try to upgrade to SVN plugin 1.44, and try to reproduce afterwards?

            Show
            forderud Fredrik Orderud added a comment - - edited The fix has now been released with Subversion plugin 1.44. Linards L: Could you please try to upgrade to SVN plugin 1.44, and try to reproduce afterwards?
            Hide
            hx_unbanned Linards L added a comment -

            Ok. Please update fixVersion field

            Show
            hx_unbanned Linards L added a comment - Ok. Please update fixVersion field
            Hide
            forderud Fredrik Orderud added a comment -

            Agree, but for it appears like the version field(s) aren't properly configured. I receive an "1.44 is not a valid version" error message when attempting to update the field.

            Show
            forderud Fredrik Orderud added a comment - Agree, but for it appears like the version field(s) aren't properly configured. I receive an "1.44 is not a valid version" error message when attempting to update the field.
            Hide
            kutzi kutzi added a comment -

            The version fields in JIRA only have the Jenkins core versions. This is a known limitation of the current JIRA setup.

            Show
            kutzi kutzi added a comment - The version fields in JIRA only have the Jenkins core versions. This is a known limitation of the current JIRA setup.

              People

              • Assignee:
                Unassigned
                Reporter:
                simabeis Marcel Beister
              • Votes:
                8 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: