Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41626

Branch indexing on subversion repo does not work properly

    Details

    • Similar Issues:

      Description

      The multibranch indexer does not traverse sub-folders in subversion branches, if their revision is older than the parent directory.

      Include Branches: tags/*/projects/productname
      

      Given this folder structure in the trunk:

      projects/                -> r1
        productname/           -> r2
          Jenkinsfile          -> r3
      

      Branching projects into tags/1.0/projects yields

      tags/                     -> r4
        1.0/                    -> r4
          projects/             -> r1
             productname/       -> r2
               Jenkinsfile      -> r3
      

      Note that r4 is only applied to the new folders, and the branched folders retain their original revision (because they haven't changed).

      The expected behaviour here (and surely, the purpose of the multi-branch system), is for it to automatically detect the new branch and generate jobs from its Jenkinsfiles.

      The branch indexer should be able to work out that the 1.0/ folder didn't exist before, and index everything inside it at least once regardless of its revision.

        Attachments

          Issue Links

            Activity

            Hide
            codingspiderfox Chris H added a comment -

            Is there any workaround?

            Show
            codingspiderfox Chris H added a comment - Is there any workaround?
            Hide
            jfemia James Femia added a comment -

            I work around it by manually committing whitespace changes to the Jenkinsfiles after branching, to force the revision to be newer - the indexer then detects it.

            It's a pain because that is an additional manual step I have to perform after branching in order to have any new branches detected immediately.

            Show
            jfemia James Femia added a comment - I work around it by manually committing whitespace changes to the Jenkinsfiles after branching, to force the revision to be newer - the indexer then detects it. It's a pain because that is an additional manual step I have to perform after branching in order to have any new branches detected immediately.
            Hide
            rs Reinhard S added a comment -

            I am facing the same problem. Jenkins does not find Jenkinsfile's in branch subfolders, unless you manually push the SVN revision by a dummy commit. In the indexing output you immediately see the problem, that SVN is looking for the revision of the child element, instead of looking into the HEAD revision. If the revision of the child element is older than the branch, SVN is not finding anything.

            It seems the fix is rather trivial. When looking at https://github.com/jenkinsci/subversion-plugin/blob/136cf89ab1a0017b241bcecee4f5e40045c1abaf/src/main/java/jenkins/scm/impl/subversion/SubversionSCMSource.java#L344 the fetch function should always use HEAD instead of the child revision, especially in the recursive fetch call in line 421 (just replace svnEntry.getRevision() by -1?). Can anyone comment why the code currently uses the child revision instead of HEAD?

             

            Show
            rs Reinhard S added a comment - I am facing the same problem. Jenkins does not find Jenkinsfile's in branch subfolders, unless you manually push the SVN revision by a dummy commit. In the indexing output you immediately see the problem, that SVN is looking for the revision of the child element, instead of looking into the HEAD revision. If the revision of the child element is older than the branch, SVN is not finding anything. It seems the fix is rather trivial. When looking at https://github.com/jenkinsci/subversion-plugin/blob/136cf89ab1a0017b241bcecee4f5e40045c1abaf/src/main/java/jenkins/scm/impl/subversion/SubversionSCMSource.java#L344  the fetch function should always use HEAD instead of the child revision, especially in the recursive fetch call in line 421 (just replace svnEntry.getRevision() by -1?). Can anyone comment why the code currently uses the child revision instead of HEAD?  
            Hide
            jfemia James Femia added a comment -

            I spent some time looking at that code before filing this issue. I don't know for sure but I believe this is because jobs don't necessarily want to use the head revision. For example, if you want a build per revision and your job ends up in a queue, by the time it is run on an agent there might have been other commits to the repo causing the HEAD to be newer than expected.

            Show
            jfemia James Femia added a comment - I spent some time looking at that code before filing this issue. I don't know for sure but I believe this is because jobs don't necessarily want to use the head revision. For example, if you want a build per revision and your job ends up in a queue, by the time it is run on an agent there might have been other commits to the repo causing the HEAD to be newer than expected.
            Hide
            rs Reinhard S added a comment - - edited

            That's true, you do not want to get commits into your result while the job is running. But this can be solved by resolving "HEAD" to the first real revision, in your example r4, and then giving the revision to the recursive fetch calls, i.e. requesting all sub-folders with revision r4.

            The first fetch call gets -1 as parameter, which means resolve "HEAD" and all recursive calls get specific revision numbers. That should solve both problems, only fetch code that was commit at job start and second, branches will checkout subfolders with the same revision as the branch.

            Show
            rs Reinhard S added a comment - - edited That's true, you do not want to get commits into your result while the job is running. But this can be solved by resolving "HEAD" to the first real revision, in your example r4, and then giving the revision to the recursive fetch calls, i.e. requesting all sub-folders with revision r4. The first fetch call gets -1 as parameter, which means resolve "HEAD" and all recursive calls get specific revision numbers. That should solve both problems, only fetch code that was commit at job start and second, branches will checkout subfolders with the same revision as the branch.
            Hide
            jfemia James Femia added a comment -

            That sounds reasonable to me ( as a non-expert )

            Show
            jfemia James Femia added a comment - That sounds reasonable to me ( as a non-expert )
            Hide
            bakkume Eric Bakkum added a comment -

            FYI, there is an open poll request that resolves this: https://github.com/jenkinsci/subversion-plugin/pull/205

            Show
            bakkume Eric Bakkum added a comment - FYI, there is an open poll request that resolves this: https://github.com/jenkinsci/subversion-plugin/pull/205
            Show
            oleg_nenashev Oleg Nenashev added a comment - https://github.com/jenkinsci/subversion-plugin/pull/205
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Andy Neebel
            Path:
            src/main/java/jenkins/scm/impl/subversion/SubversionSCMSource.java
            http://jenkins-ci.org/commit/subversion-plugin/82e77a01979798ede7ca5f22dceaca03c6d2a737
            Log:
            JENKINS-41626 Fix path detection when subfolder older than root folder (#205)

            Checks for folders in HEAD, then checks if the candidate folder is available using it's revision, if not report it as the HEAD revision.

            • Clean up parameter that no longer matters
            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andy Neebel Path: src/main/java/jenkins/scm/impl/subversion/SubversionSCMSource.java http://jenkins-ci.org/commit/subversion-plugin/82e77a01979798ede7ca5f22dceaca03c6d2a737 Log: JENKINS-41626 Fix path detection when subfolder older than root folder (#205) Fix for JENKINS-41626 Checks for folders in HEAD, then checks if the candidate folder is available using it's revision, if not report it as the HEAD revision. Clean up parameter that no longer matters

              People

              • Assignee:
                Unassigned
                Reporter:
                jfemia James Femia
              • Votes:
                9 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: