Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30744

multibranch issues if branch contains /

    Details

    • Similar Issues:

      Description

      Our repository contains a variant of the git-flow branch naming convention, where you will see
      branches such as dev/main or release/3.2

      Using multibranch, strange things happen:

      • dev/main gets displayed as dev%2Fmain, then on restart dev%252Fmain, then dev%25252Fmain and all history seems to be lost.

      This is likely down to / being a dodgy delimiter for a project name. I tried to fix this by adjusting WorkflowBranchProjectFactory thusly:

      -        WorkflowJob job = new WorkflowJob((WorkflowMultiBranchProject) getOwner(), branch.getName());
      +        String branchName = branch.getName().replace("/", "-");
      +        WorkflowJob job = new WorkflowJob((WorkflowMultiBranchProject) getOwner(), branchName);
      

      but it didn't seem to help.

      Happy to try to fix if I can get a pointer as to what to look at.

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            Seems like your proposed patch should have addressed it, though at the expense of potential clashes with branches actually named dev-main or release-3.2. Solving that would require a reversible encoding such as the URL-encoding you observed, but working of course.

            Not sure if this has to be handled in workflow-multibranch, branch-api, or both. Need to think about it, write tests, and perhaps consult with Stephen Connolly to see if this ever came up for Literate and how it was handled. I suspect the encoding in MultiBranchProject.getRootDirFor is relevant, but using a / inside an Item.getName is invalid generally (regardless of disk storage location), as that breaks the semantics of Item.getFullName and much else besides.

            Show
            jglick Jesse Glick added a comment - Seems like your proposed patch should have addressed it, though at the expense of potential clashes with branches actually named dev-main or release-3.2 . Solving that would require a reversible encoding such as the URL-encoding you observed, but working of course. Not sure if this has to be handled in workflow-multibranch , branch-api , or both. Need to think about it, write tests, and perhaps consult with Stephen Connolly to see if this ever came up for Literate and how it was handled. I suspect the encoding in MultiBranchProject.getRootDirFor is relevant, but using a / inside an Item.getName is invalid generally (regardless of disk storage location), as that breaks the semantics of Item.getFullName and much else besides.
            Hide
            magnayn magnayn added a comment - - edited

            Correction - fix did work, but I had to re-create the project. I'll see if it works (I get other oddities with projects with %2F in them, like maven fails in really weird places!)

            It occurred to me that maybe the correct place to fix was in Branch (something like branch.getSafeName() or similar) but I was unsure of the implications of the project name != branch name

            Show
            magnayn magnayn added a comment - - edited Correction - fix did work, but I had to re-create the project. I'll see if it works (I get other oddities with projects with %2F in them, like maven fails in really weird places!) It occurred to me that maybe the correct place to fix was in Branch (something like branch.getSafeName() or similar) but I was unsure of the implications of the project name != branch name
            Hide
            jglick Jesse Glick added a comment -

            maven fails in really weird places

            Probably failing to use the correct APIs to encode filenames as file-protocol URLs.

            I was unsure of the implications of the project name != branch name

            Will break the current workarounds for JENKINS-30252 and JENKINS-30595.

            Show
            jglick Jesse Glick added a comment - maven fails in really weird places Probably failing to use the correct APIs to encode filenames as file -protocol URLs. I was unsure of the implications of the project name != branch name Will break the current workarounds for JENKINS-30252 and JENKINS-30595 .
            Hide
            jglick Jesse Glick added a comment -

            FTR I talked with Stephen Connolly who said that he was not positive this was ever properly fixed in branch-api + literate. MultiBranchProject.getItem path-decodes the Item.name, allowing Item.fullName etc. to be meaningful, though it has a hack to deal with browsers which double-decode paths or something like that. (Meaning a functional test may not suffice: also need an acceptance test verifying that a real browser + servlet container can deal with the resulting mess!) Given that the Item.name is encoded, I am not sure why getRootDirFor redundantly encodes it; that may have been the source of your problem after restart. Note also calls to Util.rawEncode from LiterateBranchProject.getRootDir and LiterateEnvironmentBuild.getShortUrl. Also CC Vincent Latombe.

            Show
            jglick Jesse Glick added a comment - FTR I talked with Stephen Connolly who said that he was not positive this was ever properly fixed in branch-api + literate . MultiBranchProject.getItem path-decodes the Item.name , allowing Item.fullName etc. to be meaningful, though it has a hack to deal with browsers which double-decode paths or something like that. (Meaning a functional test may not suffice: also need an acceptance test verifying that a real browser + servlet container can deal with the resulting mess!) Given that the Item.name is encoded, I am not sure why getRootDirFor redundantly encodes it; that may have been the source of your problem after restart. Note also calls to Util.rawEncode from LiterateBranchProject.getRootDir and LiterateEnvironmentBuild.getShortUrl . Also CC Vincent Latombe .
            Hide
            vlatombe Vincent Latombe added a comment -

            I don't remember such issues when using literate. so I think it handles it properly.

            Show
            vlatombe Vincent Latombe added a comment - I don't remember such issues when using literate. so I think it handles it properly.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/jenkins/branch/Branch.java
            src/main/java/jenkins/branch/BranchProjectFactory.java
            src/main/java/jenkins/branch/MultiBranchProject.java
            http://jenkins-ci.org/commit/branch-api-plugin/02dc202f355e2a6b28d0069882614bee6c620bae
            Log:
            JENKINS-30744 Introduced Branch.encodedName, which factories are expected to use for Item.name.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/jenkins/branch/Branch.java src/main/java/jenkins/branch/BranchProjectFactory.java src/main/java/jenkins/branch/MultiBranchProject.java http://jenkins-ci.org/commit/branch-api-plugin/02dc202f355e2a6b28d0069882614bee6c620bae Log: JENKINS-30744 Introduced Branch.encodedName, which factories are expected to use for Item.name.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/jenkins/branch/Branch.java
            src/main/java/jenkins/branch/BranchProjectFactory.java
            src/main/java/jenkins/branch/MultiBranchProject.java
            http://jenkins-ci.org/commit/branch-api-plugin/9c4bd01e48bcb749f65bcde1f8bc2a9c12c23066
            Log:
            Merge pull request #18 from jglick/slashy-branches-JENKINS-30744

            JENKINS-30744 Introduced Branch.encodedName

            Compare: https://github.com/jenkinsci/branch-api-plugin/compare/4d7ce28aedda...9c4bd01e48bc

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/jenkins/branch/Branch.java src/main/java/jenkins/branch/BranchProjectFactory.java src/main/java/jenkins/branch/MultiBranchProject.java http://jenkins-ci.org/commit/branch-api-plugin/9c4bd01e48bcb749f65bcde1f8bc2a9c12c23066 Log: Merge pull request #18 from jglick/slashy-branches- JENKINS-30744 JENKINS-30744 Introduced Branch.encodedName Compare: https://github.com/jenkinsci/branch-api-plugin/compare/4d7ce28aedda...9c4bd01e48bc
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            pom.xml
            src/main/java/org/cloudbees/literate/jenkins/LiterateBranchProject.java
            http://jenkins-ci.org/commit/literate-plugin/a9499bc34ee4201b0ed21c09dbfe008c2ef52942
            Log:
            JENKINS-30744 Updated to use Branch.encodedName.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: pom.xml src/main/java/org/cloudbees/literate/jenkins/LiterateBranchProject.java http://jenkins-ci.org/commit/literate-plugin/a9499bc34ee4201b0ed21c09dbfe008c2ef52942 Log: JENKINS-30744 Updated to use Branch.encodedName.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            pom.xml
            src/main/java/org/cloudbees/literate/jenkins/LiterateBranchProject.java
            http://jenkins-ci.org/commit/literate-plugin/d72b8c94a1d99ae88e6b83641c0cd0b97061b246
            Log:
            Merge pull request #8 from jglick/slashy-branches-JENKINS-30744

            JENKINS-30744 Updated to use Branch.encodedName

            Compare: https://github.com/jenkinsci/literate-plugin/compare/677e289761ce...d72b8c94a1d9

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: pom.xml src/main/java/org/cloudbees/literate/jenkins/LiterateBranchProject.java http://jenkins-ci.org/commit/literate-plugin/d72b8c94a1d99ae88e6b83641c0cd0b97061b246 Log: Merge pull request #8 from jglick/slashy-branches- JENKINS-30744 JENKINS-30744 Updated to use Branch.encodedName Compare: https://github.com/jenkinsci/literate-plugin/compare/677e289761ce...d72b8c94a1d9
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            multibranch/pom.xml
            multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java
            multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
            http://jenkins-ci.org/commit/workflow-plugin/a0b21acecfabf1b0f19cc0193f6f49796b42910a
            Log:
            JENKINS-30744 Updated to use Branch.encodedName, and added functional test.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: multibranch/pom.xml multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java http://jenkins-ci.org/commit/workflow-plugin/a0b21acecfabf1b0f19cc0193f6f49796b42910a Log: JENKINS-30744 Updated to use Branch.encodedName, and added functional test.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            multibranch/pom.xml
            multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java
            multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
            http://jenkins-ci.org/commit/workflow-plugin/a8ff4ecf71b5d4d9895cb5b3cd53f7916000e11d
            Log:
            Merge pull request #243 from jglick/slashy-branches-JENKINS-30744

            JENKINS-30744 Updated to use Branch.encodedName

            Compare: https://github.com/jenkinsci/workflow-plugin/compare/b5082ba5b09c...a8ff4ecf71b5

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: multibranch/pom.xml multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java http://jenkins-ci.org/commit/workflow-plugin/a8ff4ecf71b5d4d9895cb5b3cd53f7916000e11d Log: Merge pull request #243 from jglick/slashy-branches- JENKINS-30744 JENKINS-30744 Updated to use Branch.encodedName Compare: https://github.com/jenkinsci/workflow-plugin/compare/b5082ba5b09c...a8ff4ecf71b5
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
            http://jenkins-ci.org/commit/workflow-plugin/fca115faca36a62968c1500ffd89e06b982a56f4
            Log:
            Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603.
            (Multibranch Git webhooks rely only on the url parameter to detect changes.)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java http://jenkins-ci.org/commit/workflow-plugin/fca115faca36a62968c1500ffd89e06b982a56f4 Log: Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603 . (Multibranch Git webhooks rely only on the url parameter to detect changes.)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
            http://jenkins-ci.org/commit/workflow-plugin/521099bd8a31963ea6149de0a4f4ab9fe5617e97
            Log:
            Merge pull request #276 from jglick/test-JENKINS-29603

            Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603

            Compare: https://github.com/jenkinsci/workflow-plugin/compare/3840d8ca847e...521099bd8a31

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java http://jenkins-ci.org/commit/workflow-plugin/521099bd8a31963ea6149de0a4f4ab9fe5617e97 Log: Merge pull request #276 from jglick/test- JENKINS-29603 Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603 Compare: https://github.com/jenkinsci/workflow-plugin/compare/3840d8ca847e...521099bd8a31
            Hide
            jajansen Jan Arend Jansen added a comment - - edited

            The encoded path seems to create problems on windows when using 'bat'.

            Any branch that has a slash in it for example ('feature/myfeature') will get an encoded path on windows like <workspace_path>/feature%2Fmyfeature@script and <workspace_path>/feature%2Fmyfeature.

            When I apply the following workflow in a multibranch job in a root file 'Jenkinsfile', Jenkins hangs forever on the bat command

            node {
                stage 'Checkout'
                checkout scm
                stage 'Build'
                bat "echo 'test'"
            }
            

            When I add this same script on the 'master' branch everything is fine.

            Environment:
            Windows 8.1
            Jenkins 1.643
            Workflow Plugin 1.12

            Show
            jajansen Jan Arend Jansen added a comment - - edited The encoded path seems to create problems on windows when using 'bat'. Any branch that has a slash in it for example ('feature/myfeature') will get an encoded path on windows like <workspace_path>/feature%2Fmyfeature@script and <workspace_path>/feature%2Fmyfeature . When I apply the following workflow in a multibranch job in a root file 'Jenkinsfile', Jenkins hangs forever on the bat command node { stage 'Checkout' checkout scm stage 'Build' bat "echo 'test' " } When I add this same script on the 'master' branch everything is fine. Environment: Windows 8.1 Jenkins 1.643 Workflow Plugin 1.12
            Hide
            jajansen Jan Arend Jansen added a comment -

            Confirmed that it is related to the encoding of the filepath by dirty-hacking the Branch API Plugin

             public String getEncodedName() {
                    return Util.rawEncode(getName()).replace("%2F","_");
                }
            

            After updating the plugin to this hacked one on my Windows Jenkins installation, the branches with / in the name work in the MultiBranch Workflow Project

            Show
            jajansen Jan Arend Jansen added a comment - Confirmed that it is related to the encoding of the filepath by dirty-hacking the Branch API Plugin public String getEncodedName() { return Util.rawEncode(getName()).replace( "%2F" , "_" ); } After updating the plugin to this hacked one on my Windows Jenkins installation, the branches with / in the name work in the MultiBranch Workflow Project
            Hide
            gitt Slawa Giterman added a comment -

            I have a problem which might be related to this one. I use SVN with trunk and branches named like featureBranch1, featureBranch1, etc. (i.e. without / character in branch names).

            I try simple Jenkinsfile:

            env.MY_VAR1 = 'ABC'
            node('NODE_1') {
                bat "echo MY_VAR1 is set to %MY_VAR1% on NODE_1"
            }
            

            When I create a multibranch job, this script executes correctly for the trunk but always hangs for branches. Last what I see in console is:

            [Workflow] bat
            [branches%2FfeatureBranch1] Running batch script
            

            More details:
            In workspace folder (on Windows slave) folders named like branches%2FfeatureBranch1\.jenkins-05123d82 are created for each job. They have only 2 files

            • jenkins-main.bat:
              echo MY_VAR1 is set to %MY_VAR1% on NODE_1
              
            • and jenkins-wrap.bat:
              cmd /c "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-main.bat" > "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-log.txt" 2>&1
              echo %ERRORLEVEL% > "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-result.txt"
              

            I use Jenkins 1.625.3, Workflow plug-in 1.12. Can %2F in folder name cause the batch command hanging?

            Show
            gitt Slawa Giterman added a comment - I have a problem which might be related to this one. I use SVN with trunk and branches named like featureBranch1, featureBranch1, etc. (i.e. without / character in branch names). I try simple Jenkinsfile: env.MY_VAR1 = 'ABC' node( 'NODE_1' ) { bat "echo MY_VAR1 is set to %MY_VAR1% on NODE_1" } When I create a multibranch job , this script executes correctly for the trunk but always hangs for branches. Last what I see in console is: [Workflow] bat [branches%2FfeatureBranch1] Running batch script More details: In workspace folder (on Windows slave) folders named like branches%2FfeatureBranch1\.jenkins-05123d82 are created for each job. They have only 2 files jenkins-main.bat: echo MY_VAR1 is set to %MY_VAR1% on NODE_1 and jenkins-wrap.bat: cmd /c "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-main.bat" > "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-log.txt" 2>&1 echo %ERRORLEVEL% > "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-result.txt" I use Jenkins 1.625.3, Workflow plug-in 1.12. Can %2F in folder name cause the batch command hanging?
            Hide
            cbennett Colin Bennett added a comment -

            @gitt I see the same thing.

            Running on master in C:\Jenkins\Home\workspace\dgx_mcpu\topic%2Fpipelinetest
            [Pipeline] node {
            [Pipeline] echo
            Posting GitLab commit status: pending
            [Pipeline] Enforce time limit : Start
            [Pipeline] timeout {
            [Pipeline] isUnix
            [Pipeline] bat
            [topic%2Fpipelinetest] Running batch script
            Sending interrupt signal to process
            Sending interrupt signal to process
            Sending interrupt signal to process
            Click here to forcibly terminate running steps
            Aborted by Colin D Bennett
            Sending interrupt signal to process
            Click here to forcibly terminate running steps
            Click here to forcibly terminate running steps
            
            Show
            cbennett Colin Bennett added a comment - @gitt I see the same thing. Running on master in C:\Jenkins\Home\workspace\dgx_mcpu\topic%2Fpipelinetest [Pipeline] node { [Pipeline] echo Posting GitLab commit status: pending [Pipeline] Enforce time limit : Start [Pipeline] timeout { [Pipeline] isUnix [Pipeline] bat [topic%2Fpipelinetest] Running batch script Sending interrupt signal to process Sending interrupt signal to process Sending interrupt signal to process Click here to forcibly terminate running steps Aborted by Colin D Bennett Sending interrupt signal to process Click here to forcibly terminate running steps Click here to forcibly terminate running steps
            Hide
            jglick Jesse Glick added a comment -

            This issue is fixed. If there are problems with Windows bat steps that are related to particular characters, file them in the durable-task plugin with workflow label and link to this issue.

            Show
            jglick Jesse Glick added a comment - This issue is fixed. If there are problems with Windows bat steps that are related to particular characters, file them in the durable-task plugin with workflow label and link to this issue.
            Hide
            jajansen Jan Arend Jansen added a comment -

            Done

            Show
            jajansen Jan Arend Jansen added a comment - Done
            Hide
            alex_ouzounis Alex Ouzounis added a comment - - edited

            Jesse Glick has this been released in the workflow (pipeline the past month) plugin ?

            Alex

            Show
            alex_ouzounis Alex Ouzounis added a comment - - edited Jesse Glick has this been released in the workflow (pipeline the past month) plugin ? Alex
            Hide
            alex_ouzounis Alex Ouzounis added a comment -

            Jesse Glick

            So my understanding from the above is that for multi branch this is considered resolved but if we want to make use of multi branch with gitflow on Windows jenkins slaves where our git repo has branches like feature/XXX hotfix/XXX etc we have to either fork and hack this plugin or hope someone resolves it in the durable task ?

            I am just trying to understand the possible ways forward as we really like the workflow plugin but can't use it basically due to the above.

            Alex

            Show
            alex_ouzounis Alex Ouzounis added a comment - Jesse Glick So my understanding from the above is that for multi branch this is considered resolved but if we want to make use of multi branch with gitflow on Windows jenkins slaves where our git repo has branches like feature/XXX hotfix/XXX etc we have to either fork and hack this plugin or hope someone resolves it in the durable task ? I am just trying to understand the possible ways forward as we really like the workflow plugin but can't use it basically due to the above. Alex
            Hide
            joostvdg Joost van der Griendt added a comment -

            Alex Ouzounis
            We've run into the same issue with our Windows slaves.
            I've been able to make a simple workaround, although still a workaround, that allows us to continue to use the multibranch feature.

            def getWorkspace() {
                pwd().replace("%2F", "_")
            }
            
            node {
              ws(getWorkspace()) {
               ....
              }
            }
                       
            
            Show
            joostvdg Joost van der Griendt added a comment - Alex Ouzounis We've run into the same issue with our Windows slaves. I've been able to make a simple workaround, although still a workaround, that allows us to continue to use the multibranch feature. def getWorkspace() { pwd().replace( "%2F" , "_" ) } node { ws(getWorkspace()) { .... } }
            Hide
            alex_ouzounis Alex Ouzounis added a comment -

            now this is pure genius! We will try this ASAP Also shows how flexible workflow is with the JenkinsFile definitions though.

            Many thanks Joost van der Griendt !! We owe you a beer !

            Show
            alex_ouzounis Alex Ouzounis added a comment - now this is pure genius! We will try this ASAP Also shows how flexible workflow is with the JenkinsFile definitions though. Many thanks Joost van der Griendt !! We owe you a beer !
            Hide
            mscharp Michael Scharp added a comment - - edited

            There is another variant to this problem that i am experiencing on a Linux box.

            I have a maven project that uses Jaxb to load a schema document and generate something from it. This project fails with:

            [ERROR] [ERROR] schema_reference.4: Failed to read schema document 'file:/apps/.jenkins/workspace/project/feature%2FmyFeature/somePath/src/main/resources/MySchema.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
            [ERROR] unknown location
            [ERROR]

            Likewise, I have a project with unit tests that read in a file from the filesystem, make modifications, and write it back out. However, these test fail seemingly because the file can't be written out.

            When I change my feature branch name from "feature/myFeature" to just "myFeature" the projects succeed in building.

            I will create a separate issue since this one has been resolved.

            Show
            mscharp Michael Scharp added a comment - - edited There is another variant to this problem that i am experiencing on a Linux box. I have a maven project that uses Jaxb to load a schema document and generate something from it. This project fails with: [ERROR] [ERROR] schema_reference.4: Failed to read schema document 'file:/apps/.jenkins/workspace/project/feature%2FmyFeature/somePath/src/main/resources/MySchema.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>. [ERROR] unknown location [ERROR] Likewise, I have a project with unit tests that read in a file from the filesystem, make modifications, and write it back out. However, these test fail seemingly because the file can't be written out. When I change my feature branch name from "feature/myFeature" to just "myFeature" the projects succeed in building. I will create a separate issue since this one has been resolved.
            Hide
            alex_ouzounis Alex Ouzounis added a comment -

            Joost van der Griendt works as expected, the problem has now gone away. many thanks for sharing your workaround

            Show
            alex_ouzounis Alex Ouzounis added a comment - Joost van der Griendt works as expected, the problem has now gone away. many thanks for sharing your workaround
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            multibranch/pom.xml
            multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java
            multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
            http://jenkins-ci.org/commit/workflow-multibranch-plugin/b11fd4d0b9c61682df0f4eece40acd8e0029148c
            Log:
            JENKINS-30744 Updated to use Branch.encodedName, and added functional test.
            Originally-Committed-As: a0b21acecfabf1b0f19cc0193f6f49796b42910a

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: multibranch/pom.xml multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java http://jenkins-ci.org/commit/workflow-multibranch-plugin/b11fd4d0b9c61682df0f4eece40acd8e0029148c Log: JENKINS-30744 Updated to use Branch.encodedName, and added functional test. Originally-Committed-As: a0b21acecfabf1b0f19cc0193f6f49796b42910a
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
            http://jenkins-ci.org/commit/workflow-multibranch-plugin/c9738eb45a25c1ae6a6e55df0d9688654d328725
            Log:
            Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603.
            (Multibranch Git webhooks rely only on the url parameter to detect changes.)
            Originally-Committed-As: fca115faca36a62968c1500ffd89e06b982a56f4

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java http://jenkins-ci.org/commit/workflow-multibranch-plugin/c9738eb45a25c1ae6a6e55df0d9688654d328725 Log: Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603 . (Multibranch Git webhooks rely only on the url parameter to detect changes.) Originally-Committed-As: fca115faca36a62968c1500ffd89e06b982a56f4
            Hide
            franzbecker Franz Becker added a comment - - edited

            I'm still encountering this issue (using Jenkins 2.6 and all plugins up-to-date). The workaround of Joost van der Griendt works but this issues is quite frustrating when starting with pipelines. Especially when having a build that works perfectly as a "standalone" pipeline and fails in the multibranch environment (with Maven plugins that claim that the path cannot be found as they use java.net.URL).

            Edit: Just read your comment on JENKINS-30744. I agree that the bug is in the Maven plugin (so far I encountered 2 such plugins). Nevertheless this is frustrating when migrating to multibranch pipeline builds.

            Show
            franzbecker Franz Becker added a comment - - edited I'm still encountering this issue (using Jenkins 2.6 and all plugins up-to-date). The workaround of Joost van der Griendt works but this issues is quite frustrating when starting with pipelines. Especially when having a build that works perfectly as a "standalone" pipeline and fails in the multibranch environment (with Maven plugins that claim that the path cannot be found as they use java.net.URL). Edit: Just read your comment on JENKINS-30744 . I agree that the bug is in the Maven plugin (so far I encountered 2 such plugins). Nevertheless this is frustrating when migrating to multibranch pipeline builds.
            Hide
            dodoent Nenad Miksa added a comment -

            We are also having this issue and are not using Maven at all. We are invoking NDK-build directly from shell and apparently ndk-build cannot work with paths containing %. Also happens when invoking MSBuild. We are currently using trick to allocate a new workspace with %2F replaced with _, however when job is finally removed (after pull request has been merged and branch deleted), the workspace remains, wasting valuable disk space.

            Show
            dodoent Nenad Miksa added a comment - We are also having this issue and are not using Maven at all. We are invoking NDK-build directly from shell and apparently ndk-build cannot work with paths containing %. Also happens when invoking MSBuild. We are currently using trick to allocate a new workspace with %2F replaced with _, however when job is finally removed (after pull request has been merged and branch deleted), the workspace remains, wasting valuable disk space.
            Hide
            razvan_botez Razvan Botez added a comment -

            Still encountering this in Jenkins 2.10 with all plugins up to date. The workaround works, but it's only a workaround and we have to have separate housekeeping tasks for removing old left-over stuff in the workspace.

            Show
            razvan_botez Razvan Botez added a comment - Still encountering this in Jenkins 2.10 with all plugins up to date. The workaround works, but it's only a workaround and we have to have separate housekeeping tasks for removing old left-over stuff in the workspace.
            Hide
            razvan_botez Razvan Botez added a comment -

            Still happening.

            Show
            razvan_botez Razvan Botez added a comment - Still happening.
            Hide
            koko Konstantinos Kostarellis added a comment - - edited

            Got an issue with branches containing slashes calling
            ant.axisWsdl2java
            from within an gradle script.
            OS: ubuntu 14.04

            the org.apache.axis.tools.ant.wsdl.Wsdl2javaAntTask somehow decodes the path and throws an exception in the next step of not being able to find feature/jenkins .. because it is feature%2Fjenkins

            Show
            koko Konstantinos Kostarellis added a comment - - edited Got an issue with branches containing slashes calling ant.axisWsdl2java from within an gradle script. OS: ubuntu 14.04 the org.apache.axis.tools.ant.wsdl.Wsdl2javaAntTask somehow decodes the path and throws an exception in the next step of not being able to find feature/jenkins .. because it is feature%2Fjenkins
            Hide
            parsonsk K P added a comment -

            I tried the workaround above from Joost van der Griendt which worked for CMake (which does not like % in paths) but then my tests, which use Matlab, fail as now there is a '@' in the path instead (Matlab will also fail with % in the path) A real fix for this would be really useful, as now I'm having to copy everything to a temp folder to run everything (and clean-up is a pain).

            Show
            parsonsk K P added a comment - I tried the workaround above from Joost van der Griendt which worked for CMake (which does not like % in paths) but then my tests, which use Matlab, fail as now there is a '@' in the path instead (Matlab will also fail with % in the path) A real fix for this would be really useful, as now I'm having to copy everything to a temp folder to run everything (and clean-up is a pain).
            Hide
            jglick Jesse Glick added a comment -

            Please refer to JENKINS-34564 for the request for alternate workspace naming.

            Show
            jglick Jesse Glick added a comment - Please refer to JENKINS-34564 for the request for alternate workspace naming.
            Hide
            yakobe jake bishop added a comment -

            This is not really resolved "fixed", the issue will be resolved when JENKINS-34564 is resolved. Can it be resolved as a duplicate?

            Show
            yakobe jake bishop added a comment - This is not really resolved "fixed", the issue will be resolved when JENKINS-34564 is resolved. Can it be resolved as a duplicate?
            Hide
            joth Jonas Thyregod added a comment - - edited

            Will this codefix also be applied to the old "multi-branch-project-plugin" (and not only to "pipeline multi branch" plugin) ??

            A lot of issues (etc. JENKINS-37677 and JENKINS-32239 etc) are regarding the same error - but just for the old "multi-branch-project-plugin" plugin...
            All of these issues are marked as duplicated of this fix (which is for the new "pipeline multi branch" plugin).

            Show
            joth Jonas Thyregod added a comment - - edited Will this codefix also be applied to the old "multi-branch-project-plugin" (and not only to "pipeline multi branch" plugin) ?? A lot of issues (etc. JENKINS-37677 and JENKINS-32239 etc) are regarding the same error - but just for the old "multi-branch-project-plugin" plugin... All of these issues are marked as duplicated of this fix (which is for the new "pipeline multi branch" plugin).
            Hide
            tyrelh Tyrel Haveman added a comment -

            Changing my workspace name from workspace/foo/bar%2Fstuff to workspace/foo_bar-stuff-ISUDFHISDHFG345934SDHFSDYG82734SDFH has created a lot of problems because it's much longer. I don't find this to be an acceptable solution.

            %2F wasn't causing any problems for our org. Why not actually use a / so we can have the workspaces organized with subdirectories? workspace/foo/bar/stuff would be great. Or, alternately, workspace/foo/bar_stuff would be fine. But workspace/foo_bar-stuff-ISUDFHISDHFG345934SDHFSDYG82734SDFH is not.

            As noted in JENKINS-38706, this is causing files within them to have extremely long paths such that Windows won't actually let us delete them (Jenkins, CMD, and Windows Explorer all fail to delete them. We have to rename them to shorter names first), which ends up causing us to run out of disk space because our branch builds never get deleted.

            Show
            tyrelh Tyrel Haveman added a comment - Changing my workspace name from workspace/foo/bar%2Fstuff to workspace/foo_bar-stuff-ISUDFHISDHFG345934SDHFSDYG82734SDFH has created a lot of problems because it's much longer. I don't find this to be an acceptable solution. %2F wasn't causing any problems for our org. Why not actually use a / so we can have the workspaces organized with subdirectories? workspace/foo/bar/stuff would be great. Or, alternately, workspace/foo/bar_stuff would be fine. But workspace/foo_bar-stuff-ISUDFHISDHFG345934SDHFSDYG82734SDFH is not. As noted in JENKINS-38706 , this is causing files within them to have extremely long paths such that Windows won't actually let us delete them (Jenkins, CMD, and Windows Explorer all fail to delete them. We have to rename them to shorter names first), which ends up causing us to run out of disk space because our branch builds never get deleted.
            Hide
            nerumo Marco Bachmann added a comment -

            Same for me. Even the Windows 10 long path trick is not working since applications like NuGet are not prepared for long paths. So I can't use the multi branch workflow anymore.

            Show
            nerumo Marco Bachmann added a comment - Same for me. Even the Windows 10 long path trick is not working since applications like NuGet are not prepared for long paths. So I can't use the multi branch workflow anymore.
            Hide
            git Thomas Gimpel added a comment - - edited

            The provided fix does not work on Windows using Visual Studio. The path gets too long, so that VS cannot read/write files in deep directory hierarchies. What is the extra suffix good for? Is it a "path friendly" version of the whole branch name? If so, why not just translate the "/" or other forbidden characters into to a short sequence of valid characters? E.g.

            /

            becomes

            _2f

            ,

            %

            becomes

            _25

            and

            _

            becomes

            _5f

            . This is almost the same as the previous solution "%2f". But when using the underline as escape character no "smart" software like VS or cmd.exe will try to interpret the % character with all these seen funny results.

            Show
            git Thomas Gimpel added a comment - - edited The provided fix does not work on Windows using Visual Studio. The path gets too long, so that VS cannot read/write files in deep directory hierarchies. What is the extra suffix good for? Is it a "path friendly" version of the whole branch name? If so, why not just translate the "/" or other forbidden characters into to a short sequence of valid characters? E.g. / becomes _2f , % becomes _25 and _ becomes _5f . This is almost the same as the previous solution "%2f". But when using the underline as escape character no "smart" software like VS or cmd.exe will try to interpret the % character with all these seen funny results.
            Hide
            creste3 John Smith added a comment -

            For anyone that needs escaped file paths and doesn't want them to be too long, I submitted a Pull Request to reduce the minimum length of workspace directories from 54 characters to 12 characters when

            -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX
            

            is set to a value between 1 and 54. The pull request is at https://github.com/jenkinsci/branch-api-plugin/pull/62.

            Show
            creste3 John Smith added a comment - For anyone that needs escaped file paths and doesn't want them to be too long, I submitted a Pull Request to reduce the minimum length of workspace directories from 54 characters to 12 characters when -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX is set to a value between 1 and 54. The pull request is at https://github.com/jenkinsci/branch-api-plugin/pull/62 .
            Hide
            stephenconnolly Stephen Connolly added a comment -

            branch-api-plugin 2.0.7 has https://github.com/jenkinsci/branch-api-plugin/pull/62 merged

            Show
            stephenconnolly Stephen Connolly added a comment - branch-api-plugin 2.0.7 has https://github.com/jenkinsci/branch-api-plugin/pull/62 merged
            Hide
            pogorman Philip O'Gorman added a comment -

            Stephen Connolly I've updates all my plugins and jenkins to the latest. As far as I can tell by build job with the "/" in the branch name completed, but I cannot browse to the job:

            I get the following error:

            404 - File or directory not found.

            The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.

            The url is in the following format:

            https://myserver.ca/job/myjobname/job/hotfix%252Fbranchname/

            This only happens when with branches that contain "/". I've tried

            -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 and

            -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=1

            What am I missing here?

            Thanks,

             

            Show
            pogorman Philip O'Gorman added a comment - Stephen Connolly I've updates all my plugins and jenkins to the latest. As far as I can tell by build job with the "/" in the branch name completed, but I cannot browse to the job: I get the following error: 404 - File or directory not found. The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable. The url is in the following format: https://myserver.ca/job/myjobname/job/hotfix%252Fbranchname/ This only happens when with branches that contain "/". I've tried -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 and -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=1 What am I missing here? Thanks,  
            Hide
            thois Thijs Borst added a comment -

            Philip O'Gorman I ran into that issue too, are you running IIS as a reverse proxy? Try setting the allow double escaping (Go to Request Filtering, go to Edit feature settings and tick the checkbox). That fixed it for me.

            What is still a problem though is building branches with a slash using msbuild.

            I've been searching and searching, but can't find a solution. I've got a pipeline script running my build. The develop branch builds fine, but now I have a Release/1.5.0.4-hotfix branch. This is what happens (among others):

             
            C:\Jenkins\workspace****\Releases%2F1.5.0.4-hotfix*******.csproj" (default target) (3:3) ->
            (CoreCompile target) ->
            CSC : error CS0006: Metadata file 'C:\Jenkins\workspace*****\Releases/1.5.0.4-hotfix\packages\Autofac.4.0.1\lib\net45\Autofac.dll' could not be found [C:\Jenkins\workspace\*****\Releases%2F1.5.0.4-hotfix\*****\*****.csproj]
            CSC : error CS0006: Metadata file 'C:\Jenkins\workspace*****\Releases/1.5.0.4-hotfix_references\CookComputing.XmlRpcV2.dll' could not be found [C:\Jenkins\workspace\*****\Releases%2F1.5.0.4-hotfix\*****\*****.csproj]
             

            Some of the build goes fine... it checks out the code, it does the nuget restore, but during the build inside msbuild it changes the encoding back and voila there is the slash again... So when is this encoding stuff finally being fixed? Just replace the special characters on the disk. Somewhere it says that you do, but it really doesn't happen...

             

            I need that encoding gone in my folders it suits no purpose and doesn't make it more readable either.

            Show
            thois Thijs Borst added a comment - Philip O'Gorman I ran into that issue too, are you running IIS as a reverse proxy? Try setting the allow double escaping (Go to Request Filtering, go to Edit feature settings and tick the checkbox). That fixed it for me. What is still a problem though is building branches with a slash using msbuild. I've been searching and searching, but can't find a solution. I've got a pipeline script running my build. The develop branch builds fine, but now I have a Release/1.5.0.4-hotfix branch. This is what happens (among others):   C:\Jenkins\workspace**** \Releases%2F1.5.0.4-hotfix* ** * ***.csproj" (default target) (3:3) -> (CoreCompile target) -> CSC : error CS0006: Metadata file 'C:\Jenkins\workspace*****\Releases/1.5.0.4-hotfix\packages\Autofac.4.0.1\lib\net45\Autofac.dll' could not be found [C:\Jenkins\workspace\*****\Releases%2F1.5.0.4-hotfix\*****\*****.csproj] CSC : error CS0006: Metadata file 'C:\Jenkins\workspace*****\Releases/1.5.0.4-hotfix_references\CookComputing.XmlRpcV2.dll' could not be found [C:\Jenkins\workspace\*****\Releases%2F1.5.0.4-hotfix\*****\*****.csproj]   Some of the build goes fine... it checks out the code, it does the nuget restore, but during the build inside msbuild it changes the encoding back and voila there is the slash again... So when is this encoding stuff finally being fixed? Just replace the special characters on the disk. Somewhere it says that you do, but it really doesn't happen...   I need that encoding gone in my folders it suits no purpose and doesn't make it more readable either.
            Hide
            varun7447 Varun Reddy added a comment -

            Thijs Borst is this issue resolved for you? I am facing the same exact same issue. The / in the branch name(Releases/test-db) for branches is not working. Any help or workaround that you are aware of ?

            Show
            varun7447 Varun Reddy added a comment - Thijs Borst is this issue resolved for you? I am facing the same exact same issue. The / in the branch name(Releases/test-db) for branches is not working. Any help or workaround that you are aware of ?
            Hide
            thois Thijs Borst added a comment -

            Varun Reddy No, we've adjusted the way we name our branches, which is a pain.

            Show
            thois Thijs Borst added a comment - Varun Reddy No, we've adjusted the way we name our branches, which is a pain.
            Hide
            varun7447 Varun Reddy added a comment -

            Is this issue being addressed in the coming release? Or is this has to be fixed by the build tools? In this msbuild and nuget.exe?

            Show
            varun7447 Varun Reddy added a comment - Is this issue being addressed in the coming release? Or is this has to be fixed by the build tools? In this msbuild and nuget.exe?
            Hide
            pogorman Philip O'Gorman added a comment -

            I couldn't figure this out either. If we use gitflow - we have to make sure that we don't use the default branch names:

            feature/myfeat

            bug/themess

            I don't know why this issue is marked as closed? What are we missing here?

             

            Show
            pogorman Philip O'Gorman added a comment - I couldn't figure this out either. If we use gitflow - we have to make sure that we don't use the default branch names: feature/myfeat bug/themess I don't know why this issue is marked as closed? What are we missing here?  
            Hide
            varun7447 Varun Reddy added a comment -

            Yes we are using git-flow and we are having same above issue as Philip O'Gorman . Is there another defect that we need to track this issue? 

            Show
            varun7447 Varun Reddy added a comment - Yes we are using git-flow and we are having same above issue as Philip O'Gorman . Is there another defect that we need to track this issue? 
            Hide
            nemligtim Timothy Harris added a comment -

            Why is this issue closed? I mean any MSBuild run will fail with this conversion of "/". Many branching strategies include branch names that have a "/" in them. Pretty standard for Bitbucket installations as it defaults to.

            feature/

            hotfix/

            bugfix/

            release/

             

            Please reopen and convert to something less painful! 

            Show
            nemligtim Timothy Harris added a comment - Why is this issue closed? I mean any MSBuild run will fail with this conversion of "/". Many branching strategies include branch names that have a "/" in them. Pretty standard for Bitbucket installations as it defaults to. feature/ hotfix/ bugfix/ release/   Please reopen and convert to something less painful! 
            Hide
            markewaite Mark Waite added a comment -

            Timothy Harris please refer to the duplicate bug JENKINS-34564 which describes more details. This is a duplicate bug and the project prefers to close duplicates so that they don't clutter searches.

            Show
            markewaite Mark Waite added a comment - Timothy Harris please refer to the duplicate bug JENKINS-34564 which describes more details. This is a duplicate bug and the project prefers to close duplicates so that they don't clutter searches.
            Hide
            varun7447 Varun Reddy added a comment -

            The bug JENKINS-34564 also does not work for MSBuild. It fixes the longpath but not the issues with /

            Show
            varun7447 Varun Reddy added a comment - The bug JENKINS-34564 also does not work for MSBuild. It fixes the longpath but not the issues with /
            Hide
            nemligtim Timothy Harris added a comment - - edited

            I got around this by giving it a specific workspace.

            agent {
                node {
                    label 'windows'
                    // Todo: This s really bad but Jenkins Multibranch is being very bad with very long paths being generated!
                    customWorkspace "${JOB_NAME}"
                }
            }

            I got around the conversion of "/" by changing the branching strategy in Bitbucket to feature-, hotfix-, release- instead of feature/, hotfix/, release/.

            Both of these are pretty frustrating. Especially the "/". 

            Show
            nemligtim Timothy Harris added a comment - - edited I got around this by giving it a specific workspace. agent {     node {         label 'windows'         // Todo: This s really bad but Jenkins Multibranch is being very bad with very long paths being generated!         customWorkspace "${JOB_NAME}"     } } I got around the conversion of "/" by changing the branching strategy in Bitbucket to feature-, hotfix-, release- instead of feature/, hotfix/, release/. Both of these are pretty frustrating. Especially the "/". 
            Hide
            varun7447 Varun Reddy added a comment -

            I am currently using the Branching strategy to include - instead of / . But the default behavior in setting up git flow branches are / .

            Show
            varun7447 Varun Reddy added a comment - I am currently using the Branching strategy to include - instead of / . But the default behavior in setting up git flow branches are / .

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                magnayn magnayn
              • Votes:
                14 Vote for this issue
                Watchers:
                47 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: