Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20086

Allow disabling use of default exclude patterns (.git, .svn, etc.) for archiving

    Details

    • Similar Issues:

      Description

      The plugin available to archive artifacts doesn't allow to chose to archive hidden files/directories like .git, .svn, etc. This feature will help Continuous Integration teams to upstream the .git information to be used for example to get last commit hash, repository server, etc.
      Today, the plugin that copy the workspace is not useful when you do not need more than that artifacts, ans, in some cases, the repository is big.
      Maybe, the plugin can be changed to allow the copy, by selecting the file types.
      I am talking about to change the plugin "Archive Artifact" and "Copy Artifact".

        Attachments

          Issue Links

            Activity

            eduardodias Eduardo Dias created issue -
            Hide
            mbaker000 mbaker000 added a comment -

            While it's a fair amount of hackery you can do this to work around this limitation:

            1. use 7-Zip to package up your workspace
            2. archive the zip file as an artifact
            3. copy the zip file as an artifact to your next pipeline build
            4. unzip the zip file
            5. do your normal build steps

            Show
            mbaker000 mbaker000 added a comment - While it's a fair amount of hackery you can do this to work around this limitation: 1. use 7-Zip to package up your workspace 2. archive the zip file as an artifact 3. copy the zip file as an artifact to your next pipeline build 4. unzip the zip file 5. do your normal build steps
            Hide
            momoto mike omoto added a comment -

            You can also emulate this behavior by using the post-build task plugin with the following setup:

            Log Text: Archiving artifacts

            cp -r .git ../builds/$BUILD_NUMBER/archive/.git/
            cp .gitignore ../builds/$BUILD_NUMBER/archive/.gitignore
            cp .gitattributes ../builds/$BUILD_NUMBER/archive/.gitattributes

            Show
            momoto mike omoto added a comment - You can also emulate this behavior by using the post-build task plugin with the following setup: Log Text: Archiving artifacts cp -r .git ../builds/$BUILD_NUMBER/archive/.git/ cp .gitignore ../builds/$BUILD_NUMBER/archive/.gitignore cp .gitattributes ../builds/$BUILD_NUMBER/archive/.gitattributes
            Hide
            lentini Gigi Lentini added a comment -

            I came across the same issue using Jenkins 1.554.1, and the zip/unzip workaround seems to work for the moment.
            But I would rather use a clear jenkins config to do so.

            Do you have any update?

            Thanks.

            Show
            lentini Gigi Lentini added a comment - I came across the same issue using Jenkins 1.554.1, and the zip/unzip workaround seems to work for the moment. But I would rather use a clear jenkins config to do so. Do you have any update? Thanks.
            Hide
            danielbeck Daniel Beck added a comment -

            Jenkins uses Ant to archive files. It excludes the following patterns related to various SCM by default (run in Script Console):

            org.apache.tools.ant.DirectoryScanner.defaultExcludes.each { println it }

            This can be changed globally by running the script below in the Script Console (until the next restart) or by adding a .groovy file with that script in JENKINS_HOME/init.groovy.d/ so this gets loaded upon startup:

            org.apache.tools.ant.DirectoryScanner.defaultExcludes.each { org.apache.tools.ant.DirectoryScanner.removeDefaultExclude(it) }

            I doubt that this is deserving a per-project preference, given the relative obscurity of the requirement. IMO a cleaner solution would be to extract the relevant data during the build and store it outside the SCM metadata folders.

            Show
            danielbeck Daniel Beck added a comment - Jenkins uses Ant to archive files. It excludes the following patterns related to various SCM by default (run in Script Console): org.apache.tools.ant.DirectoryScanner.defaultExcludes.each { println it } This can be changed globally by running the script below in the Script Console (until the next restart) or by adding a .groovy file with that script in JENKINS_HOME/init.groovy.d/ so this gets loaded upon startup: org.apache.tools.ant.DirectoryScanner.defaultExcludes.each { org.apache.tools.ant.DirectoryScanner.removeDefaultExclude(it) } I doubt that this is deserving a per-project preference, given the relative obscurity of the requirement. IMO a cleaner solution would be to extract the relevant data during the build and store it outside the SCM metadata folders.
            Hide
            nithril Nicolas Labrot added a comment -

            relative obscurity of the requirement

            why is it obscurs?

            I have the same issue. First step clean/install the maven project on a private maven repo (to avoid snapshot deps collision between a reentrant pipeline) and archive the workspace. Next steps (test, sonar, deploy on nexus, deploy on tomcat...) reuse this archive instead of rebuilding the project. Sonar steps need the .svn which are not copied by the archive plugin.

            Is this pipeline wrong?

            Show
            nithril Nicolas Labrot added a comment - relative obscurity of the requirement why is it obscurs? I have the same issue. First step clean/install the maven project on a private maven repo (to avoid snapshot deps collision between a reentrant pipeline) and archive the workspace. Next steps (test, sonar, deploy on nexus, deploy on tomcat...) reuse this archive instead of rebuilding the project. Sonar steps need the .svn which are not copied by the archive plugin. Is this pipeline wrong?
            Hide
            danielbeck Daniel Beck added a comment -

            It's not something that can be done out of the box, you need e.g. Copy Artifacts plugin for that.

            Still, it does seem like a somewhat surprising behavior – it's only unsurprising if you know it uses Ant, and that's a build tool. I'm not sure what the best way to introduce an option is though – just clearing the default excludes can well be considered a regression. Per-instance option is too heavy-handed. Per-job option seems to be the only sensible one left. Given the apparent reluctance of others to merge my 'archive only if successful' option in 1.567, I'm not looking forward to an argument about this

            I'm adding this to my list of things I'd like to fix, but anyone feel free to take over.

            Show
            danielbeck Daniel Beck added a comment - It's not something that can be done out of the box, you need e.g. Copy Artifacts plugin for that. Still, it does seem like a somewhat surprising behavior – it's only unsurprising if you know it uses Ant, and that's a build tool. I'm not sure what the best way to introduce an option is though – just clearing the default excludes can well be considered a regression. Per-instance option is too heavy-handed. Per-job option seems to be the only sensible one left. Given the apparent reluctance of others to merge my 'archive only if successful' option in 1.567, I'm not looking forward to an argument about this I'm adding this to my list of things I'd like to fix, but anyone feel free to take over.
            danielbeck Daniel Beck made changes -
            Field Original Value New Value
            Assignee Daniel Beck [ danielbeck ]
            danielbeck Daniel Beck made changes -
            Labels archiving artifact plugin archiving artifact
            Component/s core [ 15593 ]
            Component/s plugin [ 15491 ]
            danielbeck Daniel Beck made changes -
            Summary Improve the archive artifacts plugin to incude hidden files/directories like .git Allow disabling use of default exclude patterns (.git, .svn, etc.)
            Fix Version/s current [ 10162 ]
            Affects Version/s current [ 10162 ]
            Environment Jenkins 1.509.4 LTS
            Ubuntu 12.04 LTS
            Any
            Due Date 2013-10-31
            Hide
            nithril Nicolas Labrot added a comment -

            As a workaround I have remove ".svn" from the defaultExcludes with an init.groovy hook.

            Per job option seems to me the right way. And maybe per copy/archive task (a checkbox to include defaultExcludes).

            Show
            nithril Nicolas Labrot added a comment - As a workaround I have remove ".svn" from the defaultExcludes with an init.groovy hook. Per job option seems to me the right way. And maybe per copy/archive task (a checkbox to include defaultExcludes).
            Hide
            danielbeck Daniel Beck added a comment -

            Right, I meant per post-build step. As there's usually only one, it's effectively "per job".

            I don't know whether the same behavior applies to Copy Artifact plugin. I'd consider that a separate issue though, as it's a different component.

            Show
            danielbeck Daniel Beck added a comment - Right, I meant per post-build step. As there's usually only one, it's effectively "per job". I don't know whether the same behavior applies to Copy Artifact plugin. I'd consider that a separate issue though, as it's a different component.
            Hide
            nithril Nicolas Labrot added a comment - - edited

            I'm giving it a try to pull request a modification of the ArtifactArchiver. I will check the copy artifact plugin.

            Show
            nithril Nicolas Labrot added a comment - - edited I'm giving it a try to pull request a modification of the ArtifactArchiver . I will check the copy artifact plugin.
            danielbeck Daniel Beck made changes -
            Assignee Daniel Beck [ danielbeck ] Nicolas Labrot [ nithril ]
            Hide
            nithril Nicolas Labrot added a comment -

            Artifact archiver issue is fixed on github. I agree with you Daniel, this issue should be split

            Show
            nithril Nicolas Labrot added a comment - Artifact archiver issue is fixed on github. I agree with you Daniel, this issue should be split
            Hide
            danielbeck Daniel Beck added a comment -

            Nicolas' pull request: https://github.com/jenkinsci/jenkins/pull/1287

            Unless someone else merges it, I'm going to let it soak for a few more days to allow others to provide feedback. This week is JUC Boston, next week JUC Berlin, so most reviewers will be distracted right now.

            Show
            danielbeck Daniel Beck added a comment - Nicolas' pull request: https://github.com/jenkinsci/jenkins/pull/1287 Unless someone else merges it, I'm going to let it soak for a few more days to allow others to provide feedback. This week is JUC Boston, next week JUC Berlin, so most reviewers will be distracted right now.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: nlabrot
            Path:
            core/src/main/java/hudson/tasks/ArtifactArchiver.java
            core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly
            core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties
            core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties
            core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy
            core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties
            core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties
            test/src/test/java/hudson/tasks/ArtifactArchiverTest.java
            http://jenkins-ci.org/commit/jenkins/c726667162084e0199cea858735d35477d04feb9
            Log:
            [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: nlabrot Path: core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties test/src/test/java/hudson/tasks/ArtifactArchiverTest.java http://jenkins-ci.org/commit/jenkins/c726667162084e0199cea858735d35477d04feb9 Log: [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.)
            scm_issue_link SCM/JIRA link daemon made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            core/src/main/java/hudson/tasks/ArtifactArchiver.java
            core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly
            core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties
            core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties
            core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy
            core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties
            core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties
            test/src/test/java/hudson/tasks/ArtifactArchiverTest.java
            http://jenkins-ci.org/commit/jenkins/6d5c25ae7e88fa2e870209b4fe2e55b7b75fc0aa
            Log:
            Merge pull request #1287 from nithril/master

            [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.)

            Compare: https://github.com/jenkinsci/jenkins/compare/65bc0fb146bf...6d5c25ae7e88

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties test/src/test/java/hudson/tasks/ArtifactArchiverTest.java http://jenkins-ci.org/commit/jenkins/6d5c25ae7e88fa2e870209b4fe2e55b7b75fc0aa Log: Merge pull request #1287 from nithril/master [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.) Compare: https://github.com/jenkinsci/jenkins/compare/65bc0fb146bf...6d5c25ae7e88
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #3490
            [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.) (Revision c726667162084e0199cea858735d35477d04feb9)

            Result = SUCCESS
            nithril : c726667162084e0199cea858735d35477d04feb9
            Files :

            • core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties
            • core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy
            • core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties
            • core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties
            • test/src/test/java/hudson/tasks/ArtifactArchiverTest.java
            • core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties
            • core/src/main/java/hudson/tasks/ArtifactArchiver.java
            • core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #3490 [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.) (Revision c726667162084e0199cea858735d35477d04feb9) Result = SUCCESS nithril : c726667162084e0199cea858735d35477d04feb9 Files : core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties test/src/test/java/hudson/tasks/ArtifactArchiverTest.java core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly
            danielbeck Daniel Beck made changes -
            Link This issue is related to JENKINS-14355 [ JENKINS-14355 ]
            danielbeck Daniel Beck made changes -
            Summary Allow disabling use of default exclude patterns (.git, .svn, etc.) Allow disabling use of default exclude patterns (.git, .svn, etc.) for archiving
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 151608 ] JNJira + In-Review [ 194004 ]
            Hide
            jess Gordon Jess added a comment -

            Sorry, still experiencing the issue, 

            pipeline code:

            archiveArtifacts artifacts: '*/', defaultExcludes: false, onlyIfSuccessful: true

            Show
            jess Gordon Jess added a comment - Sorry, still experiencing the issue,  pipeline code: archiveArtifacts artifacts: '*/', defaultExcludes: false, onlyIfSuccessful: true
            jess Gordon Jess made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            jess Gordon Jess made changes -
            Comment [ Status set to resolved but I'm still missing .svn folders despite setting 'defaultExcludes: false'... ]
            Hide
            danielbeck Daniel Beck added a comment -

            Gordon Jess Works for me on Jenkins 2.89.3 and up to date plugins (sample Jenkinsfile: https://github.com/daniel-beck/pipeline-test/blob/43a0602597dd80995afa1714928a4b0e643ff18e/Jenkinsfile ). Please make sure you're not trying to archive in the wrong directory. Please file a new bug with complete and detailed steps to reproduce.

            Show
            danielbeck Daniel Beck added a comment - Gordon Jess Works for me on Jenkins 2.89.3 and up to date plugins (sample Jenkinsfile: https://github.com/daniel-beck/pipeline-test/blob/43a0602597dd80995afa1714928a4b0e643ff18e/Jenkinsfile ). Please make sure you're not trying to archive in the wrong directory. Please file a new bug with complete and detailed steps to reproduce.
            danielbeck Daniel Beck made changes -
            Status Reopened [ 4 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]

              People

              • Assignee:
                nithril Nicolas Labrot
                Reporter:
                eduardodias Eduardo Dias
              • Votes:
                1 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: