Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-25759

FilePath.validateAntFileMask sucks up heap

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Although FilePath.validateAntFileMask is typically bounded to check at most 10000 files before giving up and announcing that it cannot find a match, it turns out that DirectoryScanner collects the names of non-matches in filesNotIncluded and dirsNotIncluded in the meantime. In a big workspace, this can cause the JVM (usually a slave agent) to allocate gigabytes of heap, eventually causing the slave to become overwhelmed and crash.

        Attachments

          Issue Links

            Activity

            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #3833
            [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. (Revision 534328b264f9338e48418d2bcc0d28daaf48b3a0)

            Result = SUCCESS
            jesse glick : 534328b264f9338e48418d2bcc0d28daaf48b3a0
            Files :

            • core/src/main/java/hudson/util/FormFieldValidator.java
            • core/src/main/java/hudson/FilePath.java
            • core/src/main/java/hudson/tasks/ArtifactArchiver.java
            • core/src/test/java/hudson/FilePathTest.java
            • changelog.html
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #3833 [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. (Revision 534328b264f9338e48418d2bcc0d28daaf48b3a0) Result = SUCCESS jesse glick : 534328b264f9338e48418d2bcc0d28daaf48b3a0 Files : core/src/main/java/hudson/util/FormFieldValidator.java core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/test/java/hudson/FilePathTest.java changelog.html
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/hudson/FilePath.java
            core/src/main/java/hudson/tasks/ArtifactArchiver.java
            core/src/main/java/hudson/util/FormFieldValidator.java
            core/src/test/java/hudson/FilePathTest.java
            http://jenkins-ci.org/commit/jenkins/534328b264f9338e48418d2bcc0d28daaf48b3a0
            Log:
            [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask.
            Not fully solved, since the scannedDirs field can still grow to be large, but at least clearing files/dirsNotIncluded.
            Also imposing a 5s timeout on the scan regardless of file count, and defining a user-customizable bound.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/java/hudson/util/FormFieldValidator.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/534328b264f9338e48418d2bcc0d28daaf48b3a0 Log: [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. Not fully solved, since the scannedDirs field can still grow to be large, but at least clearing files/dirsNotIncluded. Also imposing a 5s timeout on the scan regardless of file count, and defining a user-customizable bound.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/FilePath.java
            core/src/main/java/hudson/tasks/ArtifactArchiver.java
            core/src/main/java/hudson/util/FormFieldValidator.java
            core/src/test/java/hudson/FilePathTest.java
            http://jenkins-ci.org/commit/jenkins/23e0d6ad1964e0efdd1e0f1c90360321acc8a073
            Log:
            [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask.
            Not fully solved, since the scannedDirs field can still grow to be large, but at least clearing files/dirsNotIncluded.
            Also imposing a 5s timeout on the scan regardless of file count, and defining a user-customizable bound.
            (cherry picked from commit 534328b264f9338e48418d2bcc0d28daaf48b3a0)

            Conflicts:
            changelog.html

            Compare: https://github.com/jenkinsci/jenkins/compare/03600e925c3d...23e0d6ad1964

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/java/hudson/util/FormFieldValidator.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/23e0d6ad1964e0efdd1e0f1c90360321acc8a073 Log: [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. Not fully solved, since the scannedDirs field can still grow to be large, but at least clearing files/dirsNotIncluded. Also imposing a 5s timeout on the scan regardless of file count, and defining a user-customizable bound. (cherry picked from commit 534328b264f9338e48418d2bcc0d28daaf48b3a0) Conflicts: changelog.html Compare: https://github.com/jenkinsci/jenkins/compare/03600e925c3d...23e0d6ad1964
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #4292
            [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. (Revision 23e0d6ad1964e0efdd1e0f1c90360321acc8a073)

            Result = UNSTABLE
            ogondza : 23e0d6ad1964e0efdd1e0f1c90360321acc8a073
            Files :

            • core/src/main/java/hudson/util/FormFieldValidator.java
            • core/src/main/java/hudson/tasks/ArtifactArchiver.java
            • core/src/test/java/hudson/FilePathTest.java
            • core/src/main/java/hudson/FilePath.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #4292 [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. (Revision 23e0d6ad1964e0efdd1e0f1c90360321acc8a073) Result = UNSTABLE ogondza : 23e0d6ad1964e0efdd1e0f1c90360321acc8a073 Files : core/src/main/java/hudson/util/FormFieldValidator.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/test/java/hudson/FilePathTest.java core/src/main/java/hudson/FilePath.java

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                jglick Jesse Glick
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: