Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60938

Solve problem of too many .groovy files in job directories

    Details

    • Similar Issues:

      Description

      When using Jenkins shared libraries (JSLs), by default, all the .groovy files are stored in the job directory along with the rest of the logs and data. The files are small, usually smaller than logs. However, recently, we've received warnings that the volume with the job directories is low on free INODES, which means "too many files". So, file count actually is significant at scale.

      We want to keep our job logs as long as possible, but care much less about keeping the .groovy files. Deleting only the .groovy files would enable us to keep roughly 100 times the number of logs. Currently, buildDiscarder property gives us a per-job cleanup policy which is good. However, currently it provides no granularity over the files in the directory which is understandable,

      This feature request is to suggest looking at the many options available for solving such a problem, and hopefully finding one which is easy to implement.

      For example:

      • enhance build buildDiscarder property with special handling for groovy files
      • create similar-but-different property called "buildPostProcessor" (generic)
      • create similar-but-different property called "discardGroovyFiles" (specific)
      • provide a granular / generic cleanup step which allows access to the "current job log" dir and arbitrary groovy code for cleanup.
      • add a JSL option to auto-zip or auto-jar the groovy files in the job directory

      I believe I saw a plugin which allows JSL's to be loaded from JAR files already, however I can't seem to find it now.

        Attachments

          Activity

          Hide
          solvingj jerry wiltse added a comment - - edited

          Just did a recursive walk of our jobs directory.  There are 5,597,016 groovy files.   Our JSL has about 60 groovy files. 

          Show
          solvingj jerry wiltse added a comment - - edited Just did a recursive walk of our jobs directory.  There are 5,597,016 groovy files.   Our JSL has about 60 groovy files. 

            People

            • Assignee:
              Unassigned
              Reporter:
              solvingj jerry wiltse
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: