Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18578

Default jar cache location is hardcoded to ~/.jenkins/cache/jars

    Details

    • Similar Issues:

      Description

      The default jar cache location is now hardcoded to ~/.jenkins/cache/jars which is a really bad idea for a big production environment with networked home directories and multiple accounts running slaves.

      This behaviour was introduced with this commit:
      https://github.com/jenkinsci/remoting/commit/b5145a06876f4390ef3229d70fa5a7edf0739dae

      This cache location needs to be configurable.

      A bonus would be if permissions within it were handled in a way that supports multiple accounts using it. That might be the case already though.

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            I think the default needs to be changed to use the slave FS root. We should default to a safe and expected configuration; in the rare case that someone runs multiple slaves on one machine, if they find the duplicated caches to consume too much space, they can decide to share them in a location of their choice.

            Show
            jglick Jesse Glick added a comment - I think the default needs to be changed to use the slave FS root. We should default to a safe and expected configuration; in the rare case that someone runs multiple slaves on one machine, if they find the duplicated caches to consume too much space, they can decide to share them in a location of their choice.
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            The -jar-cache option in slave.jar lets you set the location of the cache since remoting 2.24.

            Show
            kohsuke Kohsuke Kawaguchi added a comment - The -jar-cache option in slave.jar lets you set the location of the cache since remoting 2.24.
            Hide
            cowwoc cowwoc added a comment -

            I disagree with the way this was resolved, so I filed JENKINS-18956.

            Show
            cowwoc cowwoc added a comment - I disagree with the way this was resolved, so I filed JENKINS-18956 .
            Hide
            jglick Jesse Glick added a comment -

            Did you miss my comment? IMHO the default should be to use the slave FS root.

            Show
            jglick Jesse Glick added a comment - Did you miss my comment? IMHO the default should be to use the slave FS root.
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            I agree with Jesse. Additional concerns:

            • Slaves may have read-only access to their home directories (in order to protect user's stuff from automation environment)
            • Usage of remote home directories from a Windows service may cause even more surprises

            Due to the first bullet, migration to 1.509.4 suddenly corrupted our slaves. It is not good.

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited I agree with Jesse. Additional concerns: Slaves may have read-only access to their home directories (in order to protect user's stuff from automation environment) Usage of remote home directories from a Windows service may cause even more surprises Due to the first bullet, migration to 1.509.4 suddenly corrupted our slaves. It is not good.
            Hide
            epkabeg Andréas Berg added a comment -

            Using the slave FS root as default sounds like a good solution to me too.

            Show
            epkabeg Andréas Berg added a comment - Using the slave FS root as default sounds like a good solution to me too.
            Hide
            lymanva limin wang added a comment -

            Quota exceeded again caused by this folder. Will different slaves need to share this folder? otherwize use slave FS root is a solution.

            Show
            lymanva limin wang added a comment - Quota exceeded again caused by this folder. Will different slaves need to share this folder? otherwize use slave FS root is a solution.
            Hide
            epkabeg Andréas Berg added a comment -

            No, sharing is not a requirement from my side, I just see it as a good option to have, if possible.

            The default however I agree should be slave FS root as many have asked for already.

            Show
            epkabeg Andréas Berg added a comment - No, sharing is not a requirement from my side, I just see it as a good option to have, if possible. The default however I agree should be slave FS root as many have asked for already.
            Hide
            froethen Felix Roethenbacher added a comment -

            Path should consider JENKINS_HOME system variable (Jenkins running as WAR with user tomcat7).

            Workaround is to create a sym link from shared tomcat7 directory to Jenkins home directory:

            ln -s /var/lib/jenkins /usr/share/tomcat7/.jenkins
            
            Show
            froethen Felix Roethenbacher added a comment - Path should consider JENKINS_HOME system variable (Jenkins running as WAR with user tomcat7). Workaround is to create a sym link from shared tomcat7 directory to Jenkins home directory: ln -s /var/lib/jenkins /usr/share/tomcat7/.jenkins
            Hide
            epkabeg Andréas Berg added a comment -

            No, using JENKINS_HOME would only work for the master node.

            Using slave FS root as default is likely a good solution for most people.

            Show
            epkabeg Andréas Berg added a comment - No, using JENKINS_HOME would only work for the master node. Using slave FS root as default is likely a good solution for most people.
            Hide
            froethen Felix Roethenbacher added a comment -

            Well, according to the docs you can set the Jenkins working directory with JENKINS_HOME system variable. That's how it used to work, at least in the master only environment.

            Maybe there is a a solution that works for both environments, master and slave.

            Show
            froethen Felix Roethenbacher added a comment - Well, according to the docs you can set the Jenkins working directory with JENKINS_HOME system variable. That's how it used to work, at least in the master only environment. Maybe there is a a solution that works for both environments, master and slave.
            Hide
            epkabeg Andréas Berg added a comment -

            This issue is not about the Jenkins working directory. It is about the default jar cache location.

            Yes, using the slave FS root as default should work for both environments. master is after all just another slave.

            Show
            epkabeg Andréas Berg added a comment - This issue is not about the Jenkins working directory. It is about the default jar cache location. Yes, using the slave FS root as default should work for both environments. master is after all just another slave.
            Hide
            froethen Felix Roethenbacher added a comment -

            Actually, it is related to the Jenkins working/home directory (see JENKINS-18459) as this is where Jenkins usually is given read/write access.

            In an environment where Jenkins WAR is running inside an existing Tomcat application server with user 'tomcat7', Jenkins fails as the user does not have write access to the user's home directory (usually that's '/usr/share/tomcat7') for security reasons.

            If the slave FS root solves this problem, that's perfect. Just make sure that the actual user running Jenkins has write access to this directory.

            Show
            froethen Felix Roethenbacher added a comment - Actually, it is related to the Jenkins working/home directory (see JENKINS-18459 ) as this is where Jenkins usually is given read/write access. In an environment where Jenkins WAR is running inside an existing Tomcat application server with user 'tomcat7', Jenkins fails as the user does not have write access to the user's home directory (usually that's '/usr/share/tomcat7') for security reasons. If the slave FS root solves this problem, that's perfect. Just make sure that the actual user running Jenkins has write access to this directory.
            Hide
            epkabeg Andréas Berg added a comment -

            Good point, your case is actually even worse than ours then, as it fails due to this.

            We only get decreased performance and the risk of garbage filling up the home directory quota, at which point it will fail for us too.

            Show
            epkabeg Andréas Berg added a comment - Good point, your case is actually even worse than ours then, as it fails due to this. We only get decreased performance and the risk of garbage filling up the home directory quota, at which point it will fail for us too.
            Hide
            calebmayeux Caleb Mayeux added a comment -

            Another workaround is to use what Mr. Kohsuke said: In the slave configuration, under the Advanced method of Launch Option,
            Put the following in the Suffix Start Slave Command " -jar-cache <path to jar cache directory>" (no quotes)
            Make sure there's a leading space so the parameters aren't tacked directly onto the slave.jar itself.
            Still, I'd much prefer if the default jar cache was in the slave FS root. I have a number of slaves using an automounted home directory, and am hitting the quota exceeded error when upgrading from 1.544 to 1.551.

            Show
            calebmayeux Caleb Mayeux added a comment - Another workaround is to use what Mr. Kohsuke said: In the slave configuration, under the Advanced method of Launch Option, Put the following in the Suffix Start Slave Command " -jar-cache <path to jar cache directory>" (no quotes) Make sure there's a leading space so the parameters aren't tacked directly onto the slave.jar itself. Still, I'd much prefer if the default jar cache was in the slave FS root. I have a number of slaves using an automounted home directory, and am hitting the quota exceeded error when upgrading from 1.544 to 1.551.
            Hide
            jglick Jesse Glick added a comment -

            JENKINS-26199 notes that if ~/.jenkins/ is not writable, functional tests will fail, which is unexpected.

            Show
            jglick Jesse Glick added a comment - JENKINS-26199 notes that if ~/.jenkins/ is not writable, functional tests will fail, which is unexpected.
            Hide
            randroid Roberto Andrade added a comment -

            Another workaround I've found that worked better for me both on master and slave is changing the `user.home` System property by setting the `_JAVA_OPTIONS` environment variable as described [here](http://stackoverflow.com/questions/1501235/change-user-home-system-property).

            Show
            randroid Roberto Andrade added a comment - Another workaround I've found that worked better for me both on master and slave is changing the `user.home` System property by setting the `_JAVA_OPTIONS` environment variable as described [here] ( http://stackoverflow.com/questions/1501235/change-user-home-system-property ).
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            I'm going to change the default cache location in remoting 3

            Show
            oleg_nenashev Oleg Nenashev added a comment - I'm going to change the default cache location in remoting 3
            Hide
            epkabeg Andréas Berg added a comment -

            Will this also affect the Jenkins master or only the slaves?

            What will the new default be?

            Show
            epkabeg Andréas Berg added a comment - Will this also affect the Jenkins master or only the slaves? What will the new default be?
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            My plan is to make the behavior change a part of the Remoting Working directory story (JENKINS-39370), see https://github.com/jenkinsci/remoting/pull/129. By default work directory will be pointing to the Agent working directory, so the default path will be something like AGENT_DIR/remoting/.jarCache

            It should not affect the master

            Show
            oleg_nenashev Oleg Nenashev added a comment - My plan is to make the behavior change a part of the Remoting Working directory story ( JENKINS-39370 ), see https://github.com/jenkinsci/remoting/pull/129 . By default work directory will be pointing to the Agent working directory, so the default path will be something like AGENT_DIR/remoting/.jarCache It should not affect the master
            Hide
            oleg_nenashev Oleg Nenashev added a comment -
            Show
            oleg_nenashev Oleg Nenashev added a comment - The fix has been added to  https://github.com/jenkinsci/remoting/pull/129
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            README.md
            docs/logging.md
            docs/workDir.md
            src/main/java/hudson/remoting/Engine.java
            src/main/java/hudson/remoting/FileSystemJarCache.java
            src/main/java/hudson/remoting/Launcher.java
            src/main/java/hudson/remoting/TeeOutputStream.java
            src/main/java/hudson/remoting/jnlp/Main.java
            src/main/java/org/jenkinsci/remoting/engine/WorkDirManager.java
            src/test/java/org/jenkinsci/remoting/engine/WorkDirManagerTest.java
            http://jenkins-ci.org/commit/remoting/76c9b8ccf14f7def1141565b0dee2e4d1c5508d4
            Log:
            [JENKINS-39370,JENKINS-39369] - Support of work directories in Remoting (#129)

            • [JENKINS-39370,JENKINS-39369] - Add workDir parameter and automatically create logs there if specified
            • Save the progress
            • JENKINS-39370 - Generalize the workspace manager for Java Web Start
            • JENKINS-39370 - Restrict the range of supported symbols in the remoting work directory
            • JENKINS-39130 - Allow specifying flag for failing initialization if workdir is missing
            • JENKINS-39370 - @stephenc noticed that workDir may be null, Intellij IDEA adoption fun
            • JENKINS-39370 - Another message, which likely breaks the CI instance
            • JENKINS-39817 - Introduce the agentLog parameter in remoting.jnlp.Main

            @stephenc suggested doing it in the PR, so I decided to address it as a part of JENKINS-39370.
            But the code still has initialization in hudson.remoting.Launcher for other logging modes.

            • Enable JUL logging to a log-rotated file by default
            • JENKINS-18578 - If workspace manager is defined, use JAR Cache within its interbal directory
            • JENKINS-39370 - Fix the workDirManager's log initialization in Launcher
            • JENKINS-39369 - Make JUL logging system configurable via property file
            • JENKINS-39369 - Fixes in logging management after the manual testing
            • JENKINS-39369 - Respect configuration being passed from java.util.logging.config.file system property
            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: README.md docs/logging.md docs/workDir.md src/main/java/hudson/remoting/Engine.java src/main/java/hudson/remoting/FileSystemJarCache.java src/main/java/hudson/remoting/Launcher.java src/main/java/hudson/remoting/TeeOutputStream.java src/main/java/hudson/remoting/jnlp/Main.java src/main/java/org/jenkinsci/remoting/engine/WorkDirManager.java src/test/java/org/jenkinsci/remoting/engine/WorkDirManagerTest.java http://jenkins-ci.org/commit/remoting/76c9b8ccf14f7def1141565b0dee2e4d1c5508d4 Log: [JENKINS-39370,JENKINS-39369] - Support of work directories in Remoting (#129) [JENKINS-39370,JENKINS-39369] - Add workDir parameter and automatically create logs there if specified Save the progress JENKINS-39370 - Generalize the workspace manager for Java Web Start JENKINS-39370 - WiP - Save progress in the test suite JENKINS-39370 - Add tests for WorkDirManager JENKINS-39370 - Restrict the range of supported symbols in the remoting work directory JENKINS-39370 - Generalize the workspace initialization checks JENKINS-39130 - Allow specifying flag for failing initialization if workdir is missing JENKINS-39370 - Reference DirType in WorkDirManager Javadocs JENKINS-39370 - @stephenc noticed that workDir may be null, Intellij IDEA adoption fun JENKINS-39370 - Seems this message breaks our CI JENKINS-39370 - Another message, which likely breaks the CI instance JENKINS-39370 - Simplify the log handling logic JENKINS-39817 - Introduce the agentLog parameter in remoting.jnlp.Main @stephenc suggested doing it in the PR, so I decided to address it as a part of JENKINS-39370 . But the code still has initialization in hudson.remoting.Launcher for other logging modes. Enable JUL logging to a log-rotated file by default JENKINS-18578 - If workspace manager is defined, use JAR Cache within its interbal directory JENKINS-39370 - Fix the workDirManager's log initialization in Launcher JENKINS-39370 - Draft the documentation JENKINS-39369 - Make JUL logging system configurable via property file JENKINS-39369 - Fixes in logging management after the manual testing JENKINS-39369 - Add tests for the logging subsystem JENKINS-39369 - Respect configuration being passed from java.util.logging.config.file system property
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Merged towards Remoting 3.8. You will need to activate remoting Work Directory to get the changes applied

            Show
            oleg_nenashev Oleg Nenashev added a comment - Merged towards Remoting 3.8. You will need to activate remoting Work Directory to get the changes applied
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            src/main/java/hudson/remoting/Launcher.java
            http://jenkins-ci.org/commit/remoting/a547765c8b9cca411c2db999b53074aa2ed8d9d9
            Log:
            JENKINS-18578 - Do not use the old cache when starting agents from CLI

            Just a leftover from the previous fix, nulls are handled in hudson.remoting.Engine.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/hudson/remoting/Launcher.java http://jenkins-ci.org/commit/remoting/a547765c8b9cca411c2db999b53074aa2ed8d9d9 Log: JENKINS-18578 - Do not use the old cache when starting agents from CLI Just a leftover from the previous fix, nulls are handled in hudson.remoting.Engine.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            src/main/java/hudson/remoting/Launcher.java
            http://jenkins-ci.org/commit/remoting/2649c4c35350d0fcf8da3de685064a9f7f55c051
            Log:
            Merge pull request #166 from oleg-nenashev/bug/JENKINS-18578-2

            JENKINS-18578 - Do not use the old cache when starting agents from CLI

            Compare: https://github.com/jenkinsci/remoting/compare/5f7f79eb016d...2649c4c35350

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/hudson/remoting/Launcher.java http://jenkins-ci.org/commit/remoting/2649c4c35350d0fcf8da3de685064a9f7f55c051 Log: Merge pull request #166 from oleg-nenashev/bug/ JENKINS-18578 -2 JENKINS-18578 - Do not use the old cache when starting agents from CLI Compare: https://github.com/jenkinsci/remoting/compare/5f7f79eb016d...2649c4c35350
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            The work directory feature has been integrated towards 2.68, hence now it is possible to change the default location. Follow JENKINS-44108, which has stories for enabling work directories by default on different agent types

            Show
            oleg_nenashev Oleg Nenashev added a comment - The work directory feature has been integrated towards 2.68, hence now it is possible to change the default location. Follow JENKINS-44108 , which has stories for enabling work directories by default on different agent types
            Hide
            psunjohnny Peng Sun added a comment - - edited

            I just updated jenkins from 2.58 to 2.68, after that I got:

            ...

            ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins. java.lang.RuntimeException: Root directory not writable: /folk/jenkins/.jenkins/cache/jars

            ...

            It was fine on 2.58!

            Show
            psunjohnny Peng Sun added a comment - - edited I just updated jenkins from 2.58 to 2.68, after that I got: ... ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins. java.lang.RuntimeException: Root directory not writable: /folk/jenkins/.jenkins/cache/jars ... It was fine on 2.58!
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Peng Sun The Work Dir functionality is disabled by default, but there are some extra checks in the FileJarCache logic. Please file a separate bug and attach a full stacktrace.

            Show
            oleg_nenashev Oleg Nenashev added a comment - Peng Sun The Work Dir functionality is disabled by default, but there are some extra checks in the FileJarCache logic. Please file a separate bug and attach a full stacktrace.

              People

              • Assignee:
                oleg_nenashev Oleg Nenashev
                Reporter:
                epkabeg Andréas Berg
              • Votes:
                13 Vote for this issue
                Watchers:
                26 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: