Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55354

Permissions issue in remoting image jenkins/slave:latest

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Labels:
      None
    • Environment:
    • Similar Issues:

      Description

      Using jenkins/slave:latest remoting image in Docker Slaves plugin, our builds stopped working a few days ago with a permissions error when initializing the JAR Cache location. This is the relevant console output for the build:

      [Pipeline] dockerNode
      Start Docker container to host the build
      Unable to find image 'jenkins/slave:latest' locally
      latest: Pulling from jenkins/slave
      54f7e8ac135a: Already exists
      d6341e30912f: Already exists
      087a57faf949: Already exists
      5d71636fb824: Already exists
      9da6b28682cf: Already exists
      203f1094a1e2: Already exists
      ee38d9f85cf6: Already exists
      7f692fae02b6: Already exists
      eaa976dc543c: Already exists
      173a06ff64cc: Pulling fs layer
      28fc185aee23: Pulling fs layer
      e02013eddffd: Pulling fs layer
      cc49fe331e2e: Pulling fs layer
      cc49fe331e2e: Waiting
      173a06ff64cc: Verifying Checksum
      173a06ff64cc: Download complete
      28fc185aee23: Verifying Checksum
      28fc185aee23: Download complete
      e02013eddffd: Verifying Checksum
      e02013eddffd: Download complete
      173a06ff64cc: Pull complete
      28fc185aee23: Pull complete
      e02013eddffd: Pull complete
      cc49fe331e2e: Verifying Checksum
      cc49fe331e2e: Download complete
      cc49fe331e2e: Pull complete
      Digest: sha256:5683f906bab2e28abe332619bcec340a4da8553f10c34bbcbe6b555d32f76196
      Status: Downloaded newer image for jenkins/slave:latest
      [12/28/18 01:02:51] Launching agent
      $ docker start --interactive --attach 56be9bbd2bba60384bb02e9ba11a2a8cc4bbb954620b4dcc9a03ca9f3399b315
      Exception in thread "main" java.io.IOException: Failed to initialize the default JAR Cache location
      	at hudson.remoting.JarCache.getDefault(JarCache.java:41)
      	at hudson.remoting.ChannelBuilder.withJarCacheOrDefault(ChannelBuilder.java:220)
      	at hudson.remoting.Launcher.main(Launcher.java:736)
      	at hudson.remoting.Launcher.runWithStdinStdout(Launcher.java:691)
      	at hudson.remoting.Launcher.run(Launcher.java:373)
      	at hudson.remoting.Launcher.main(Launcher.java:283)
      Caused by: java.lang.IllegalArgumentException: Root directory not writable: ?/.jenkins/cache/jars
      	at hudson.remoting.FileSystemJarCache.<init>(FileSystemJarCache.java:62)
      	at hudson.remoting.JarCache.getDefault(JarCache.java:39)
      	... 5 more
      Caused by: java.nio.file.AccessDeniedException: /home/jenkins/?
      	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
      	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
      	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
      	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
      	at java.nio.file.Files.createDirectory(Files.java:674)
      	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781)
      	at java.nio.file.Files.createDirectories(Files.java:767)
      	at hudson.remoting.Util.mkdirs(Util.java:231)
      	at hudson.remoting.FileSystemJarCache.<init>(FileSystemJarCache.java:60)
      	... 6 more 

       

      I filed an issue on GitHub: https://github.com/jenkinsci/docker-slave/issues/45

      As I mentioned there, it seems like this commit could be related, as it's changing the uid of the user in the Docker image: https://github.com/jenkinsci/docker-slave/commit/723d12eea77b5d5bc3082612c12e30e422e237f2

      Neither the new uid 1000 nor the previous one 10000 matches the uid of the jenkins user where my build is running (uid:gid is 996:993), so I'm not sure if that's really the reason, but for sure the build was still working until Dec 19, and then I noticed it stopped working when I came back from vacation on Dec 26. This commit was done on Dec 22 and is related to user/permissions, so that's what makes me suspicious

      As a workaround I changed the remoting image to specific version jenkins/slave:3.27-1, which was published around 2 months ago according to https://hub.docker.com/r/jenkins/slave/tags. This fixed the issue.

        Attachments

          Activity

          Hide
          antoinetran Antoine Tran added a comment -

          We have the same issue! Thank you for the workaround, it works!

           

          Show
          antoinetran Antoine Tran added a comment - We have the same issue! Thank you for the workaround, it works!  
          Hide
          tdoubliez Thibault Doubliez added a comment -

          Digging into this issue more recently, I realized that Docker Slaves plugin actually hardcodes the 10000 uid when running the jenkins/slave remoting container, and after the change in jenkins/slave to use 1000 instead of 10000, there's now a mismatch which causes the permission issue.

          I opened a PR in Docker Slaves plugin to have that fixed, along with another issue I encountered recently after upgrading to Docker 19.03.0: https://github.com/jenkinsci/docker-slaves-plugin/pull/28

          Show
          tdoubliez Thibault Doubliez added a comment - Digging into this issue more recently, I realized that Docker Slaves plugin actually hardcodes the 10000 uid when running the jenkins/slave remoting container, and after the change in jenkins/slave to use 1000 instead of 10000, there's now a mismatch which causes the permission issue. I opened a PR in Docker Slaves plugin to have that fixed, along with another issue I encountered recently after upgrading to Docker 19.03.0:  https://github.com/jenkinsci/docker-slaves-plugin/pull/28

            People

            • Assignee:
              Unassigned
              Reporter:
              tdoubliez Thibault Doubliez
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: