Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40745

Kubernetes plugin mkdirs error when configuring a persistent volume claim for /home/jenkins on slave

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Won't Fix
    • Component/s: kubernetes-plugin
    • Labels:
    • Environment:
      kubernetes 1.4.4
      jenkins 2.32.1
      kubernetes-plugin 0.10
      slave image jenkinsci/jnlp-slave
    • Similar Issues:

      Description

      I'm attempting to create a persistent volume claim at /home/jenkins for my build container so that I can get the working directory off of the kubernetes node. I'm running into kubernetes eviction errors due to the size of the container / source code that is being checked out on the node.

      However, I'm running into a permissions error it looks like when hitting the checkout step. Any thoughts on what to try here?

      java.io.IOException: Failed to mkdirs: /home/jenkins/workspace/vendasta_AA_develop-IEOQWSXH4FZN4YOLA4GNTUPEHK5V6WLUSFSBDOOPY2CKPWC5MHVQ
      	at hudson.FilePath.mkdirs(FilePath.java:1169)
      	at hudson.plugins.git.GitSCM.createClient(GitSCM.java:736)
      	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1088)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:83)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:73)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
      	at hudson.security.ACL.impersonate(ACL.java:221)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Finished: FAILURE
      

      kubernetes GCE persistent volume claim:

      apiVersion: "v1"
      kind: "List"
      items:
        - apiVersion: "v1"
          kind: "PersistentVolume"
          metadata:
            name: "jenkins-slaves"
          spec:
            accessModes:
              - ReadWriteMany
            capacity:
              storage: 200Gi
            gcePersistentDisk:
              pdName: jenkins-slaves
              fsType: ext4
      
        - apiVersion: "v1"
          kind: "PersistentVolumeClaim"
          metadata:
            name: "jenkins-slaves"
            namespace: "jenkins"
          spec:
            accessModes:
              - ReadWriteMany
            resources:
              requests:
                storage: 1Gi
      
      

      jenkinsfile

      #!groovy
      
      podTemplate(label: 'build', inheritFrom: 'default', containers: [
          containerTemplate(
              name: 'build-dependencies',
              image: 'myorg/build-dependencies',
              ttyEnabled: true,
              command: 'cat',
              alwaysPullImage: true,
              workingDir: '/home/jenkins',
              resourceRequestCpu: '4',
              resourceLimitCpu: '4',
              resourceRequestMemory: '8Gi',
              resourceLimitMemory: '8Gi'
          )],
          volumes: [
              persistentVolumeClaim(mountPath: '/home/jenkins', claimName: 'jenkins-slaves', readOnly: false)
          ]) {
          node('build') {
      
              stage("Checkout") {
                  checkout scm
              }
          }
      }
      

        Attachments

          Issue Links

            Activity

            Hide
            jredl Jesse Redl added a comment -

            Well...I was able to workaround this error but i'm not entirely sure why this fixed the issue. Rather than using inheritFrom I specified the containerTemplate for the jnlp slave and it worked just fine?

            #!groovy
            
            podTemplate(label: 'build', containers: [
                containerTemplate(
                    name: 'jnlp',
                    image: 'jenkinsci/jnlp-slave/',
                    args: '${computer.jnlpmac} ${computer.name}',
                    resourceRequestCpu: '.5',
                    resourceLimitCpu: '1',
                    resourceRequestMemory: '500Mi',
                    resourceLimitMemory: '1Gi',
                ),
                containerTemplate(
                    name: 'build-dependencies',
                    image: 'myorg/build-dependencies',
                    ttyEnabled: true,
                    command: 'cat',
                    alwaysPullImage: true,
                    resourceRequestCpu: '4',
                    resourceLimitCpu: '4',
                    resourceRequestMemory: '8Gi',
                    resourceLimitMemory: '8Gi'
                )],
                volumes: [
                    persistentVolumeClaim(mountPath: '/home/jenkins', claimName: 'jenkins-slaves', readOnly: false)
                ]) {
                node('build') {
            
                    stage("Checkout") {
                        checkout scm
                    }
                }
            }
            
            
            Show
            jredl Jesse Redl added a comment - Well...I was able to workaround this error but i'm not entirely sure why this fixed the issue. Rather than using inheritFrom I specified the containerTemplate for the jnlp slave and it worked just fine? #!groovy podTemplate(label: 'build' , containers: [ containerTemplate( name: 'jnlp' , image: 'jenkinsci/jnlp-slave/' , args: '${computer.jnlpmac} ${computer.name}' , resourceRequestCpu: '.5' , resourceLimitCpu: '1' , resourceRequestMemory: '500Mi' , resourceLimitMemory: '1Gi' , ), containerTemplate( name: 'build-dependencies' , image: 'myorg/build-dependencies' , ttyEnabled: true , command: 'cat' , alwaysPullImage: true , resourceRequestCpu: '4' , resourceLimitCpu: '4' , resourceRequestMemory: '8Gi' , resourceLimitMemory: '8Gi' )], volumes: [ persistentVolumeClaim(mountPath: '/home/jenkins' , claimName: 'jenkins-slaves' , readOnly: false ) ]) { node( 'build' ) { stage( "Checkout" ) { checkout scm } } }
            Hide
            skulanov Sergey Kulanov added a comment -

            I think that your code was running on jnlp slave before you explicitly added `custom jnlp` container. You can try to check this theory

            Show
            skulanov Sergey Kulanov added a comment - I think that your code was running on jnlp slave before you explicitly added `custom jnlp` container. You can try to check this theory
            Hide
            jredl Jesse Redl added a comment -

            I think you're right. Going to close this issue as we've been running fine since making these tweaks.

            Show
            jredl Jesse Redl added a comment - I think you're right. Going to close this issue as we've been running fine since making these tweaks.
            Hide
            jredl Jesse Redl added a comment -

            Closing issue I logged as this seems to be user error.

            Show
            jredl Jesse Redl added a comment - Closing issue I logged as this seems to be user error.

              People

              • Assignee:
                csanchez Carlos Sanchez
                Reporter:
                jredl Jesse Redl
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: