Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41191

read-only artifact files and build slaves

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Cannot Reproduce
    • Component/s: copyartifact-plugin
    • Labels:
    • Environment:
      Jenkins 1.658 with Copy Artifact plugin 1.38.1
      Master on 64-bit RHEL6, build slaves on 32-bit RHEL6, MacOS and Solaris.
    • Similar Issues:

      Description

      If an artifact file was created read-only (i.e. with permissions rrr-) when it was archived, the copyartifact plugin installs it as read-only too, which is what I would expect. Unfortunately when the build job runs a second time, the next attempt to copy the same artifact file then fails because it's trying to overwrite a read-only file.

      This doesn't seem to cause a problem with builds running on the master node, but it fails on my Linux, macOS and Solaris build slaves (I'm using a multi-config job to build the same code on all 4 OSs). The console output from one such slave job is shown below. If I wipe out the workspaces on the slaves, the next build that runs will succeed.

      {{FATAL: Failed to copy /var/lib/jenkins/jobs/extensions-3.14-ezca/configurations/axis-OS/linux32/builds/5/archive/include/ezca.h to /local/jenkins/workspace/extensions-3.14-ezcaScan/OS/linux32/include/ezca.h
      hudson.util.IOException2: Failed to copy /var/lib/jenkins/jobs/extensions-3.14-ezca/configurations/axis-OS/linux32/builds/5/archive/include/ezca.h to /local/jenkins/workspace/extensions-3.14-ezcaScan/OS/linux32/include/ezca.h
      at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:118)
      at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyAll(FingerprintingCopyMethod.java:67)
      at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:526)
      at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:460)
      at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
      at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
      at hudson.model.Build$BuildExecution.build(Build.java:205)
      at hudson.model.Build$BuildExecution.doRun(Build.java:162)
      at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
      at hudson.model.Run.execute(Run.java:1738)
      at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
      at hudson.model.ResourceController.execute(ResourceController.java:98)
      at hudson.model.Executor.run(Executor.java:410)
      Caused by: java.io.IOException: remote file operation failed: /local/jenkins/workspace/extensions-3.14-ezcaScan/OS/linux32/include/ezca.h at hudson.remoting.Channel@16e77985:linux32-uranus: java.io.FileNotFoundException: /local/jenkins/workspace/extensions-3.14-ezcaScan/OS/linux32/include/ezca.h (Permission denied)
      at hudson.FilePath.act(FilePath.java:986)
      at hudson.FilePath.act(FilePath.java:968)
      at hudson.FilePath.write(FilePath.java:1882)
      at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:82)
      ... 13 more
      Caused by: java.io.FileNotFoundException: /local/jenkins/workspace/extensions-3.14-ezcaScan/OS/linux32/include/ezca.h (Permission denied)
      at java.io.FileOutputStream.open0(Native Method)
      at java.io.FileOutputStream.open(FileOutputStream.java:270)
      at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
      at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
      at hudson.FilePath$36.invoke(FilePath.java:1887)
      at hudson.FilePath$36.invoke(FilePath.java:1882)
      at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2719)
      at hudson.remoting.UserRequest.perform(UserRequest.java:120)
      at hudson.remoting.UserRequest.perform(UserRequest.java:48)
      at hudson.remoting.Request$2.run(Request.java:332)
      at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      at ......remote call to linux32-uranus(Native Method)
      at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
      at hudson.remoting.UserResponse.retrieve(UserRequest.java:220)
      at hudson.remoting.Channel.call(Channel.java:781)
      at hudson.FilePath.act(FilePath.java:979)
      ... 16 more
      }}

        Attachments

          Activity

          Hide
          ikedam ikedam added a comment -

          Failing writing non-writeble files doesn't sound a bug, but a correct behavior
          (Rather, it sounds strange that it succeeds on the master).
          You should consider to use Workspace Cleanup Plugin.

          Show
          ikedam ikedam added a comment - Failing writing non-writeble files doesn't sound a bug, but a correct behavior (Rather, it sounds strange that it succeeds on the master). You should consider to use Workspace Cleanup Plugin .
          Hide
          ikedam ikedam added a comment -

          Could not reproduce.
          Read-only files fail to be overwritten even on the master.
          It's a correct behavior that copyartifact fail to overwrite read-only files.

          Steps to reproduce:

          • Tested with Ubuntu 12.04, Jenkins 2.32.1, Copyartifact 1.38.1
          • Create a slave "slave1"
          • Create a freestyle project "copiee"
            • Restrict where this project can be run: slave1
            • Execute shell
              rm -rf *
              touch readwrite.txt
              touch readonly.txt
              chmod 400 readonly.txt
              
            • Archive the artifacts
              • Files to archive: **/*
          • Run copiee
          • See that artifacts are archived as expected:
            $ ls -l /var/lib/jenkins/jobs/copiee/builds/2/archive/
            total 0
            -r-------- 1 jenkins jenkins 0 Jan 22 12:03 readonly.txt
            -rw-rw-r-- 1 jenkins jenkins 0 Jan 22 12:03 readwrite.txt
            
          • Create a freestyle project "copier"
            • Restrict where this project can be run: master
            • Copy artifacts from another project
              • Project name: copiee
              • Artifacts to copy: **/*
          • Run copier (1st time). The build succeeds.
          • See that artifacts are copied as expected:
            $ ls -l /var/lib/jenkins/workspace/copier/
            total 0
            -r-------- 1 jenkins jenkins 0 Jan 22 12:03 readonly.txt
            -rw-rw-r-- 1 jenkins jenkins 0 Jan 22 12:03 readwrite.txt
            
          • Run copiee (2nd time). The build fails.
            Started by user admin
            Building on master in workspace /var/lib/jenkins/workspace/copier
            FATAL: Failed to copy /var/lib/jenkins/jobs/copiee/builds/2/archive/readonly.txt to /var/lib/jenkins/workspace/copier/readonly.txt
            hudson.util.IOException2: Failed to copy /var/lib/jenkins/jobs/copiee/builds/2/archive/readonly.txt to /var/lib/jenkins/workspace/copier/readonly.txt
            	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:118)
            	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyAll(FingerprintingCopyMethod.java:67)
            	at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:526)
            	at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:460)
            	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
            	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
            	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
            	at hudson.model.Build$BuildExecution.build(Build.java:205)
            	at hudson.model.Build$BuildExecution.doRun(Build.java:162)
            	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
            	at hudson.model.Run.execute(Run.java:1728)
            	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
            	at hudson.model.ResourceController.execute(ResourceController.java:98)
            	at hudson.model.Executor.run(Executor.java:404)
            Caused by: java.io.FileNotFoundException: /var/lib/jenkins/workspace/copier/readonly.txt (Permission denied)
            	at java.io.FileOutputStream.open(Native Method)
            	at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
            	at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
            	at hudson.FilePath.write(FilePath.java:1879)
            	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:82)
            	... 13 more
            Finished: FAILURE
            
          Show
          ikedam ikedam added a comment - Could not reproduce. Read-only files fail to be overwritten even on the master. It's a correct behavior that copyartifact fail to overwrite read-only files. Steps to reproduce: Tested with Ubuntu 12.04, Jenkins 2.32.1, Copyartifact 1.38.1 Create a slave "slave1" Create a freestyle project "copiee" Restrict where this project can be run: slave1 To avoid JENKINS-13128 Execute shell rm -rf * touch readwrite.txt touch readonly.txt chmod 400 readonly.txt Archive the artifacts Files to archive: **/* Run copiee See that artifacts are archived as expected: $ ls -l /var/lib/jenkins/jobs/copiee/builds/2/archive/ total 0 -r-------- 1 jenkins jenkins 0 Jan 22 12:03 readonly.txt -rw-rw-r-- 1 jenkins jenkins 0 Jan 22 12:03 readwrite.txt Create a freestyle project "copier" Restrict where this project can be run: master Copy artifacts from another project Project name: copiee Artifacts to copy: **/* Run copier (1st time). The build succeeds. See that artifacts are copied as expected: $ ls -l /var/lib/jenkins/workspace/copier/ total 0 -r-------- 1 jenkins jenkins 0 Jan 22 12:03 readonly.txt -rw-rw-r-- 1 jenkins jenkins 0 Jan 22 12:03 readwrite.txt Run copiee (2nd time). The build fails. Started by user admin Building on master in workspace /var/lib/jenkins/workspace/copier FATAL: Failed to copy /var/lib/jenkins/jobs/copiee/builds/2/archive/readonly.txt to /var/lib/jenkins/workspace/copier/readonly.txt hudson.util.IOException2: Failed to copy /var/lib/jenkins/jobs/copiee/builds/2/archive/readonly.txt to /var/lib/jenkins/workspace/copier/readonly.txt at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:118) at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyAll(FingerprintingCopyMethod.java:67) at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:526) at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:460) at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779) at hudson.model.Build$BuildExecution.build(Build.java:205) at hudson.model.Build$BuildExecution.doRun(Build.java:162) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534) at hudson.model.Run.execute(Run.java:1728) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) Caused by: java.io.FileNotFoundException: /var/lib/jenkins/workspace/copier/readonly.txt (Permission denied) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.<init>(FileOutputStream.java:221) at java.io.FileOutputStream.<init>(FileOutputStream.java:171) at hudson.FilePath.write(FilePath.java:1879) at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:82) ... 13 more Finished: FAILURE
          Hide
          anjohnson Andrew Johnson added a comment -

          Presumably the behavior on the master has been fixed in a newer version of Jenkins than we're currently running.

          It still seems weird to me though that an upstream job can break downstream jobs that use this plugin merely by changing the permission of the files that it provides as artifacts. The permissions of the artifact files are not visible anywhere in the Jenkins GUI that I can see, so there's no easy way for a downstream job's author to know in advance whether they have to allow for this or not.

          This is obviously trivial to fix in a build script or with some other plugin so it's not really a problem once you realize what's causing the failure, just slightly annoying.

          Thanks anyway - Andrew

          Show
          anjohnson Andrew Johnson added a comment - Presumably the behavior on the master has been fixed in a newer version of Jenkins than we're currently running. It still seems weird to me though that an upstream job can break downstream jobs that use this plugin merely by changing the permission of the files that it provides as artifacts. The permissions of the artifact files are not visible anywhere in the Jenkins GUI that I can see, so there's no easy way for a downstream job's author to know in advance whether they have to allow for this or not. This is obviously trivial to fix in a build script or with some other plugin so it's not really a problem once you realize what's causing the failure, just slightly annoying. Thanks anyway - Andrew
          Hide
          anjohnson Andrew Johnson added a comment -

          Not a bug according to the maintainer, easily worked around.

          Show
          anjohnson Andrew Johnson added a comment - Not a bug according to the maintainer, easily worked around.

            People

            • Assignee:
              anjohnson Andrew Johnson
              Reporter:
              anjohnson Andrew Johnson
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: