Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40429

Copy artifacts from multi-branch pipeline project with project name as a build parameter fails with "Unable to find project for artifact copy"

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Blocker
    • Resolution: Unresolved
    • Component/s: copyartifact-plugin
    • Labels:
      None
    • Environment:
      Master & slave: Debian Jessie 64 bit, Oracle JRE, installed from deb repo
      Jenkins: 2.19.3
      CopyArtifactPlugin: 1.38.1
    • Similar Issues:

      Description

      We have a multi-branch pipeline project that produces artifacts, we also have a freeform project that is configured with build parameters to copy artifacts from the pipeline job based on the parameters supplied.

      When I set the project name parameter to something like

      myproject/${BRANCH_NAME} 

      the job fails with

      Unable to find project for artifact copy: myproject/mybranch

      If it set the project name parameter to

      myproject/mybranch

      it works fine.

      We aren't using matrix permissions, logged in users can do anything and Allow anonymous read access is checked. I tried setting the project property CopyArtifactPermissionProperty in the JenkinsFile to

      properties([[$class: 'CopyArtifactPermissionProperty', projectNames: '*']])
      

      but this made no difference.

        Attachments

          Activity

          Hide
          haggai_e Haggai Eran added a comment -

          Not sure if this is the same issue, but I just spent a couple of days debugging a similar error message when trying to copy artifact from one multibranch pipeline into another. Eventually the problem was that one of the branches in the source job had the same name as the destination job. E.g. copying from job "a/master" to job "b/a" failed because if I understand correctly the plugin was looking for an item named "b/a/master". I solved the issue by using a full path instead (e.g. "/a/master").

          Show
          haggai_e Haggai Eran added a comment - Not sure if this is the same issue, but I just spent a couple of days debugging a similar error message when trying to copy artifact from one multibranch pipeline into another. Eventually the problem was that one of the branches in the source job had the same name as the destination job. E.g. copying from job "a/master" to job "b/a" failed because if I understand correctly the plugin was looking for an item named "b/a/master". I solved the issue by using a full path instead (e.g. "/a/master").
          Hide
          apikas Anders Pikas added a comment -

          Thanks, Haggai Eran! I solved this in 30 minutes thanks to you, not a couple of days. I had the same problem as you: the pipeline I was copying from started with "core/" and the pipeline I was running had a name ending in "/core".

          I always specify absolute job/project/pipeline paths as absolute, BUT Jenkins itself gives ${env.JOB_NAME} as a relative path for some reason (maybe backward compatibility?).

          env.JOB_NAME.replaceFirst('^(?!/)', '/')
          
          Show
          apikas Anders Pikas added a comment - Thanks, Haggai Eran ! I solved this in 30 minutes thanks to you, not a couple of days. I had the same problem as you: the pipeline I was copying from started with "core/" and the pipeline I was running had a name ending in "/core". I always specify absolute job/project/pipeline paths as absolute, BUT Jenkins itself gives ${env.JOB_NAME} as a relative path for some reason (maybe backward compatibility?). env.JOB_NAME.replaceFirst( '^(?!/)' , '/' )
          Hide
          maschall Mark Schall added a comment -

          I was running into the same issues, until I dove into the code of the plugin.

          If you have a multi branch pipeline job, say project/branch and you add the 

          copyArtifactPermission('downstream');

          You need to make it

          copyArtifactPermission('/downstream');

          because the downstream project is not in the "folder" of the project, it's one more level down.

          Then passing in `project/branch` as a parameter should work.

          The root cause, is that permissions are not checked at run time, if you hardcode the project name, but if you set as a variable, they are checked and names don't match what is expected.

          Show
          maschall Mark Schall added a comment - I was running into the same issues, until I dove into the code of the plugin. If you have a multi branch pipeline job, say project/branch and you add the  copyArtifactPermission( 'downstream' ); You need to make it copyArtifactPermission( '/downstream' ); because the downstream project is not in the "folder" of the project, it's one more level down. Then passing in `project/branch` as a parameter should work. The root cause, is that permissions are not checked at run time, if you hardcode the project name, but if you set as a variable, they are checked and names don't match what is expected.
          Hide
          mrw Marc Wäckerlin added a comment - - edited

          First of all, it seems, that I am running into the same problem. But for me, it looks strange that this error has been reported such a long time ago (and why is it not fixed in these years?), but I am sure that it used to work short time ago. I cannot exactly say, how long it used to work for me, but I had successful builds. I ran into it now that I am moving my jenkins to a new server and re-running all builds. I am wondering why the priority should be minor, it's a blocker, the plugin is no more usable.

          My situation:

          Jenkins ver. 2.225
          Tested Copy Artifact 1.43.1 and 1.43

          Global Jenkins environment variable:

          • name: MINGW_ARCH
          • value: x86_64-w64-mingw32 i686-w64-mingw32

          ProjectA:

          • matrix build:
            • name: mingw
            • variable: MINGW_ARCH
          • Artifacts: */usr and }}{{.zip

          ProjectB (built after ProjectA):

          • matrix build:
            • name: mingw
            • variable: MINGW_ARCH
          • Copy Artifacts:
            • project name: ProjectA/mingw=$mingw
          • Artifacts: */usr and }}{{.zip

          This results in:

          ERROR: Unable to find project for artifact copy: ProjectA/mingw=x86_64-w64-mingw32

          Then, if I replace the project name to ProjectA/mingw=x86_64-w64-mingw32, that means hard coded, no variable, then it works.

          As you see from the error log, the variable substitution is 100% correct! But still there is somehow a failure with regard to the variable.

          The projects configurations (ProjectA is mrw-cxx.win and ProjectB is libxml-cxx.win):

          ProjectA:

          <?xml version="1.1" encoding="UTF-8" standalone="no"?><matrix-project plugin="matrix-project@1.14">
          <actions/>
          <description/>
          <keepDependencies>false</keepDependencies>
          <properties>

          </properties>
          <scm class="hudson.plugins.git.GitSCM" plugin="git@4.2.2">
          <configVersion>2</configVersion>
          <userRemoteConfigs>
          <hudson.plugins.git.UserRemoteConfig>
          <url>https://mrw.dev/libraries/mrw-cxx</url>
          </hudson.plugins.git.UserRemoteConfig>
          </userRemoteConfigs>
          <branches>
          <hudson.plugins.git.BranchSpec>
          <name>*/master</name>
          </hudson.plugins.git.BranchSpec>
          </branches>
          <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
          <submoduleCfg class="list"/>
          <extensions/>
          </scm>
          <quietPeriod>5</quietPeriod>
          <scmCheckoutRetryCount>10</scmCheckoutRetryCount>
          <canRoam>true</canRoam>
          <disabled>false</disabled>
          <blockBuildWhenDownstreamBuilding>true</blockBuildWhenDownstreamBuilding>
          <blockBuildWhenUpstreamBuilding>true</blockBuildWhenUpstreamBuilding>
          <triggers>
          <hudson.triggers.TimerTrigger>
          <spec>@monthly</spec>
          </hudson.triggers.TimerTrigger>
          <hudson.triggers.SCMTrigger>
          <spec>H/5 * * * *</spec>
          <ignorePostCommitHooks>false</ignorePostCommitHooks>
          </hudson.triggers.SCMTrigger>
          </triggers>
          <concurrentBuild>false</concurrentBuild>
          <axes>
          <ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis plugin="dynamic-axis@1.0.3">
          <name>mingw</name>
          <values>
          <string>MINGW_ARCHS</string>
          </values>
          <varName>MINGW_ARCHS</varName>
          <axisValues>
          <string>default</string>
          </axisValues>
          </ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis>
          </axes>
          <builders>
          <hudson.tasks.Shell>
          <command>docker pull mwaeckerlin/mingw
          docker run --rm -v $(pwd):/workdir -u $(id -u) -e MINGW=$mingw -e BUILD_NUMBER=$BUILD_NUMBER mwaeckerlin/mingw /build.sh -z

          FILE=mrw-c++-*_${mingw%-w64-mingw32}.zip
          TARGET=~/repository/windows/${mingw%-w64-mingw32}/${JOB_NAME%.win/*}
          test -d ${TARGET} || mkdir -p ${TARGET}
          cp ${FILE} ${TARGET}/
          </command>
          </hudson.tasks.Shell>
          </builders>
          <publishers>
          <hudson.tasks.ArtifactArchiver>
          <artifacts>usr/**, *.zip</artifacts>
          <allowEmptyArchive>false</allowEmptyArchive>
          <onlyIfSuccessful>false</onlyIfSuccessful>
          <fingerprint>false</fingerprint>
          <defaultExcludes>true</defaultExcludes>
          <caseSensitive>true</caseSensitive>
          </hudson.tasks.ArtifactArchiver>
          </publishers>
          <buildWrappers>
          <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="ws-cleanup@0.38">
          <deleteDirs>false</deleteDirs>
          <cleanupParameter/>
          <externalDelete/>
          <disableDeferredWipeout>false</disableDeferredWipeout>
          </hudson.plugins.ws__cleanup.PreBuildCleanup>
          </buildWrappers>
          <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl">
          <runSequentially>false</runSequentially>
          </executionStrategy>
          </matrix-project>

          ProjectB:

          <?xml version="1.1" encoding="UTF-8" standalone="no"?><matrix-project plugin="matrix-project@1.14">
          <actions/>
          <description/>
          <keepDependencies>false</keepDependencies>
          <properties>

          </properties>
          <scm class="hudson.plugins.git.GitSCM" plugin="git@4.2.2">
          <configVersion>2</configVersion>
          <userRemoteConfigs>
          <hudson.plugins.git.UserRemoteConfig>
          <url>https://mrw.dev/libraries/libxml-cxx</url>
          </hudson.plugins.git.UserRemoteConfig>
          </userRemoteConfigs>
          <branches>
          <hudson.plugins.git.BranchSpec>
          <name>*/master</name>
          </hudson.plugins.git.BranchSpec>
          </branches>
          <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
          <submoduleCfg class="list"/>
          <extensions>
          <hudson.plugins.git.extensions.impl.WipeWorkspace/>
          </extensions>
          </scm>
          <quietPeriod>5</quietPeriod>
          <scmCheckoutRetryCount>10</scmCheckoutRetryCount>
          <canRoam>true</canRoam>
          <disabled>false</disabled>
          <blockBuildWhenDownstreamBuilding>true</blockBuildWhenDownstreamBuilding>
          <blockBuildWhenUpstreamBuilding>true</blockBuildWhenUpstreamBuilding>
          <triggers>
          <jenkins.triggers.ReverseBuildTrigger>
          <spec/>
          <upstreamProjects>mrw-cxx.win</upstreamProjects>
          <threshold>
          <name>SUCCESS</name>
          <ordinal>0</ordinal>
          <color>BLUE</color>
          <completeBuild>true</completeBuild>
          </threshold>
          </jenkins.triggers.ReverseBuildTrigger>
          <hudson.triggers.TimerTrigger>
          <spec>@monthly</spec>
          </hudson.triggers.TimerTrigger>
          <hudson.triggers.SCMTrigger>
          <spec>H/5 * * * *</spec>
          <ignorePostCommitHooks>false</ignorePostCommitHooks>
          </hudson.triggers.SCMTrigger>
          </triggers>
          <concurrentBuild>false</concurrentBuild>
          <axes>
          <ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis plugin="dynamic-axis@1.0.3">
          <name>mingw</name>
          <values>
          <string>MINGW_ARCHS</string>
          </values>
          <varName>MINGW_ARCHS</varName>
          <axisValues>
          <string>default</string>
          </axisValues>
          </ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis>
          </axes>
          <builders>
          <hudson.plugins.copyartifact.CopyArtifact plugin="copyartifact@1.43">
          <project>mrw-cxx.win/mingw=$mingw</project>
          <filter/>
          <target/>
          <excludes/>
          <selector class="hudson.plugins.copyartifact.StatusBuildSelector"/>
          <doNotFingerprintArtifacts>false</doNotFingerprintArtifacts>
          </hudson.plugins.copyartifact.CopyArtifact>
          <hudson.tasks.Shell>
          <command>docker pull mwaeckerlin/mingw
          docker run --rm -v $(pwd):/workdir -u $(id -u) -e MINGW=$mingw -e BUILD_NUMBER=$BUILD_NUMBER mwaeckerlin/mingw /build.sh -z

          FILE=${JOB_NAME%.win/}-_${mingw%-w64-mingw32}.zip
          TARGET=~/repository/windows/${mingw%-w64-mingw32}/${JOB_NAME%.win/*}
          test -d ${TARGET} || mkdir -p ${TARGET}
          cp ${FILE} ${TARGET}/
          </command>
          </hudson.tasks.Shell>
          </builders>
          <publishers>
          <hudson.tasks.ArtifactArchiver>
          <artifacts>usr/**, *.zip</artifacts>
          <allowEmptyArchive>false</allowEmptyArchive>
          <onlyIfSuccessful>false</onlyIfSuccessful>
          <fingerprint>false</fingerprint>
          <defaultExcludes>true</defaultExcludes>
          <caseSensitive>true</caseSensitive>
          </hudson.tasks.ArtifactArchiver>
          </publishers>
          <buildWrappers>
          <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="ws-cleanup@0.38">
          <deleteDirs>false</deleteDirs>
          <cleanupParameter/>
          <externalDelete/>
          <disableDeferredWipeout>false</disableDeferredWipeout>
          </hudson.plugins.ws__cleanup.PreBuildCleanup>
          </buildWrappers>
          <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl">
          <runSequentially>false</runSequentially>
          </executionStrategy>
          </matrix-project>

          Show
          mrw Marc Wäckerlin added a comment - - edited First of all, it seems, that I am running into the same problem. But for me, it looks strange that this error has been reported such a long time ago (and why is it not fixed in these years?), but I am sure that it used to work short time ago. I cannot exactly say, how long it used to work for me, but I had successful builds. I ran into it now that I am moving my jenkins to a new server and re-running all builds. I am wondering why the priority should be minor, it's a blocker, the plugin is no more usable. My situation: Jenkins ver. 2.225 Tested Copy Artifact 1.43.1 and 1.43 Global Jenkins environment variable: name: MINGW_ARCH value: x86_64-w64-mingw32 i686-w64-mingw32 ProjectA: matrix build: name: mingw variable: MINGW_ARCH Artifacts: * /usr and }} {{.zip ProjectB (built after ProjectA): matrix build: name: mingw variable: MINGW_ARCH Copy Artifacts: project name: ProjectA/mingw=$mingw Artifacts: * /usr and }} {{.zip This results in: ERROR: Unable to find project for artifact copy: ProjectA/mingw=x86_64-w64-mingw32 Then, if I replace the project name to ProjectA/mingw=x86_64-w64-mingw32 , that means hard coded, no variable, then it works. As you see from the error log, the variable substitution is 100% correct! But still there is somehow a failure with regard to the variable. The projects configurations ( ProjectA is mrw-cxx.win and ProjectB is libxml-cxx.win ): ProjectA: <?xml version="1.1" encoding="UTF-8" standalone="no"?><matrix-project plugin="matrix-project@1.14"> <actions/> <description/> <keepDependencies>false</keepDependencies> <properties> </properties> <scm class="hudson.plugins.git.GitSCM" plugin="git@4.2.2"> <configVersion>2</configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <url> https://mrw.dev/libraries/mrw-cxx </url> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name>*/master</name> </hudson.plugins.git.BranchSpec> </branches> <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations> <submoduleCfg class="list"/> <extensions/> </scm> <quietPeriod>5</quietPeriod> <scmCheckoutRetryCount>10</scmCheckoutRetryCount> <canRoam>true</canRoam> <disabled>false</disabled> <blockBuildWhenDownstreamBuilding>true</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>true</blockBuildWhenUpstreamBuilding> <triggers> <hudson.triggers.TimerTrigger> <spec>@monthly</spec> </hudson.triggers.TimerTrigger> <hudson.triggers.SCMTrigger> <spec>H/5 * * * *</spec> <ignorePostCommitHooks>false</ignorePostCommitHooks> </hudson.triggers.SCMTrigger> </triggers> <concurrentBuild>false</concurrentBuild> <axes> <ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis plugin="dynamic-axis@1.0.3"> <name>mingw</name> <values> <string>MINGW_ARCHS</string> </values> <varName>MINGW_ARCHS</varName> <axisValues> <string>default</string> </axisValues> </ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis> </axes> <builders> <hudson.tasks.Shell> <command>docker pull mwaeckerlin/mingw docker run --rm -v $(pwd):/workdir -u $(id -u) -e MINGW=$mingw -e BUILD_NUMBER=$BUILD_NUMBER mwaeckerlin/mingw /build.sh -z FILE=mrw-c++-*_${mingw%-w64-mingw32}.zip TARGET=~/repository/windows/${mingw%-w64-mingw32}/${JOB_NAME%.win/*} test -d ${TARGET} || mkdir -p ${TARGET} cp ${FILE} ${TARGET}/ </command> </hudson.tasks.Shell> </builders> <publishers> <hudson.tasks.ArtifactArchiver> <artifacts>usr/**, *.zip</artifacts> <allowEmptyArchive>false</allowEmptyArchive> <onlyIfSuccessful>false</onlyIfSuccessful> <fingerprint>false</fingerprint> <defaultExcludes>true</defaultExcludes> <caseSensitive>true</caseSensitive> </hudson.tasks.ArtifactArchiver> </publishers> <buildWrappers> <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="ws-cleanup@0.38"> <deleteDirs>false</deleteDirs> <cleanupParameter/> <externalDelete/> <disableDeferredWipeout>false</disableDeferredWipeout> </hudson.plugins.ws__cleanup.PreBuildCleanup> </buildWrappers> <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl"> <runSequentially>false</runSequentially> </executionStrategy> </matrix-project> ProjectB: <?xml version="1.1" encoding="UTF-8" standalone="no"?><matrix-project plugin="matrix-project@1.14"> <actions/> <description/> <keepDependencies>false</keepDependencies> <properties> </properties> <scm class="hudson.plugins.git.GitSCM" plugin="git@4.2.2"> <configVersion>2</configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <url> https://mrw.dev/libraries/libxml-cxx </url> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name>*/master</name> </hudson.plugins.git.BranchSpec> </branches> <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations> <submoduleCfg class="list"/> <extensions> <hudson.plugins.git.extensions.impl.WipeWorkspace/> </extensions> </scm> <quietPeriod>5</quietPeriod> <scmCheckoutRetryCount>10</scmCheckoutRetryCount> <canRoam>true</canRoam> <disabled>false</disabled> <blockBuildWhenDownstreamBuilding>true</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>true</blockBuildWhenUpstreamBuilding> <triggers> <jenkins.triggers.ReverseBuildTrigger> <spec/> <upstreamProjects>mrw-cxx.win</upstreamProjects> <threshold> <name>SUCCESS</name> <ordinal>0</ordinal> <color>BLUE</color> <completeBuild>true</completeBuild> </threshold> </jenkins.triggers.ReverseBuildTrigger> <hudson.triggers.TimerTrigger> <spec>@monthly</spec> </hudson.triggers.TimerTrigger> <hudson.triggers.SCMTrigger> <spec>H/5 * * * *</spec> <ignorePostCommitHooks>false</ignorePostCommitHooks> </hudson.triggers.SCMTrigger> </triggers> <concurrentBuild>false</concurrentBuild> <axes> <ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis plugin="dynamic-axis@1.0.3"> <name>mingw</name> <values> <string>MINGW_ARCHS</string> </values> <varName>MINGW_ARCHS</varName> <axisValues> <string>default</string> </axisValues> </ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis> </axes> <builders> <hudson.plugins.copyartifact.CopyArtifact plugin="copyartifact@1.43"> <project>mrw-cxx.win/mingw=$mingw</project> <filter/> <target/> <excludes/> <selector class="hudson.plugins.copyartifact.StatusBuildSelector"/> <doNotFingerprintArtifacts>false</doNotFingerprintArtifacts> </hudson.plugins.copyartifact.CopyArtifact> <hudson.tasks.Shell> <command>docker pull mwaeckerlin/mingw docker run --rm -v $(pwd):/workdir -u $(id -u) -e MINGW=$mingw -e BUILD_NUMBER=$BUILD_NUMBER mwaeckerlin/mingw /build.sh -z FILE=${JOB_NAME%.win/ }- _${mingw%-w64-mingw32}.zip TARGET=~/repository/windows/${mingw%-w64-mingw32}/${JOB_NAME%.win/*} test -d ${TARGET} || mkdir -p ${TARGET} cp ${FILE} ${TARGET}/ </command> </hudson.tasks.Shell> </builders> <publishers> <hudson.tasks.ArtifactArchiver> <artifacts>usr/**, *.zip</artifacts> <allowEmptyArchive>false</allowEmptyArchive> <onlyIfSuccessful>false</onlyIfSuccessful> <fingerprint>false</fingerprint> <defaultExcludes>true</defaultExcludes> <caseSensitive>true</caseSensitive> </hudson.tasks.ArtifactArchiver> </publishers> <buildWrappers> <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="ws-cleanup@0.38"> <deleteDirs>false</deleteDirs> <cleanupParameter/> <externalDelete/> <disableDeferredWipeout>false</disableDeferredWipeout> </hudson.plugins.ws__cleanup.PreBuildCleanup> </buildWrappers> <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl"> <runSequentially>false</runSequentially> </executionStrategy> </matrix-project>
          Hide
          mrw Marc Wäckerlin added a comment -

          My personal Workaround:

          I removed the artifact copy plugin and replaced the copy artifact build step by a freeform shell script as follows, which does the same as this plugin, just set the name of the source project in FROM:

          FROM=mrw-cxx.win

          STABLE=$(sed -n 's/^lastSuccessfulBuild //p' ~/jobs/${FROM}/builds/permalinks)
          SOURCE=~/jobs/${FROM}/configurations/axis-mingw/${mingw}/builds/${STABLE}/archive/usr

          cp -a ${SOURCE} .

          Show
          mrw Marc Wäckerlin added a comment - My personal Workaround: I removed the artifact copy plugin and replaced the copy artifact build step by a freeform shell script as follows, which does the same as this plugin, just set the name of the source project in FROM : FROM=mrw-cxx.win STABLE=$(sed -n 's/^lastSuccessfulBuild //p' ~/jobs/${FROM}/builds/permalinks) SOURCE=~/jobs/${FROM}/configurations/axis-mingw/${mingw}/builds/${STABLE}/archive/usr cp -a ${SOURCE} .

            People

            • Assignee:
              Unassigned
              Reporter:
              matty666 Matt Evans
            • Votes:
              10 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated: