Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7752

Allow copyartifact to iterate and fetch all artifacts from all configurations in a matrix-build.

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We would like to see support for iterating over all configurations and artifacts created by a matrix job.

      A typical scenario:

      1. Master Job do some init stuff
      2. Master Job triggers:
      2.1 Matrix job (with a large number of configurations)
      2.2 Matrix job finishes
      3. Master job now collects the artifacts from the matrix-job and do some post processing.

      We need now to add a large amount of copy artifact steps in the master-job. Which is not really manageable.

      It would be nice to have a build selector which iterates over all matrix configurations and copies the artifacts to the specified destination path.

        Attachments

          Activity

          Hide
          mindless Alan Harder added a comment -

          How do you think this should work?
          If you create a copy-artifact build step in a job and put a matrix job for the project name (without specifying /PARAM=VALUE to pick a particular configuration) then it could loop through all configs and copy stuff.. what if the output filenames in the different configurations conflict? They would overwrite eachother I guess.. I wonder if the target directory in the build step could somehow reference an axis value to create a unique subdir.
          ie, Matrix job "foo" creates and archives "target/out.zip" in 3 different configurations, MYAXIS=abc, def or ghi.
          downstream job has copy-artifact build step pointing to "foo" project with target directory "data/$MYAXIS".. it could then copy files to:
          data/abc/target/out.zip
          data/def/target/out.zip
          data/ghi/target/out.zip
          or "flatten" could be checked to omit the "target/" part of the path here.
          This sounds nice, but parameter references here would be from params of THIS job, not axis values in the other job.. so not sure how to make this work cleanly if there are filename conflicts. I suppose it could automatically create subdirs base on the name of each matrix configuration.

          Show
          mindless Alan Harder added a comment - How do you think this should work? If you create a copy-artifact build step in a job and put a matrix job for the project name (without specifying /PARAM=VALUE to pick a particular configuration) then it could loop through all configs and copy stuff.. what if the output filenames in the different configurations conflict? They would overwrite eachother I guess.. I wonder if the target directory in the build step could somehow reference an axis value to create a unique subdir. ie, Matrix job "foo" creates and archives "target/out.zip" in 3 different configurations, MYAXIS=abc, def or ghi. downstream job has copy-artifact build step pointing to "foo" project with target directory "data/$MYAXIS".. it could then copy files to: data/abc/target/out.zip data/def/target/out.zip data/ghi/target/out.zip or "flatten" could be checked to omit the "target/" part of the path here. This sounds nice, but parameter references here would be from params of THIS job, not axis values in the other job.. so not sure how to make this work cleanly if there are filename conflicts. I suppose it could automatically create subdirs base on the name of each matrix configuration.
          Hide
          glundh glundh added a comment -

          > I wonder if the target directory in the build step
          > could somehow reference an axis value to create a
          > unique subdir.
          > ie, Matrix job "foo" creates and archives
          > "target/out.zip" in 3 different configurations,
          > MYAXIS=abc, def or ghi.
          > downstream job has copy-artifact build step pointing
          > to "foo" project with target directory "data/$MYAXIS"..
          > it could then copy files to:
          > data/abc/target/out.zip
          > data/def/target/out.zip
          > data/ghi/target/out.zip

          This sounds perfect!

          > This sounds nice, but parameter references here
          > would be from params of THIS job, not axis values
          > in the other job..

          Yes. In that case I guess the axis parameters has to be fetched from the target matrix build and then expended together with the destination builds parameters. Not sure about the best to do it though.

          > I suppose it could automatically create subdirs
          > base on the name of each matrix configuration.

          Something like that.

          I'm really glad that this feature request is being looked into. Thanks!

          Show
          glundh glundh added a comment - > I wonder if the target directory in the build step > could somehow reference an axis value to create a > unique subdir. > ie, Matrix job "foo" creates and archives > "target/out.zip" in 3 different configurations, > MYAXIS=abc, def or ghi. > downstream job has copy-artifact build step pointing > to "foo" project with target directory "data/$MYAXIS".. > it could then copy files to: > data/abc/target/out.zip > data/def/target/out.zip > data/ghi/target/out.zip This sounds perfect! > This sounds nice, but parameter references here > would be from params of THIS job, not axis values > in the other job.. Yes. In that case I guess the axis parameters has to be fetched from the target matrix build and then expended together with the destination builds parameters. Not sure about the best to do it though. > I suppose it could automatically create subdirs > base on the name of each matrix configuration. Something like that. I'm really glad that this feature request is being looked into. Thanks!
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : mindless
          Path:
          trunk/hudson/plugins/copyartifact/pom.xml
          trunk/hudson/plugins/copyartifact/src/main/java/hudson/plugins/copyartifact/CopyArtifact.java
          trunk/hudson/plugins/copyartifact/src/main/java/hudson/plugins/copyartifact/FilePathCopyMethod.java
          trunk/hudson/plugins/copyartifact/src/main/resources/hudson/plugins/copyartifact/CopyArtifact/help-projectName.html
          trunk/hudson/plugins/copyartifact/src/main/resources/hudson/plugins/copyartifact/Messages.properties
          trunk/hudson/plugins/copyartifact/src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java
          trunk/hudson/plugins/copyartifact/src/test/resources/hudson/plugins/copyartifact/maven-job.zip
          http://jenkins-ci.org/commit/36753
          Log:
          [copyartifact] [FIXED JENKINS-7752] copy artifacts from ALL modules/configurations
          when a maven/matrix project is selected.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : mindless Path: trunk/hudson/plugins/copyartifact/pom.xml trunk/hudson/plugins/copyartifact/src/main/java/hudson/plugins/copyartifact/CopyArtifact.java trunk/hudson/plugins/copyartifact/src/main/java/hudson/plugins/copyartifact/FilePathCopyMethod.java trunk/hudson/plugins/copyartifact/src/main/resources/hudson/plugins/copyartifact/CopyArtifact/help-projectName.html trunk/hudson/plugins/copyartifact/src/main/resources/hudson/plugins/copyartifact/Messages.properties trunk/hudson/plugins/copyartifact/src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java trunk/hudson/plugins/copyartifact/src/test/resources/hudson/plugins/copyartifact/maven-job.zip http://jenkins-ci.org/commit/36753 Log: [copyartifact] [FIXED JENKINS-7752] copy artifacts from ALL modules/configurations when a maven/matrix project is selected.
          Hide
          paoloferracin paoloferracin added a comment -

          Hello,
          the fix for issue 7752 prevents the plugin to copy any manually archived artifact (using the 'Archive the artifacts' post-build option) for maven projects as the previous version did, and completely breaks the build if 'Disable automatic artifact archiving' is selected in the source job.

          this is a simple patch to keep the prev this behaviour, I'd be glad if you consider applying upstream.

          Show
          paoloferracin paoloferracin added a comment - Hello, the fix for issue 7752 prevents the plugin to copy any manually archived artifact (using the 'Archive the artifacts' post-build option) for maven projects as the previous version did, and completely breaks the build if 'Disable automatic artifact archiving' is selected in the source job. this is a simple patch to keep the prev this behaviour, I'd be glad if you consider applying upstream.
          Hide
          mindless Alan Harder added a comment -

          (from 30-Jan 10:30am)

          Oops, lost this in my inbox.. I'll incorporate asap.

          One question.. the "exists()" checks you added don't seem related to the maven job problem.. why did you add those?

          Show
          mindless Alan Harder added a comment - (from 30-Jan 10:30am) Oops, lost this in my inbox.. I'll incorporate asap. One question.. the "exists()" checks you added don't seem related to the maven job problem.. why did you add those?
          Hide
          mindless Alan Harder added a comment -

          (from 30-Jan 3:55pm)

          If you hit a case where those checks gave a better error message, seems ok to add that for good measure, so I'm including that in 1.12.. thanks.

          Show
          mindless Alan Harder added a comment - (from 30-Jan 3:55pm) If you hit a case where those checks gave a better error message, seems ok to add that for good measure, so I'm including that in 1.12.. thanks.

            People

            • Assignee:
              mindless Alan Harder
              Reporter:
              glundh glundh
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: