Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-5807

Promotion operations occurs on slave and not master where artifacts are located.

    Details

    • Similar Issues:

      Description

      There is no way to copy the artifacts as a promotion to another directory. When using existing plugins (promotion + scp) the jobs execute on the wrong slave.

      What I really want is the archived artifact from the master (not the masters workspace, or the original slaves workspace, but what the master serves from the UI) to be copied into another directory.

      Background:

      The artifacts from hudson are so large that we can no longer back them up across a single 60GB tape. So we want to promote specific QA-approved artifacts and copy them to another directory which IS backed up. The ability to delete artifacts while retaining metadata would also be helpful, but is unsafe and destructive rather than making another copy in a separate location.

        Attachments

          Activity

          jlongman jlongman created issue -
          jlongman jlongman made changes -
          Field Original Value New Value
          Description There is no way to copy the artifacts as a promotion to another directory. When using existing plugins (promotion + scp) the jobs execute on the wrong slave.

          What I *really& want is the archived artifact from the _master_ (not the masters workspace, or the original slaves workspace, but what the master serves from the UI) to be copied into another directory.

          Background:

          The artifacts from hudson are so large that we can no longer back them up across a single 60GB tape. So we want to promote specific QA-approved artifacts and copy them to another directory which IS backed up. The ability to delete artifacts while retaining metadata would also be helpful, but is unsafe and destructive rather than making another copy in a separate location.
          There is no way to copy the artifacts as a promotion to another directory. When using existing plugins (promotion + scp) the jobs execute on the wrong slave.

          What I *really* want is the archived artifact from the _master_ (not the masters workspace, or the original slaves workspace, but what the master serves from the UI) to be copied into another directory.

          Background:

          The artifacts from hudson are so large that we can no longer back them up across a single 60GB tape. So we want to promote specific QA-approved artifacts and copy them to another directory which IS backed up. The ability to delete artifacts while retaining metadata would also be helpful, but is unsafe and destructive rather than making another copy in a separate location.
          Hide
          mindless Alan Harder added a comment -

          See related JENKINS-4089. I think the consensus is that promotion should run where the promoted build ran, so it wouldn't run on the master unless your job is tied to the master.
          FYI, ability to delete artifacts from old builds w/o deleting build metadata itself was recently added in Hudson core.

          Show
          mindless Alan Harder added a comment - See related JENKINS-4089 . I think the consensus is that promotion should run where the promoted build ran, so it wouldn't run on the master unless your job is tied to the master. FYI, ability to delete artifacts from old builds w/o deleting build metadata itself was recently added in Hudson core.
          Hide
          mindless Alan Harder added a comment -

          You could use wget or Copy Artifacts plugin to get artifacts to the local dir, then use scp directly or SCP plugin to move files elsewhere.. let us know if there is something more we can do, or if we can close this in favor of JENKINS-4089, thanks.

          Show
          mindless Alan Harder added a comment - You could use wget or Copy Artifacts plugin to get artifacts to the local dir, then use scp directly or SCP plugin to move files elsewhere.. let us know if there is something more we can do, or if we can close this in favor of JENKINS-4089 , thanks.
          Hide
          jlongman jlongman added a comment - - edited

          I really want to access the artifacts on the master. These should be the same as what is on the original slave where the build ran, unless that workspace has been deleted. Which means I really want to access the artifacts on the master.

          Again, I want an adhoc copy of the artifacts, so I can't make that decision at build time. It must be made after the workspace has potentially been deleted.

          (Copy artifacts has the same problem afaik.)

          Wget is is surprisingly tricky considering we use **/*.tgz globs to define artifacts, which means we'd have to spider the webpage in order to get the artifacts, which means we might walk down a link to another build and get artifacts we don't want.

          I'm starting to see that the problem is more about how plugins work in hudson?

          Show
          jlongman jlongman added a comment - - edited I really want to access the artifacts on the master. These should be the same as what is on the original slave where the build ran, unless that workspace has been deleted. Which means I really want to access the artifacts on the master. Again, I want an adhoc copy of the artifacts, so I can't make that decision at build time. It must be made after the workspace has potentially been deleted. (Copy artifacts has the same problem afaik.) Wget is is surprisingly tricky considering we use **/*.tgz globs to define artifacts, which means we'd have to spider the webpage in order to get the artifacts, which means we might walk down a link to another build and get artifacts we don't want. I'm starting to see that the problem is more about how plugins work in hudson?
          Hide
          mindless Alan Harder added a comment -

          I'm not sure I follow the statement "Copy artifacts has the same problem afaik".. that plugin copies from the artifacts area, not the workspace. I thought you could add a "Copy artifacts from another project" build step into your promotion. This will copy artifacts from the master to wherever your promotion runs (as this may change if JENKINS-4089 is fixed). You can then have a shell step in your promotion to do whatever is needed with the artifact(s) that were copied. Does this help?

          Show
          mindless Alan Harder added a comment - I'm not sure I follow the statement "Copy artifacts has the same problem afaik".. that plugin copies from the artifacts area, not the workspace. I thought you could add a "Copy artifacts from another project" build step into your promotion. This will copy artifacts from the master to wherever your promotion runs (as this may change if JENKINS-4089 is fixed). You can then have a shell step in your promotion to do whatever is needed with the artifact(s) that were copied. Does this help?
          Hide
          mindless Alan Harder added a comment -

          Will close soon without further input, thanks.

          Show
          mindless Alan Harder added a comment - Will close soon without further input, thanks.
          Hide
          jlongman jlongman added a comment -

          I'll try your workaround.

          Show
          jlongman jlongman added a comment - I'll try your workaround.
          Hide
          jlongman jlongman added a comment -

          I can't test the workaround: the job is a matrix job and it looks like CopyArtifact plugin is not getting the artifacts successfully.

          Copied 0 artifacts from 3.1.0 #93
          failed build hudson.plugins.copyartifact.CopyArtifact@399e7660 SUCCESS
          

          To be clear my config for CopyArtifacts is the host project and the file selector */archive.tgz. I tried using 3.1.0/PRODUCT=$PRODUCT for the project specifier, as the CopyArchive plugin suggests, but of course the promotion isn't a matrix. MAYBE Aggregate the Archived Artifacts will grab this, or another plugin. Matrix jobs seem to have problems.

          It is also difficult without being able to grab the original build number since obviously I want to promote the promoted jobs artifacts, but that is a separate issue.

          Show
          jlongman jlongman added a comment - I can't test the workaround: the job is a matrix job and it looks like CopyArtifact plugin is not getting the artifacts successfully. Copied 0 artifacts from 3.1.0 #93 failed build hudson.plugins.copyartifact.CopyArtifact@399e7660 SUCCESS To be clear my config for CopyArtifacts is the host project and the file selector * /archive .tgz . I tried using 3.1.0/PRODUCT=$PRODUCT for the project specifier, as the CopyArchive plugin suggests, but of course the promotion isn't a matrix. MAYBE Aggregate the Archived Artifacts will grab this, or another plugin. Matrix jobs seem to have problems. It is also difficult without being able to grab the original build number since obviously I want to promote the promoted jobs artifacts, but that is a separate issue.
          Hide
          mindless Alan Harder added a comment -

          Make sure you tell CopyArtifacts the particular matrix configuration to pull from (click the help icon for info about this), and make sure the selector is valid..

          Show
          mindless Alan Harder added a comment - Make sure you tell CopyArtifacts the particular matrix configuration to pull from (click the help icon for info about this), and make sure the selector is valid..
          Hide
          petehayes Peter Hayes added a comment -

          Seems like this is probably not an issue and further. I will close if not.

          Show
          petehayes Peter Hayes added a comment - Seems like this is probably not an issue and further. I will close if not.
          Hide
          stigkj Stig Kleppe-Jørgensen added a comment -

          Seems the workaround is good enough, so closing this issue as 'won't fix'.

          Show
          stigkj Stig Kleppe-Jørgensen added a comment - Seems the workaround is good enough, so closing this issue as 'won't fix'.
          stigkj Stig Kleppe-Jørgensen made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Won't Fix [ 2 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 135899 ] JNJira + In-Review [ 187069 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              jlongman jlongman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: