Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-17680

Upgrade to new Copy Artifacts version erases all job names to copy from

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      After an update to the new copy artifacts version 1.26 existing configurations are broken because all job names to copy artifacts from are empty. The configuration files still contain the correct names, e.g.:

      <hudson.plugins.copyartifact.CopyArtifact plugin="copyartifact@1.25">
      <projectName>rsc-trunk/label=$label</projectName>
      <filter>*/.tar.gz</filter>
      <target>upstream</target>
      <selector class="hudson.plugins.copyartifact.StatusBuildSelector"/>
      <flatten>true</flatten>
      </hudson.plugins.copyartifact.CopyArtifact>

      As you can see, most of our jobs use variables in the projectName tag. Maybe this is the reason for the problem. I actually cannot find a job without this, so I cannot verify this idea.

      With 1.25 everything is ok.

        Attachments

          Issue Links

            Activity

            languitar Johannes Wienke created issue -
            Hide
            maxzilla Max Wahler added a comment -

            I've also seen this issue. In 1.26, the tag name changed from projectName to project. I edited all our config.xml files manually and it does the trick. Bad news is, that saving the configuration in Jenkins throws an exception.

            Show
            maxzilla Max Wahler added a comment - I've also seen this issue. In 1.26, the tag name changed from projectName to project. I edited all our config.xml files manually and it does the trick. Bad news is, that saving the configuration in Jenkins throws an exception.
            Hide
            languitar Johannes Wienke added a comment -

            I'd really like to see an automatic migration path for this.

            Show
            languitar Johannes Wienke added a comment - I'd really like to see an automatic migration path for this.
            Hide
            marvix Marvi Benedet added a comment - - edited

            if you have a lot of projects to correct:

            cd <JENKINS-HOME>/jobs
            tar cfz config_bak.tgz */config.xml
            sed 's/projectName>/project>/g' -i */config.xml

            (if you use it in promotions also: )

            tar cfz config_bak_promotions.tgz */promotions/*/config.xml
            sed 's/projectName>/project>/g' -i */promotions/*/config.xml

            in web interface:
            manage jenkins/reload configurations from disk

            Show
            marvix Marvi Benedet added a comment - - edited if you have a lot of projects to correct: cd <JENKINS-HOME>/jobs tar cfz config_bak.tgz */config.xml sed 's/projectName>/project>/g' -i */config.xml (if you use it in promotions also: ) tar cfz config_bak_promotions.tgz */promotions/*/config.xml sed 's/projectName>/project>/g' -i */promotions/*/config.xml in web interface: manage jenkins/reload configurations from disk
            Hide
            alanbirtles Alan Birtles added a comment -

            I've upgraded from 1.25 to 1.27 and it seems to automatically perform the upgrade when a project is built. Unfortunately I modified a few jobs before they were built so encountered this bug

            Show
            alanbirtles Alan Birtles added a comment - I've upgraded from 1.25 to 1.27 and it seems to automatically perform the upgrade when a project is built. Unfortunately I modified a few jobs before they were built so encountered this bug
            Hide
            ikedam ikedam added a comment -

            This seems caused by 6243f6bcd5, splitting projectName into project and parameters.
            This can be fixed with readResolve(), as described in https://wiki.jenkins-ci.org/display/JENKINS/Hint+on+retaining+backward+compatibility

            Show
            ikedam ikedam added a comment - This seems caused by 6243f6bcd5 , splitting projectName into project and parameters . This can be fixed with readResolve() , as described in  https://wiki.jenkins-ci.org/display/JENKINS/Hint+on+retaining+backward+compatibility
            Hide
            ikedam ikedam added a comment -

            There seems a little complex problem:

            • project names in Copy Artifact < 1.26 accept axis parameters and build parameters by splitting with a slash.
            • project names in Copy Artifact >= 1.26 accept only axis parameters, no longer build parameters.
              • This is the same bahavior to Jenkins core.

            So readResolve() cannot simply split a old project name into a project name and parameters. It must consider whether values after a slash are axis parameters or build parameters.

            Show
            ikedam ikedam added a comment - There seems a little complex problem: project names in Copy Artifact < 1.26 accept axis parameters and build parameters by splitting with a slash. project names in Copy Artifact >= 1.26 accept only axis parameters, no longer build parameters. This is the same bahavior to Jenkins core. So readResolve() cannot simply split a old project name into a project name and parameters. It must consider whether values after a slash are axis parameters or build parameters.
            Hide
            ikedam ikedam added a comment -

            I posted a pull request.
            For it is difficult to fix the configuration automatically, a warning message will be displayed.
            https://github.com/jenkinsci/copyartifact-plugin/pull/21

            But it is not reviewed for a month...
            There also left some other reviews, and no maintainer seems available...

            Show
            ikedam ikedam added a comment - I posted a pull request. For it is difficult to fix the configuration automatically, a warning message will be displayed. https://github.com/jenkinsci/copyartifact-plugin/pull/21 But it is not reviewed for a month... There also left some other reviews, and no maintainer seems available...
            Hide
            jglick Jesse Glick added a comment -

            The upgrade is automatic if you simply build your project after updating the plugin. You must not configure the project before building it. I closed PR #21 since it does not really solve the problem; there are various ways the storage could be upgraded automatically without requiring a build.

            Show
            jglick Jesse Glick added a comment - The upgrade is automatic if you simply build your project after updating the plugin. You must not configure the project before building it. I closed PR #21 since it does not really solve the problem; there are various ways the storage could be upgraded automatically without requiring a build.
            Hide
            ikedam ikedam added a comment -

            > The upgrade is automatic if you simply build your project after updating the plugin.

            I know that and that cannot resolve the problem at all.
            The problem is that, modifying the configuration before building breaks the configuration, and there is a case that a project administrator and a Jenkins administrator are different.

            > there are various ways the storage could be upgraded automatically without requiring a build.

            Do you plan to work that?
            Or please let me know the outline of those ways, and I work it.

            I think the difficulty is the case a project is not contained in ItemGroup other than Jenkins, and cannot be enumerated with Jenkins#getItems.
            I have no idea about the proper way to visit all projects.

            Show
            ikedam ikedam added a comment - > The upgrade is automatic if you simply build your project after updating the plugin. I know that and that cannot resolve the problem at all. The problem is that, modifying the configuration before building breaks the configuration, and there is a case that a project administrator and a Jenkins administrator are different. > there are various ways the storage could be upgraded automatically without requiring a build. Do you plan to work that? Or please let me know the outline of those ways, and I work it. I think the difficulty is the case a project is not contained in ItemGroup other than Jenkins , and cannot be enumerated with Jenkins#getItems . I have no idea about the proper way to visit all projects.
            Hide
            jglick Jesse Glick added a comment -

            Do you plan to work [on] that?

            Not currently.

            please let me know the outline of those ways

            Probably something like

            @Initializer(after=JOB_LOADED)
            public static void upgradeAll() {
                for (AbstractProject p : Jenkins.getInstance().getAllItems(AbstractProject.class)) {
                    try {
                        ListenerImpl.getCopiers(p); // calls upgradeIfNecessary as a side effect
                    } catch (IOException x) {
                        Logger.getLogger(CopyArtifact.class.getName()).log(Level.WARNING, "could not upgrade configuration of " + p, x);
                    }
                }
            }
            

            except you would also want to set a persisted flag in CopyArtifactPlugin so that this work is only done once, not after every startup.

            Show
            jglick Jesse Glick added a comment - Do you plan to work [on] that? Not currently. please let me know the outline of those ways Probably something like @Initializer(after=JOB_LOADED) public static void upgradeAll() { for (AbstractProject p : Jenkins.getInstance().getAllItems(AbstractProject.class)) { try { ListenerImpl.getCopiers(p); // calls upgradeIfNecessary as a side effect } catch (IOException x) { Logger.getLogger(CopyArtifact. class. getName()).log(Level.WARNING, "could not upgrade configuration of " + p, x); } } } except you would also want to set a persisted flag in CopyArtifactPlugin so that this work is only done once, not after every startup.
            Hide
            ikedam ikedam added a comment -

            I verified Jenkins#getAlItems() visits all projects even they are contained in ItemGroup other than Jenkins.
            I work with that. Thanks!

            Show
            ikedam ikedam added a comment - I verified Jenkins#getAlItems() visits all projects even they are contained in ItemGroup other than Jenkins . I work with that. Thanks!
            Hide
            ikedam ikedam added a comment -
            Show
            ikedam ikedam added a comment - Send a pull request: https://github.com/jenkinsci/copyartifact-plugin/pull/23
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/main/java/hudson/plugins/copyartifact/CopyArtifact.java
            src/main/java/hudson/plugins/copyartifact/CopyArtifactUpgradeListener.java
            http://jenkins-ci.org/commit/copyartifact-plugin/addc9530dc24d04ae21002fed8f6ef86a27e32a3
            Log:
            JENKINS-17680 Upgrade the configuration < 1.26 automatically.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/main/java/hudson/plugins/copyartifact/CopyArtifact.java src/main/java/hudson/plugins/copyartifact/CopyArtifactUpgradeListener.java http://jenkins-ci.org/commit/copyartifact-plugin/addc9530dc24d04ae21002fed8f6ef86a27e32a3 Log: JENKINS-17680 Upgrade the configuration < 1.26 automatically.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java
            http://jenkins-ci.org/commit/copyartifact-plugin/7214a147cd8a5ebb88bdfce6ae3e5b03b341e8a2
            Log:
            [FIXED JENKINS-17680] Update a test for configuration-upgrading.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java http://jenkins-ci.org/commit/copyartifact-plugin/7214a147cd8a5ebb88bdfce6ae3e5b03b341e8a2 Log: [FIXED JENKINS-17680] Update a test for configuration-upgrading.
            scm_issue_link SCM/JIRA link daemon made changes -
            Field Original Value New Value
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/main/java/hudson/plugins/copyartifact/CopyArtifact.java
            http://jenkins-ci.org/commit/copyartifact-plugin/bc9cc1894ac32934b912b004784c8ad92fa5ae19
            Log:
            JENKINS-17680 Make minimized diffs.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/main/java/hudson/plugins/copyartifact/CopyArtifact.java http://jenkins-ci.org/commit/copyartifact-plugin/bc9cc1894ac32934b912b004784c8ad92fa5ae19 Log: JENKINS-17680 Make minimized diffs.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/main/java/hudson/plugins/copyartifact/CopyArtifact.java
            src/main/java/hudson/plugins/copyartifact/CopyArtifactUpgradeListener.java
            http://jenkins-ci.org/commit/copyartifact-plugin/854deb121eace1d0f4a90664a6d2d2ad733e0d61
            Log:
            JENKINS-17680 use @Initializer instead of ItemListener.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/main/java/hudson/plugins/copyartifact/CopyArtifact.java src/main/java/hudson/plugins/copyartifact/CopyArtifactUpgradeListener.java http://jenkins-ci.org/commit/copyartifact-plugin/854deb121eace1d0f4a90664a6d2d2ad733e0d61 Log: JENKINS-17680 use @Initializer instead of ItemListener.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/main/java/hudson/plugins/copyartifact/CopyArtifact.java
            src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java
            src/test/resources/hudson/plugins/copyartifact/CopyArtifactTest/testProjectNameSplit.zip
            http://jenkins-ci.org/commit/copyartifact-plugin/b42c2158119f8c8b06d0562bbd1d9bf7b7ee5f49
            Log:
            JENKINS-17680 Upgrade is applied also to MatrixProject.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/main/java/hudson/plugins/copyartifact/CopyArtifact.java src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java src/test/resources/hudson/plugins/copyartifact/CopyArtifactTest/testProjectNameSplit.zip http://jenkins-ci.org/commit/copyartifact-plugin/b42c2158119f8c8b06d0562bbd1d9bf7b7ee5f49 Log: JENKINS-17680 Upgrade is applied also to MatrixProject.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java
            src/test/resources/hudson/plugins/copyartifact/CopyArtifactTest/testWrappedCopierProjectNameSplit.zip
            http://jenkins-ci.org/commit/copyartifact-plugin/3c6babe659e45386de6db676054d4eb0b9b9de4a
            Log:
            JENKINS-17680 Added a test for verifying upgrading on building works.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java src/test/resources/hudson/plugins/copyartifact/CopyArtifactTest/testWrappedCopierProjectNameSplit.zip http://jenkins-ci.org/commit/copyartifact-plugin/3c6babe659e45386de6db676054d4eb0b9b9de4a Log: JENKINS-17680 Added a test for verifying upgrading on building works.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/main/java/hudson/plugins/copyartifact/CopyArtifact.java
            http://jenkins-ci.org/commit/copyartifact-plugin/355aedb310d61d2f26e235630b4fc99c7c886e73
            Log:
            JENKINS-17680 Changed a confusiong method name.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/main/java/hudson/plugins/copyartifact/CopyArtifact.java http://jenkins-ci.org/commit/copyartifact-plugin/355aedb310d61d2f26e235630b4fc99c7c886e73 Log: JENKINS-17680 Changed a confusiong method name.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/main/java/hudson/plugins/copyartifact/CopyArtifact.java
            src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java
            src/test/resources/hudson/plugins/copyartifact/CopyArtifactTest/testProjectNameSplit.zip
            src/test/resources/hudson/plugins/copyartifact/CopyArtifactTest/testWrappedCopierProjectNameSplit.zip
            http://jenkins-ci.org/commit/copyartifact-plugin/f13ba75afffcd971475875873fefc398600a084b
            Log:
            Merge pull request #23 from ikedam/feature/JENKINS-17680-3

            JENKINS-17680 Upgrade configurations from version 1.25

            Compare: https://github.com/jenkinsci/copyartifact-plugin/compare/428d32a6be6b...f13ba75afffc

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/main/java/hudson/plugins/copyartifact/CopyArtifact.java src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java src/test/resources/hudson/plugins/copyartifact/CopyArtifactTest/testProjectNameSplit.zip src/test/resources/hudson/plugins/copyartifact/CopyArtifactTest/testWrappedCopierProjectNameSplit.zip http://jenkins-ci.org/commit/copyartifact-plugin/f13ba75afffcd971475875873fefc398600a084b Log: Merge pull request #23 from ikedam/feature/ JENKINS-17680 -3 JENKINS-17680 Upgrade configurations from version 1.25 Compare: https://github.com/jenkinsci/copyartifact-plugin/compare/428d32a6be6b...f13ba75afffc
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: ikedam
            Path:
            src/main/java/hudson/plugins/copyartifact/CopyArtifact.java
            src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java
            http://jenkins-ci.org/commit/copyartifact-plugin/f468296f905b23eebf0a36790612b100c8d7a8cd
            Log:
            Fixed unnecessary codes added by changes for JENKINS-17680.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: ikedam Path: src/main/java/hudson/plugins/copyartifact/CopyArtifact.java src/test/java/hudson/plugins/copyartifact/CopyArtifactTest.java http://jenkins-ci.org/commit/copyartifact-plugin/f468296f905b23eebf0a36790612b100c8d7a8cd Log: Fixed unnecessary codes added by changes for JENKINS-17680 .
            Hide
            ikedam ikedam added a comment -

            Fixed in Copy Artifact 1.28.

            Show
            ikedam ikedam added a comment - Fixed in Copy Artifact 1.28.
            ikedam ikedam made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            Assignee ikedam [ ikedam ]
            Hide
            mika Michael Prokop added a comment - - edited

            If the config.xml of a job includes both the project and projectName settings, being identical like:

            <project>demojob/architecture=$architecture</project>
            <projectName>demojob/architecture=$architecture</projectName>

            then it fails to run with current version(s) of the Copy Artifacts plugin (1.28) because no artifacts can be copied from axis architecture of demojob ("Unable to find a build for artifact copy from: demojob") even though the artifacts are there.

            The problem about it is that the web interface displays the correct configuration (Project Name: "demojob/architecture=$architecture" and "Parameter filters" being empty) but the wrong configuration is actually used (Project Name: "demojob" and "Parameter filters: "architecture=$architecture"). When you execute the job the Copy Artifacts plugin automatically converts the configuration to:

            <project>demojob</project>
            <parameters>architecture=$architecture</parameters>

            which is something different and doesn't work. When you manually select "Configure" on the original job (the one with project + projectName settings being identical) in the web interface and then click on "Save" it's fine though.

            This is a problem for folks generating config.xml in an automated way (where you never manually click "Configure" but generate the config.xml from scratch). The mentioned configuration is used for example in jenkins-job-builder:

            https://github.com/openstack-infra/jenkins-job-builder/blob/master/jenkins_jobs/modules/builders.py#L122

            I can share configurations for jenkins-job-builder to easily reproduce the behavior if needed.

            Any idea how a smoother upgrade path could be possible? I'm pretty sure this is breaking quite some setups.

            Show
            mika Michael Prokop added a comment - - edited If the config.xml of a job includes both the project and projectName settings, being identical like: <project>demojob/architecture=$architecture</project> <projectName>demojob/architecture=$architecture</projectName> then it fails to run with current version(s) of the Copy Artifacts plugin (1.28) because no artifacts can be copied from axis architecture of demojob ("Unable to find a build for artifact copy from: demojob") even though the artifacts are there. The problem about it is that the web interface displays the correct configuration (Project Name: "demojob/architecture=$architecture" and "Parameter filters" being empty) but the wrong configuration is actually used (Project Name: "demojob" and "Parameter filters: "architecture=$architecture"). When you execute the job the Copy Artifacts plugin automatically converts the configuration to: <project>demojob</project> <parameters>architecture=$architecture</parameters> which is something different and doesn't work. When you manually select "Configure" on the original job (the one with project + projectName settings being identical) in the web interface and then click on "Save" it's fine though. This is a problem for folks generating config.xml in an automated way (where you never manually click "Configure" but generate the config.xml from scratch). The mentioned configuration is used for example in jenkins-job-builder: https://github.com/openstack-infra/jenkins-job-builder/blob/master/jenkins_jobs/modules/builders.py#L122 I can share configurations for jenkins-job-builder to easily reproduce the behavior if needed. Any idea how a smoother upgrade path could be possible? I'm pretty sure this is breaking quite some setups.
            mika Michael Prokop made changes -
            Resolution Fixed [ 1 ]
            Status Closed [ 6 ] Reopened [ 4 ]
            Hide
            jglick Jesse Glick added a comment -

            The projectName field is deprecated, and may not be used together with project; there is a one-way conversion of projectName (which gets deleted) to project plus parameters. If jenkins-job-builder writes out a configuration including both fields then it is simply buggy.

            Show
            jglick Jesse Glick added a comment - The projectName field is deprecated, and may not be used together with project ; there is a one-way conversion of projectName (which gets deleted) to project plus parameters . If jenkins-job-builder writes out a configuration including both fields then it is simply buggy.
            Hide
            mika Michael Prokop added a comment -

            Alright, sad but fair enough - thanks for clarification. I've filed a patch upstream against jenkins-job-builder (FTR: https://review.openstack.org/#/c/56412/).

            Show
            mika Michael Prokop added a comment - Alright, sad but fair enough - thanks for clarification. I've filed a patch upstream against jenkins-job-builder (FTR: https://review.openstack.org/#/c/56412/ ).
            ikedam ikedam made changes -
            Link This issue is related to JENKINS-19057 [ JENKINS-19057 ]
            ikedam ikedam made changes -
            Status Reopened [ 4 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 148860 ] JNJira + In-Review [ 192909 ]

              People

              • Assignee:
                ikedam ikedam
                Reporter:
                languitar Johannes Wienke
              • Votes:
                3 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: