Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-4861

Post-build deployment of Maven artifacts does not work with scpexe-type repository

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: maven-plugin
    • Labels:
      None
    • Environment:
      Platform: PC, OS: All
    • Similar Issues:

      Description

      We use external SCP program to externalize credentials management from Maven
      configuration. Hudson Maven plug-in embeds an old version of Maven, which has
      problems with escaping of commands produced by the external SCP deployer. The
      issue is a combination of the following:

      0) scpexe-URLs get deployed with Wagon-ssh-external, which uses Plexus-utils to
      create command lines
      1) Maven modules of Hudson jobs are located in directories named
      groupId$artifactId on the filesystem
      2) Old versions of Plexus-utils command line utility does not escape $ in shell
      arguments

      1) and 2) together cause $artifactId to be environment-expanded when the SCP
      command line is executed. The replacement is usually empty string. The result is
      that the command line to tries to change to nonexistent directory and the
      deployment fails.

      Example stack trace below:

      Deploying artifacts to scpexe://server/dir/to/indagon-maven-snapshots
      Deploying the main artifact someartifact-1.1-SNAPSHOT.jar
      [INFO ] Retrieving previous build number from maven-snapshots
      [INFO ] repository metadata for: 'snapshot
      com.indagon:someartifact:1.1-SNAPSHOT' could not be found on repository:
      indagon-maven-snapshots, so will be created
      ERROR: Error deploying artifact: Exit code: 1 - /bin/sh: line 0: cd:
      /path/to/.hudson/jobs/someartifact-trunk/modules/com.indagon/builds/2009-11-15_03-52-35/archive/com.indagon/someartifact/1.1-SNAPSHOT:
      No such file or directory

      org.apache.maven.artifact.deployer.ArtifactDeploymentException: Error deploying
      artifact: Exit code: 1 - /bin/sh: line 0: cd:
      /path/to/.hudson/jobs/someartifact-trunk/modules/com.indagon/builds/2009-11-15_03-52-35/archive/com.indagon/someartifact/1.1-SNAPSHOT:
      No such file or directory

      at
      org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:94)
      at hudson.maven.reporters.MavenArtifactRecord.deploy(MavenArtifactRecord.java:119)
      at
      hudson.maven.reporters.MavenAggregatedArtifactRecord.deploy(MavenAggregatedArtifactRecord.java:79)
      at hudson.maven.RedeployPublisher.perform(RedeployPublisher.java:96)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:480)
      at
      hudson.model.AbstractBuild$AbstractRunner.performAllBuildStep(AbstractBuild.java:466)
      at hudson.maven.MavenModuleSetBuild$RunnerImpl.post2(MavenModuleSetBuild.java:590)
      at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:438)
      at hudson.model.Run.run(Run.java:1140)
      at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:300)
      at hudson.model.ResourceController.execute(ResourceController.java:93)
      at hudson.model.Executor.run(Executor.java:122)
      Caused by: org.apache.maven.wagon.TransferFailedException: Exit code: 1 -
      /bin/sh: line 0: cd:
      /path/to/.hudson/jobs/someartifact-trunk/modules/com.indagon/builds/2009-11-15_03-52-35/archive/com.indagon/someartifact/1.1-SNAPSHOT:
      No such file or directory

      at
      org.apache.maven.wagon.providers.ssh.external.ScpExternalWagon.executeScpCommand(ScpExternalWagon.java:236)
      at
      org.apache.maven.wagon.providers.ssh.external.ScpExternalWagon.put(ScpExternalWagon.java:295)
      at
      org.apache.maven.artifact.manager.DefaultWagonManager.putRemoteFile(DefaultWagonManager.java:244)
      at
      org.apache.maven.artifact.manager.DefaultWagonManager.putArtifact(DefaultWagonManager.java:160)
      at
      org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:80)
      ... 12 more
      Finished: FAILURE

      The correct path to 'cd' to in the example above would have been
      '/path/to/.hudson/jobs/someartifact-trunk/modules/com.indagon$someartifact/builds/2009-11-15_03-52-35/archive/com.indagon/someartifact/1.1-SNAPSHOT'

      I see two possible fixes to the problem:
      1) Change Hudson to not use $ in path names
      2) Change Hudson to use newer Maven (perhaps system Maven? why is Maven embedded
      in the plug-in anyway?)

      1) is rather silly, and would probably be backwards-incompatible (although
      reading of $-separated module names could still be supported), and would require
      finding some other separatorthat isn't used in any project, and doesn't get
      special treatment by shells
      2) I don't know about. On the surface it looks like the Right Thing to do, but
      probably there is a reason why a specific, old version is embedded in the
      plug-in in the first place.

        Attachments

          Issue Links

            Activity

            tazle tazle created issue -
            Hide
            tazle tazle added a comment -

            As a workaround I moved my upload credentials from ~/.ssh/config to
            ~/.m2/settings.xml of the user that runs Hudson on the build server, and changed
            the URL to scp:// form.

            Show
            tazle tazle added a comment - As a workaround I moved my upload credentials from ~/.ssh/config to ~/.m2/settings.xml of the user that runs Hudson on the build server, and changed the URL to scp:// form.
            evernat evernat made changes -
            Field Original Value New Value
            Link This issue is duplicated by JENKINS-7980 [ JENKINS-7980 ]
            Hide
            evernat evernat added a comment -

            Is it reproduced with recent Jenkins and plugins versions?

            Show
            evernat evernat added a comment - Is it reproduced with recent Jenkins and plugins versions?
            Hide
            tazle tazle added a comment -

            No idea. I don't have access to the system anymore.

            Show
            tazle tazle added a comment - No idea. I don't have access to the system anymore.
            Hide
            evernat evernat added a comment -

            So resolving as "can not reproduce".
            Anyone, please reopen if reproduced.

            Show
            evernat evernat added a comment - So resolving as "can not reproduce". Anyone, please reopen if reproduced.
            evernat evernat made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Cannot Reproduce [ 5 ]
            Hide
            rwegmann Raúl Wegmann added a comment -

            We can reproduce this error with Jenkins ver. 1.505 (using Maven 3). Please ask if you need further information.

            Show
            rwegmann Raúl Wegmann added a comment - We can reproduce this error with Jenkins ver. 1.505 (using Maven 3). Please ask if you need further information.
            rwegmann Raúl Wegmann made changes -
            Resolution Cannot Reproduce [ 5 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            Hide
            evernat evernat added a comment -

            @Raúl Wegmann
            It would probably be good to give the current stack-trace for your error, because the one above is certainly obsolete.

            Show
            evernat evernat added a comment - @Raúl Wegmann It would probably be good to give the current stack-trace for your error, because the one above is certainly obsolete.
            Hide
            rwegmann Raúl Wegmann added a comment -

            Here is the current stack trace:

            ERROR: Failed to deploy artifacts: Could not transfer artifact com.example.util:lib-util-core:jar:2.0.0.0.0-20130313.100035-11 from/to lib-deploy-snapshots (scpexe://maven.example.com/maven/snapshots/): Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT
            
            org.apache.maven.artifact.deployer.ArtifactDeploymentException: Failed to deploy artifacts: Could not transfer artifact com.example.util:lib-util-core:jar:2.0.0.0.0-20130313.100035-11 from/to lib-deploy-snapshots (scpexe://maven.example.com/maven/snapshots/): Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT
            
            	at org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:141)
            	at hudson.maven.reporters.MavenArtifactRecord.deploy(MavenArtifactRecord.java:190)
            	at hudson.maven.RedeployPublisher.perform(RedeployPublisher.java:176)
            	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
            	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:814)
            	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:786)
            	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:947)
            	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:733)
            	at hudson.model.Run.execute(Run.java:1592)
            	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477)
            	at hudson.model.ResourceController.execute(ResourceController.java:88)
            	at hudson.model.Executor.run(Executor.java:237)
            Caused by: org.sonatype.aether.deployment.DeploymentException: Failed to deploy artifacts: Could not transfer artifact com.example.util:lib-util-core:jar:2.0.0.0.0-20130313.100035-11 from/to lib-deploy-snapshots (scpexe://maven.example.com/maven/snapshots/): Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT
            
            	at org.sonatype.aether.impl.internal.DefaultDeployer.deploy(DefaultDeployer.java:280)
            	at org.sonatype.aether.impl.internal.DefaultDeployer.deploy(DefaultDeployer.java:211)
            	at org.sonatype.aether.impl.internal.DefaultRepositorySystem.deploy(DefaultRepositorySystem.java:443)
            	at org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:137)
            	... 11 more
            Caused by: org.sonatype.aether.transfer.ArtifactTransferException: Could not transfer artifact com.example.util:lib-util-core:jar:2.0.0.0.0-20130313.100035-11 from/to lib-deploy-snapshots (scpexe://maven.example.com/maven/snapshots/): Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT
            
            	at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:951)
            	at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:941)
            	at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$PutTask.run(WagonRepositoryConnector.java:837)
            	at org.sonatype.aether.connector.wagon.WagonRepositoryConnector.put(WagonRepositoryConnector.java:467)
            	at org.sonatype.aether.impl.internal.DefaultDeployer.deploy(DefaultDeployer.java:274)
            	... 14 more
            Caused by: org.apache.maven.wagon.TransferFailedException: Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT
            
            	at org.apache.maven.wagon.providers.ssh.external.ScpExternalWagon.executeScpCommand(ScpExternalWagon.java:321)
            	at org.apache.maven.wagon.providers.ssh.external.ScpExternalWagon.put(ScpExternalWagon.java:403)
            	at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$PutTask.run(WagonRepositoryConnector.java:811)
            	... 16 more
            

            It is trying to cd into /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/, while the correct directory is /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util$lib-util-core/. It seems the '$' character is not escaped correctly so that $lib is replaced with an empty string.

            Show
            rwegmann Raúl Wegmann added a comment - Here is the current stack trace: ERROR: Failed to deploy artifacts: Could not transfer artifact com.example.util:lib-util-core:jar:2.0.0.0.0-20130313.100035-11 from/to lib-deploy-snapshots (scpexe://maven.example.com/maven/snapshots/): Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT org.apache.maven.artifact.deployer.ArtifactDeploymentException: Failed to deploy artifacts: Could not transfer artifact com.example.util:lib-util-core:jar:2.0.0.0.0-20130313.100035-11 from/to lib-deploy-snapshots (scpexe://maven.example.com/maven/snapshots/): Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT at org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:141) at hudson.maven.reporters.MavenArtifactRecord.deploy(MavenArtifactRecord.java:190) at hudson.maven.RedeployPublisher.perform(RedeployPublisher.java:176) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:814) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:786) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:947) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:733) at hudson.model.Run.execute(Run.java:1592) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:237) Caused by: org.sonatype.aether.deployment.DeploymentException: Failed to deploy artifacts: Could not transfer artifact com.example.util:lib-util-core:jar:2.0.0.0.0-20130313.100035-11 from/to lib-deploy-snapshots (scpexe://maven.example.com/maven/snapshots/): Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT at org.sonatype.aether.impl.internal.DefaultDeployer.deploy(DefaultDeployer.java:280) at org.sonatype.aether.impl.internal.DefaultDeployer.deploy(DefaultDeployer.java:211) at org.sonatype.aether.impl.internal.DefaultRepositorySystem.deploy(DefaultRepositorySystem.java:443) at org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:137) ... 11 more Caused by: org.sonatype.aether.transfer.ArtifactTransferException: Could not transfer artifact com.example.util:lib-util-core:jar:2.0.0.0.0-20130313.100035-11 from/to lib-deploy-snapshots (scpexe://maven.example.com/maven/snapshots/): Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:951) at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:941) at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$PutTask.run(WagonRepositoryConnector.java:837) at org.sonatype.aether.connector.wagon.WagonRepositoryConnector.put(WagonRepositoryConnector.java:467) at org.sonatype.aether.impl.internal.DefaultDeployer.deploy(DefaultDeployer.java:274) ... 14 more Caused by: org.apache.maven.wagon.TransferFailedException: Exit code: 2 - /bin/sh: 1: cd: can't cd to /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/builds/2013-03-13_10-58-15/archive/com.example.util/lib-util-core/2.0.0.0.0-SNAPSHOT at org.apache.maven.wagon.providers.ssh.external.ScpExternalWagon.executeScpCommand(ScpExternalWagon.java:321) at org.apache.maven.wagon.providers.ssh.external.ScpExternalWagon.put(ScpExternalWagon.java:403) at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$PutTask.run(WagonRepositoryConnector.java:811) ... 16 more It is trying to cd into /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util-util-core/ , while the correct directory is /srv/jenkins/jobs/libj-util_trunk_ci/modules/com.example.util$lib-util-core/ . It seems the '$' character is not escaped correctly so that $lib is replaced with an empty string.
            Hide
            rwegmann Raúl Wegmann added a comment -

            We are seeing this problem again when using any Jenkins version greater than 1.559 so that we have been stuck for the last few months with that version.

            Apparently some CLI escaping issues have been fixed in the plexus-utils library since version 3.0.10, which is the one included in the maven-plugin, and I believe that this bug may have been fixed since 3.0.16 (https://jira.codehaus.org/browse/PLXUTILS-161).

            Please consider including the changes of the following pull request that fixes this error by bumping the plexus-utils version to the most recent one (3.0.17): https://github.com/jenkinsci/maven-plugin/pull/28

            Show
            rwegmann Raúl Wegmann added a comment - We are seeing this problem again when using any Jenkins version greater than 1.559 so that we have been stuck for the last few months with that version. Apparently some CLI escaping issues have been fixed in the plexus-utils library since version 3.0.10, which is the one included in the maven-plugin, and I believe that this bug may have been fixed since 3.0.16 ( https://jira.codehaus.org/browse/PLXUTILS-161 ). Please consider including the changes of the following pull request that fixes this error by bumping the plexus-utils version to the most recent one (3.0.17): https://github.com/jenkinsci/maven-plugin/pull/28
            rwegmann Raúl Wegmann made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Raúl Wegmann
            Path:
            pom.xml
            http://jenkins-ci.org/commit/maven-plugin/7f6145d8faead72a999d69f9bed36d601b193508
            Log:
            [FIXED JENKINS-4861] Use newer version of transitive plexus-utils dependency that allows to correctly deploy artifacts using scpexe.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Raúl Wegmann Path: pom.xml http://jenkins-ci.org/commit/maven-plugin/7f6145d8faead72a999d69f9bed36d601b193508 Log: [FIXED JENKINS-4861] Use newer version of transitive plexus-utils dependency that allows to correctly deploy artifacts using scpexe.
            scm_issue_link SCM/JIRA link daemon made changes -
            Status Reopened [ 4 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Olivier Lamy
            Path:
            pom.xml
            http://jenkins-ci.org/commit/maven-plugin/c074ac3cab43ba450ed69be6ba8610771a707396
            Log:
            Merge pull request #28 from rwegmann/JENKINS-4861

            JENKINS-4861: Fix post-build deployment of Maven artifacts with scpexe-type repository

            Compare: https://github.com/jenkinsci/maven-plugin/compare/4d1f9cc3014a...c074ac3cab43

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Olivier Lamy Path: pom.xml http://jenkins-ci.org/commit/maven-plugin/c074ac3cab43ba450ed69be6ba8610771a707396 Log: Merge pull request #28 from rwegmann/ JENKINS-4861 JENKINS-4861 : Fix post-build deployment of Maven artifacts with scpexe-type repository Compare: https://github.com/jenkinsci/maven-plugin/compare/4d1f9cc3014a...c074ac3cab43
            ircbot Jenkins IRC Bot made changes -
            Component/s maven-plugin [ 16033 ]
            Component/s maven2 [ 15487 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 134934 ] JNJira + In-Review [ 186904 ]

              People

              • Assignee:
                Unassigned
                Reporter:
                tazle tazle
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: