Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51913

Post-build action "Archive the artifacts" prints exception to console when no artifacts found

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      In my job configuration, I have Do not fail build if archiving returns nothing checked under Post-build Actions > Archive the artifacts so that build still completes even if no artifacts are created (this is intentional). The help icon describes this field as such:

      Normally, a build fails if archiving returns zero artifacts. This option allows the archiving process to return nothing without failing the build. Instead, the build will simply throw a warning.

      In Jenkins 2.107.3 this causes the following warning to be printed to the build console:

      15:30:26 Archiving artifacts
      15:30:27 WARN: No artifacts found that match the file pattern "**/data/behat-failure/**/*". Configuration error?
      15:30:28 WARN: java.lang.InterruptedException: no matches found within 10000
      

      However, in Jenkins 2.121.1 the following big ugly stack trace is now printed to the build console:

      12:28:36 Archiving artifacts
      12:28:36 java.lang.InterruptedException: no matches found within 10000
      12:28:36 	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
      12:28:36 	at hudson.FilePath$34.invoke(FilePath.java:2557)
      12:28:36 	at hudson.FilePath$34.invoke(FilePath.java:2547)
      12:28:36 	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
      12:28:36 Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to [redacted]
      12:28:36 		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
      12:28:36 		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
      12:28:36 		at hudson.remoting.Channel.call(Channel.java:955)
      12:28:36 		at hudson.FilePath.act(FilePath.java:1036)
      12:28:36 		at hudson.FilePath.act(FilePath.java:1025)
      12:28:36 		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
      12:28:36 		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
      12:28:36 		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
      12:28:36 		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
      12:28:36 		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
      12:28:36 		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
      12:28:36 		at hudson.model.Build$BuildExecution.post2(Build.java:186)
      12:28:36 		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
      12:28:36 		at hudson.model.Run.execute(Run.java:1819)
      12:28:36 		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
      12:28:36 		at hudson.model.ResourceController.execute(ResourceController.java:97)
      12:28:36 		at hudson.model.Executor.run(Executor.java:429)
      12:28:36 Caused: hudson.FilePath$TunneledInterruptedException
      12:28:36 	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
      12:28:36 	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
      12:28:36 	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
      12:28:36 	at hudson.remoting.Request$2.run(Request.java:369)
      12:28:36 	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
      12:28:36 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      12:28:36 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      12:28:36 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      12:28:36 	at java.lang.Thread.run(Thread.java:748)
      12:28:36 Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
      12:28:36 	at hudson.FilePath.act(FilePath.java:1038)
      12:28:36 	at hudson.FilePath.act(FilePath.java:1025)
      12:28:36 	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
      12:28:36 	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
      12:28:36 	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
      12:28:36 	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
      12:28:36 	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
      12:28:36 	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
      12:28:36 	at hudson.model.Build$BuildExecution.post2(Build.java:186)
      12:28:36 	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
      12:28:36 	at hudson.model.Run.execute(Run.java:1819)
      12:28:36 	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
      12:28:36 	at hudson.model.ResourceController.execute(ResourceController.java:97)
      12:28:36 	at hudson.model.Executor.run(Executor.java:429)
      12:28:36 No artifacts found that match the file pattern "**/data/behat-failure/**/*". Configuration error?
      

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            Without having dug into the details of whether validateAntFileMask preserves Ant’s own optimizations that allow fileset scans to be more efficient when the pattern is “rooted” in a literal directory prefix, I would suggest checking whether this helps:

            dir('tmp-jenkins-png') {
              archiveArtifacts artifacts: "**/*.png", allowEmptyArchive: true
            }
            

            which would guarantee that the validator is not peeking outside this directory.

            Of course if your copy command has selected only PNG files to begin with, you could just use artifacts: '**' as the pattern.

            Show
            jglick Jesse Glick added a comment - Without having dug into the details of whether validateAntFileMask preserves Ant’s own optimizations that allow fileset scans to be more efficient when the pattern is “rooted” in a literal directory prefix, I would suggest checking whether this helps: dir( 'tmp-jenkins-png' ) { archiveArtifacts artifacts: "**/*.png" , allowEmptyArchive: true } which would guarantee that the validator is not peeking outside this directory. Of course if your copy command has selected only PNG files to begin with, you could just use artifacts: '**' as the pattern.
            Hide
            vmassol Vincent Massol added a comment -

            Jesse Glick Thanks for the idea, I've implemented it. Let's see if it works better. Will report back here.

            Show
            vmassol Vincent Massol added a comment - Jesse Glick Thanks for the idea, I've implemented it. Let's see if it works better. Will report back here.
            Hide
            vmassol Vincent Massol added a comment -

            Jesse Glick So I've just realized that the "find" command is not working after all and I can't figure it out. It seems to be a potential bug of the "sh" step. Would be great if someone could confirm and whether I should open a jira issue for this (I'm still considering that it could be me making a mistake somewhere but I don't see it).

            This is what I do:

                    // Save videos generated by Docker-based tests, if any
                    // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
                    // see https://issues.jenkins-ci.org/browse/JENKINS-51913
                    echoXWiki "Looking for test videos in ${pwd()}"
                    sh "find . -path '*/target/*' -not \\( -path '*/tmp-jenkins-flv/*' -prune \\)\
                        -type f -name '*.flv' -exec rsync -R {} 'tmp-jenkins-flv/' \\;"
                    dir('tmp-jenkins-flv') {
                        archiveArtifacts artifacts: '**', allowEmptyArchive: true
                    }
            
                    // Save images generated by functional tests, if any
                    // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
                    // see https://issues.jenkins-ci.org/browse/JENKINS-51913
                    // Note: we look for screenshots only in the screenshots directory to avoid false positives such as PNG images
                    // that would be located in a XWiki distribution located in target/.
                    echoXWiki "Looking for test failure images in ${pwd()}"
                    sh "find . -path '*/target/*/screenshots' -not \\( -path '*/tmp-jenkins-png/*' -prune \\)\
                        -type f -name '*.png' -exec rsync -R {} 'tmp-jenkins-png/' \\;"
                    dir('tmp-jenkins-png') {
                        archiveArtifacts artifacts: '**', allowEmptyArchive: true
                    }
            

            And when it executes on our agent in the log it says either one of these:

            When executed using our Jenkins Docker Agent image:

            [Pipeline] sh
            + find . -path '*/target/*/screenshots' -not '(' -path '*/tmp-jenkins-png/*' -prune ')' -type f -name '*.png' -exec rsync -R '{}' tmp-jenkins-png/ ';'
            

            When executed using the "standard" non-docker SSH agent:

            [Pipeline] sh
            + find . -path */target/*/screenshots -not ( -path */tmp-jenkins-png/* -prune ) -type f -name *.png -exec rsync -R {} tmp-jenkins-png/ ;
            

            Both are invalid and I can't figure out why.

            Side note: When going through this intermediary directory"tmp-jenkins-*", the directories are not cleaned between each job execution and thus the artifacts accumulate at each run... So if you use this strategy, you need to take care of cleaning the directory (which is complex in our setup).

            FTM I've had to revert to the previous direct call of archiveArtifacts:

                    // Save videos generated by Docker-based tests, if any
                    // This can generate some not nice stack trace in the logs,
                    // see https://issues.jenkins-ci.org/browse/JENKINS-51913
                    echoXWiki "Looking for test failure videos in ${pwd()}"
                    archiveArtifacts artifacts: '*/target/*', allowEmptyArchive: true
            
                    // Save images generated by functional tests, if any
                    // This can generate some not nice stack trace in the logs,
                    // see https://issues.jenkins-ci.org/browse/JENKINS-51913
                    // Note: we look for screenshots only in the screenshots directory to avoid false positives such as PNG images
                    // that would be located in a XWiki distribution located in target/.
                    echoXWiki "Looking for test failure images in ${pwd()}"
                    archiveArtifacts artifacts: '*/target/*/screenshots', allowEmptyArchive: true
            

            https://github.com/xwiki/xwiki-jenkins-pipeline/commit/411cf4d1a2b7f206c11d01b2914fccaf403be41d

            Thanks

            Show
            vmassol Vincent Massol added a comment - Jesse Glick So I've just realized that the "find" command is not working after all and I can't figure it out. It seems to be a potential bug of the "sh" step. Would be great if someone could confirm and whether I should open a jira issue for this (I'm still considering that it could be me making a mistake somewhere but I don't see it). This is what I do: // Save videos generated by Docker-based tests, if any // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts, // see https://issues.jenkins-ci.org/browse/JENKINS-51913 echoXWiki "Looking for test videos in ${pwd()}" sh "find . -path '*/target /*' -not \\( -path '*/ tmp-jenkins-flv/*' -prune \\)\ -type f -name '*.flv' -exec rsync -R {} 'tmp-jenkins-flv/' \\;" dir( 'tmp-jenkins-flv' ) { archiveArtifacts artifacts: '**' , allowEmptyArchive: true } // Save images generated by functional tests, if any // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts, // see https://issues.jenkins-ci.org/browse/JENKINS-51913 // Note: we look for screenshots only in the screenshots directory to avoid false positives such as PNG images // that would be located in a XWiki distribution located in target/. echoXWiki "Looking for test failure images in ${pwd()}" sh "find . -path '*/target /*/screenshots' -not \\( -path '*/ tmp-jenkins-png/*' -prune \\)\ -type f -name '*.png' -exec rsync -R {} 'tmp-jenkins-png/' \\;" dir( 'tmp-jenkins-png' ) { archiveArtifacts artifacts: '**' , allowEmptyArchive: true } And when it executes on our agent in the log it says either one of these: When executed using our Jenkins Docker Agent image: [Pipeline] sh + find . -path '*/target/*/screenshots' -not '(' -path '*/tmp-jenkins-png/*' -prune ')' -type f -name '*.png' -exec rsync -R '{}' tmp-jenkins-png/ ';' When executed using the "standard" non-docker SSH agent: [Pipeline] sh + find . -path */target/*/screenshots -not ( -path */tmp-jenkins-png/* -prune ) -type f -name *.png -exec rsync -R {} tmp-jenkins-png/ ; Both are invalid and I can't figure out why. Side note: When going through this intermediary directory"tmp-jenkins-*", the directories are not cleaned between each job execution and thus the artifacts accumulate at each run... So if you use this strategy, you need to take care of cleaning the directory (which is complex in our setup). FTM I've had to revert to the previous direct call of archiveArtifacts: // Save videos generated by Docker-based tests, if any // This can generate some not nice stack trace in the logs, // see https://issues.jenkins-ci.org/browse/JENKINS-51913 echoXWiki "Looking for test failure videos in ${pwd()}" archiveArtifacts artifacts: '*/target/*' , allowEmptyArchive: true // Save images generated by functional tests, if any // This can generate some not nice stack trace in the logs, // see https://issues.jenkins-ci.org/browse/JENKINS-51913 // Note: we look for screenshots only in the screenshots directory to avoid false positives such as PNG images // that would be located in a XWiki distribution located in target/. echoXWiki "Looking for test failure images in ${pwd()}" archiveArtifacts artifacts: '*/target/*/screenshots' , allowEmptyArchive: true https://github.com/xwiki/xwiki-jenkins-pipeline/commit/411cf4d1a2b7f206c11d01b2914fccaf403be41d Thanks
            Hide
            jglick Jesse Glick added a comment -

            I suspect that the two executions are differing in shell: bash vs. traditional sh / dash. At any rate, it is hard to follow your string escapes there. I strongly recommend using single quotes for sh steps, not only to improve legibility but to ensure you do not accidentally use GString interpolation (which is rarely what you want—see inline help for withCredentials for the gory details). If the command itself has single quotes, use a Groovy multiline string:

            sh '''
                find . -path '*/target/*/screenshots' …
            '''
            

            which will be more readable.

            Or better yet, just move all this stuff into your regular build script so it produces the desired artifacts in a tidy directory ready for Jenkins to publish, which is something you can easily test locally. There is no reason to clutter Jenkinsfile with complex embedded scripts.

            Not sure why cleaning a directory would be complex in any setup. rm -rf tmp-jenkins-png && mkdir tmp-jenkins-png does not suffice?

            Show
            jglick Jesse Glick added a comment - I suspect that the two executions are differing in shell: bash vs. traditional sh / dash . At any rate, it is hard to follow your string escapes there. I strongly recommend using single quotes for sh steps, not only to improve legibility but to ensure you do not accidentally use GString interpolation (which is rarely what you want—see inline help for withCredentials for the gory details). If the command itself has single quotes, use a Groovy multiline string: sh ''' find . -path '*/target/*/screenshots' … ''' which will be more readable. Or better yet, just move all this stuff into your regular build script so it produces the desired artifacts in a tidy directory ready for Jenkins to publish, which is something you can easily test locally. There is no reason to clutter Jenkinsfile with complex embedded scripts. Not sure why cleaning a directory would be complex in any setup. rm -rf tmp-jenkins-png && mkdir tmp-jenkins-png does not suffice?
            Hide
            vmassol Vincent Massol added a comment -

            Jesse Glick I agree with you about about single quote vs double quotes. I usually use single quotes to the max and this is what I was doing but I need GString interpolation before when I had written it as https://github.com/xwiki/xwiki-jenkins-pipeline/blob/f8c4f536dcbb0524f9348c989003857a540bc70d/vars/xwikiBuild.groovy#L190

            Then I forgot to replace the double quotes by single quotes after I refactored it...

            That said using single quotes shouldn't change anything to the problem. There's no GString interpolation in my string AFAICS (no dollar sign). Thus the problem will remain IMO.

            Or better yet, just move all this stuff into your regular build script so it produces the desired artifacts in a tidy directory ready for Jenkins to publish, which is something you can easily test locally. There is no reason to clutter Jenkinsfile with complex embedded scripts.

            This is what our regular Maven build script does. It generates these results in the `target/` directory (i.e. in a clean location). The issue here is the maven reactor and multimodules. Our build script is fine IMO and gathering of artifacts is not a build responsibility. All we're asking Jenkins is to archive some of the artifacts in `target/` because we're using Docker Cloud and the docker agent container is removed after each run and thus we need to save some of the artifacts in the job results.

            BTW all this is done in our pipeline library and not inside the Jenkinsfile which remains clean and generic.

            Not sure why cleaning a directory would be complex in any setup. rm -rf tmp-jenkins-png && mkdir tmp-jenkins-png does not suffice?

            Yes, I think I was wrong here. We should be able to do this just before we do the checkout in our pipeline library.

            Side note: I thought that the `clearWorkspace` parameter in the `checkout` step (https://jenkins.io/doc/pipeline/steps/workflow-scm-step/) would clean the workspace but apparently it does not (it probably does something else). I tried to find the source code that uses this parameter to see what it does but couldn't track it (I ended up in SCM.java but I see no use of it there either). I checked it by creating a file in the workspace and then calling the pipeline and the file remained.

            Thanks

            Show
            vmassol Vincent Massol added a comment - Jesse Glick I agree with you about about single quote vs double quotes. I usually use single quotes to the max and this is what I was doing but I need GString interpolation before when I had written it as https://github.com/xwiki/xwiki-jenkins-pipeline/blob/f8c4f536dcbb0524f9348c989003857a540bc70d/vars/xwikiBuild.groovy#L190 Then I forgot to replace the double quotes by single quotes after I refactored it... That said using single quotes shouldn't change anything to the problem. There's no GString interpolation in my string AFAICS (no dollar sign). Thus the problem will remain IMO. Or better yet, just move all this stuff into your regular build script so it produces the desired artifacts in a tidy directory ready for Jenkins to publish, which is something you can easily test locally. There is no reason to clutter Jenkinsfile with complex embedded scripts. This is what our regular Maven build script does. It generates these results in the `target/` directory (i.e. in a clean location). The issue here is the maven reactor and multimodules. Our build script is fine IMO and gathering of artifacts is not a build responsibility. All we're asking Jenkins is to archive some of the artifacts in `target/` because we're using Docker Cloud and the docker agent container is removed after each run and thus we need to save some of the artifacts in the job results. BTW all this is done in our pipeline library and not inside the Jenkinsfile which remains clean and generic. Not sure why cleaning a directory would be complex in any setup. rm -rf tmp-jenkins-png && mkdir tmp-jenkins-png does not suffice? Yes, I think I was wrong here. We should be able to do this just before we do the checkout in our pipeline library. Side note: I thought that the `clearWorkspace` parameter in the `checkout` step ( https://jenkins.io/doc/pipeline/steps/workflow-scm-step/ ) would clean the workspace but apparently it does not (it probably does something else). I tried to find the source code that uses this parameter to see what it does but couldn't track it (I ended up in SCM.java but I see no use of it there either). I checked it by creating a file in the workspace and then calling the pipeline and the file remained. Thanks

              People

              • Assignee:
                Unassigned
                Reporter:
                pholden Paul Holden
              • Votes:
                4 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: