Details

    • Similar Issues:

      Description

      When a flow involves many steps, long logs, and/or many branches, it can be hard for a developer receiving a failure email (for example) to quickly see what part of the build actually failed and why. I think it would be useful for the catchError step to do a DFS search on the Throwable and its cause chain through the FlowNode graph starting at the end of the catch step, looking for ErrorAction, and setting/appending an environment variable with the URL of the LogAction. Or provide a new step to do the same. Thus you could write

      def runStuff(param) {
        ...
      }
      try {
        parallel a: {runStuff 'a'}, b: {runStuff 'b'}
      } catch (e) {
        mail to: '...', subject: 'Failure!', body: "Build failed: ${errorUrl(e)}"
      }
      

      and get a link to http://jenkins/job/myflow/123/flowGraph/77/console or the like, according to the actual step in one of the branches that threw the error.

        Attachments

          Issue Links

            Activity

            jglick Jesse Glick created issue -
            Hide
            jglick Jesse Glick added a comment -

            Should also be able to get a tail of the log.

            Show
            jglick Jesse Glick added a comment - Should also be able to get a tail of the log.
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-26107 [ JENKINS-26107 ]
            Hide
            michaelneale Michael Neale added a comment -

            In a freestyle job the config is still simpler than the proposed above, having a global "catch all" that sends failure (with pertinent details) would be nice to bring it up to parity.

            Show
            michaelneale Michael Neale added a comment - In a freestyle job the config is still simpler than the proposed above, having a global "catch all" that sends failure (with pertinent details) would be nice to bring it up to parity.
            Hide
            michaelneale Michael Neale added a comment -

            Given a common use for this will be to slap a try/catch around everything to get failure messages, would it be possible to have a:

            onFailure DSL that effectively does the same thing? (not sure if this is possible in the current form of things). Ideally this could be scoped to stage, but that may be just too much to ask. The ideal is to make it not too much more work than what people experience now with freestyle, in WF terms.

            Show
            michaelneale Michael Neale added a comment - Given a common use for this will be to slap a try/catch around everything to get failure messages, would it be possible to have a: onFailure DSL that effectively does the same thing? (not sure if this is possible in the current form of things). Ideally this could be scoped to stage, but that may be just too much to ask. The ideal is to make it not too much more work than what people experience now with freestyle, in WF terms.
            Hide
            maxfields2000 Maxfield Stewart added a comment -

            This is going to be a primary blocker to my users not adopting pipeline DSL over say, Job DSL (perhaps with some pipeline in it). It's way too cumbersome to require all that raw Groovy code just to send an email on failure.

            The mail post-build step wrapper doesn't actually execute right now because the pipeline DSL aborts the run before it finishes.

            I can of course, create a global wrapper for "node" that embeds a default mail handler and exception trap, but inside that global method I lose access to global variables, like JOB_NAME and other things that make sending the mail more intuitive.

            Show
            maxfields2000 Maxfield Stewart added a comment - This is going to be a primary blocker to my users not adopting pipeline DSL over say, Job DSL (perhaps with some pipeline in it). It's way too cumbersome to require all that raw Groovy code just to send an email on failure. The mail post-build step wrapper doesn't actually execute right now because the pipeline DSL aborts the run before it finishes. I can of course, create a global wrapper for "node" that embeds a default mail handler and exception trap, but inside that global method I lose access to global variables, like JOB_NAME and other things that make sending the mail more intuitive.
            Hide
            michaelneale Michael Neale added a comment -
            Show
            michaelneale Michael Neale added a comment - Maxfield Stewart yes agree. I made https://wiki.jenkins-ci.org/display/JENKINS/Simple+Build+For+Pipeline+Plugin to show how this could be made simpler.
            Hide
            pmarcoen Peter Marcoen added a comment -

            I agree with Michael and Maxfield. It feels like I'm taking a step back when using Pipeline. It used to be as simple as providing the emailaddress to send a mail to when the build failed. Now I have to start coding it in with try-catches.

            I too would prefer a onFailure DSL to avoid all that complexity.

            Show
            pmarcoen Peter Marcoen added a comment - I agree with Michael and Maxfield. It feels like I'm taking a step back when using Pipeline. It used to be as simple as providing the emailaddress to send a mail to when the build failed. Now I have to start coding it in with try-catches. I too would prefer a onFailure DSL to avoid all that complexity.
            Hide
            deepchip Martin d'Anjou added a comment -

            It might reduce the volume of code if you push everything down, and try-catch only once at the top:

            main()
            def main() {
                try {
                    core()
                } catch (Exception e) {
                    mail to: 'dest@domain', subject: "Failure of Jenkins", body: e.getMessage()+"\nTry harder the next time."
                    error(e.getMessage())
                }
            }
            
            def core() {
                node {
                    sh "exit 1"
                }
            }
            

            Hope this helps.

            Show
            deepchip Martin d'Anjou added a comment - It might reduce the volume of code if you push everything down, and try-catch only once at the top: main() def main() { try { core() } catch (Exception e) { mail to: 'dest@domain' , subject: "Failure of Jenkins" , body: e.getMessage()+ "\nTry harder the next time." error(e.getMessage()) } } def core() { node { sh "exit 1" } } Hope this helps.
            Hide
            jglick Jesse Glick added a comment -

            inside that global method I lose access to global variables, like JOB_NAME

            Not true, these are still accessible.

            Show
            jglick Jesse Glick added a comment - inside that global method I lose access to global variables, like JOB_NAME Not true, these are still accessible.
            Hide
            jglick Jesse Glick added a comment -

            Should rather return a Serializable struct with @Whitelisted properties that can be used to customize error handling:

            • original exception (e.g., for calling .message)
            • full stack trace of exception (as a convenience)
            • URL of LogAction
            • tail of log file leading up to error
            • label associated with failing step (using FlowNodeSerialWalker from JENKINS-26132, and especially useful in conjunction with JENKINS-26107)
            Show
            jglick Jesse Glick added a comment - Should rather return a Serializable struct with @Whitelisted properties that can be used to customize error handling: original exception (e.g., for calling .message ) full stack trace of exception (as a convenience) URL of LogAction tail of log file leading up to error label associated with failing step (using FlowNodeSerialWalker from JENKINS-26132 , and especially useful in conjunction with JENKINS-26107 )
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            jglick Jesse Glick added a comment -

            As per JENKINS-32059, we lack API-level metadata about source line location.

            Show
            jglick Jesse Glick added a comment - As per JENKINS-32059 , we lack API-level metadata about source line location.
            jglick Jesse Glick made changes -
            Link This issue depends on JENKINS-32059 [ JENKINS-32059 ]
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-25894 [ JENKINS-25894 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "workflow-basic-steps PR 2 (Web Link)" [ 14200 ]
            jglick Jesse Glick made changes -
            Epic Link JENKINS-35400 [ 171193 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 162832 ] JNJira + In-Review [ 185594 ]
            Hide
            klrmn Leah Klearman added a comment -

            I'm interested in ensuring that the solution works with a parallel() that has a large number of sub-jobs...that all of the jobs get completed and multiple failures can be identified.

            Show
            klrmn Leah Klearman added a comment - I'm interested in ensuring that the solution works with a parallel() that has a large number of sub-jobs...that all of the jobs get completed and multiple failures can be identified.
            abayer Andrew Bayer made changes -
            Component/s pipeline-general [ 21692 ]
            abayer Andrew Bayer made changes -
            Component/s workflow-plugin [ 18820 ]
            jglick Jesse Glick made changes -
            Component/s workflow-basic-steps-plugin [ 21712 ]
            Component/s pipeline [ 21692 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-38156 [ JENKINS-38156 ]
            Hide
            jamesdumay James Dumay added a comment -

            Jesse Glick is this this still relevant with the advent of Blue Ocean?

            Show
            jamesdumay James Dumay added a comment - Jesse Glick is this this still relevant with the advent of Blue Ocean?
            Hide
            dubrsl Viacheslav Dubrovskyi added a comment - - edited

            My workaround for this.

            // Add specific string used for detect id of log. Should be uniq for every step.
            sh 'echo "Run SPECIFIC_TOKEN"'
            
            // At the end detect log id and generate URL to step
            exec = """ set +x; echo "${BUILD_URL}execution/node/\$(grep -l "Run SPECIFIC_TOKEN" "$JENKINS_HOME/jobs/\$(echo "$JOB_NAME" | sed 's|/|/branches/|g')/builds/$BUILD_ID/"*.log 2>/dev/null | head -n1 | xargs basename | sed 's/\\.log//')/log/" """
            stepURL=sh ( script: exec, returnStdout: true).trim()
            

            It's ugly, but works in my case.

            Show
            dubrsl Viacheslav Dubrovskyi added a comment - - edited My workaround for this. // Add specific string used for detect id of log. Should be uniq for every step. sh 'echo "Run SPECIFIC_TOKEN" ' // At the end detect log id and generate URL to step exec = """ set +x; echo " ${BUILD_URL}execution/node/\$(grep -l "Run SPECIFIC_TOKEN" "$JENKINS_HOME/jobs/\$(echo " $JOB_NAME " | sed 's|/|/branches/|g' )/builds/$BUILD_ID/" *.log 2>/dev/ null | head -n1 | xargs basename | sed 's/\\.log //' )/log/ " " "" stepURL=sh ( script: exec, returnStdout: true ).trim() It's ugly, but works in my case.
            Hide
            ssbarnea Sorin Sbarnea added a comment -

            Is ugly and would break in too many possible ways. We need a generic solution that we can use even when Blue Ocean is not enabled. 

            We need a link that points to the first error of the current build. This looks like something where we need to use HTML bookmarks (url#bookmark) to achieve it in a reliable way. At this this approach should downgrade gracefully.  

            Show
            ssbarnea Sorin Sbarnea added a comment - Is ugly and would break in too many possible ways. We need a generic solution that we can use even when Blue Ocean is not enabled.  We need a link that points to the first error of the current build. This looks like something where we need to use HTML bookmarks (url#bookmark) to achieve it in a reliable way. At this this approach should downgrade gracefully.  
            Hide
            jglick Jesse Glick added a comment -

            James Dumay yes. Could be integrated with display-url-provider but otherwise unaffected.

            Show
            jglick Jesse Glick added a comment - James Dumay yes. Could be integrated with display-url-provider but otherwise unaffected.
            jamesdumay James Dumay made changes -
            Remote Link This issue links to "CloudBees Internal UX-616 (Web Link)" [ 18195 ]
            Hide
            jpruud Juan Pablo Bottinelli added a comment -

            Hi. Is this still being worked on?

            Thanks

            Show
            jpruud Juan Pablo Bottinelli added a comment - Hi. Is this still being worked on? Thanks
            Hide
            roel0 roel postelmans added a comment -

            You can use the rest api that comes with the stage view plugin:

            https://github.com/jenkinsci/pipeline-stage-view-plugin/tree/master/rest-api

             

            to obtain a stage-seperated log

            Show
            roel0 roel postelmans added a comment - You can use the rest api that comes with the stage view plugin: https://github.com/jenkinsci/pipeline-stage-view-plugin/tree/master/rest-api   to obtain a stage-seperated log
            Hide
            jpruud Juan Pablo Bottinelli added a comment -

            I'll take a look at that. Thanks roel postelmans

            Show
            jpruud Juan Pablo Bottinelli added a comment - I'll take a look at that. Thanks roel postelmans
            Hide
            heckebbg Hans Ecke added a comment -

            FWIW, I'm looking at the exact same usecase. Currently I email currentBuild.rawBuild.getLog(1000) to the stakeholders of the run, but it contains much unrelated chaff.

            Show
            heckebbg Hans Ecke added a comment - FWIW, I'm looking at the exact same usecase. Currently I email  currentBuild.rawBuild.getLog(1000) to the stakeholders of the run, but it contains much unrelated chaff.
            Hide
            ringerc Craig Ringer added a comment -

            This becomes even more relevant when working with Blue Ocean etc. The URLs are something like `jobname/6/pipeline/79` where the /79 is a node in the graph of steps and parallel tasks. It doesn't seem to be predictable, so it's not clear how you can generate a URL that points to exactly one parallel step, for example.

            Show
            ringerc Craig Ringer added a comment - This becomes even more relevant when working with Blue Ocean etc. The URLs are something like `jobname/6/pipeline/79` where the /79 is a node in the graph of steps and parallel tasks. It doesn't seem to be predictable, so it's not clear how you can generate a URL that points to exactly one parallel step, for example.
            Hide
            hardi249 Michael Hartmann added a comment -

            Is this still in progress?

            Show
            hardi249 Michael Hartmann added a comment - Is this still in progress?
            Hide
            jglick Jesse Glick added a comment -

            Not really. I suppose I could revisit this, given the high vote count.

            Show
            jglick Jesse Glick added a comment - Not really. I suppose I could revisit this, given the high vote count.
            jglick Jesse Glick made changes -
            Labels stalled-pr
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-43995 [ JENKINS-43995 ]
            Hide
            monger39 David Riemens added a comment -

            just up-voted;
            Me too, I have similar usecase. In our testenvironment, we run many tools, each generating a separate logfile. We have one generic log shown in the console.
            If one test fails, we can see that it failed, and typically have an exception message shown in the console, but the actual (detailed) reason is hidden in one of
            the  detailed logs. For that reason we want to include a link to the workspace directory on the exception text, such that all the log-files are just one click away.
            For the pipeline jobs that link is not easy to derive; it is hidden in one of the pipeline steps. as also indicated by Craig Ringer
            My feeling says that making sure that link, or the ID is available at runtime should be possible ... right ?

            Show
            monger39 David Riemens added a comment - just up-voted; Me too, I have similar usecase. In our testenvironment, we run many tools, each generating a separate logfile. We have one generic log shown in the console. If one test fails, we can see that it failed, and typically have an exception message shown in the console, but the actual (detailed) reason is hidden in one of the  detailed logs. For that reason we want to include a link to the workspace directory on the exception text, such that all the log-files are just one click away. For the pipeline jobs that link is not easy to derive; it is hidden in one of the pipeline steps. as also indicated by Craig Ringer My feeling says that making sure that link, or the ID is available at runtime should be possible ... right ?
            Hide
            heyleke Jan Heylen added a comment - - edited

            Is there already any way to get the link to 'blue/rest/organizations/jenkins/pipelines/<jobname>/runs/<build-id>/nodes/<PipelineNodeImpl-id>/log/ from within a pipeline parallel node?

            Show
            heyleke Jan Heylen added a comment - - edited Is there already any way to get the link to 'blue/rest/organizations/jenkins/pipelines/<jobname>/runs/<build-id>/nodes/<PipelineNodeImpl-id>/log/ from within a pipeline parallel node?

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                jglick Jesse Glick
              • Votes:
                42 Vote for this issue
                Watchers:
                57 Start watching this issue

                Dates

                • Created:
                  Updated: