Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48524

Misc infrastructure errors are missing from step view. Blue ocean should default to showing a log tail at end of failed build.

    Details

    • Similar Issues:
    • Sprint:
      Blue Ocean 1.4 - beta 3, Blue Ocean - Candidates, Blue Ocean 1.6 - beta 2

      Description

      In many cases there is a failure in a pipeline run which does not show up in a helpful way in the pipeline results screen. 

      this can manifest as some failed stages that are missing the key information. 

      eg : https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fpipeline-build-step-plugin/detail/PR-18/3/pipeline/9

      Shows just failed stages and steps (see attached), but is missing the key error: 

      Also:   hudson.AbortException: Queue task was cancelled
      		at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$CancelledItemListener.onLeft(ExecutorStepExecution.java:213)
      		at hudson.model.Queue$LeftItem.enter(Queue.java:2723)
      		at hudson.model.Queue$Item.cancel(Queue.java:2311)
      		at hudson.model.Queue.cancel(Queue.java:745)
      		at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution.stop(ExecutorStepExecution.java:130)
      		at org.jenkinsci.plugins.workflow.cps.CpsThread.stop(CpsThread.java:296)
      		at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$2.onSuccess(CpsBodyExecution.java:260)
      		at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$2.onSuccess(CpsBodyExecution.java:247)
      		at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:746)
      		at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
      		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      		at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
      		at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)

      which is in the full log. 

       

      Another case is attached where the pipeline shows just one failed stage, but no failed steps at all. 

      In this case there was a stack trace in the log file (also attached) which showed the real error. 

       

      What should happen:

      Given the myriad ways in which a pipeline run can fail, often due to infra or configuration, it seems too much "whack a mole" to try and shoehorn all errors into steps/stages. 

      perhaps a better approach is to identity failure cases and err on the side of showing the fill log "tail" on screen, perhaps as the final step visible. This would make all "misc" failures clear. 

      The pipeline graph is useful in these cases, just not the logs shown in steps. 

       

      raised by James Nord

       

      This may make the following irrelevant: 

      https://issues.jenkins-ci.org/browse/JENKINS-46715

      https://issues.jenkins-ci.org/browse/JENKINS-45380

       

       

       

        Attachments

          Activity

          Hide
          michaelneale Michael Neale added a comment -

          Vivek Pandey request for comment: do you think this is possible/doable? (ie to fall back to grabbing a tail of the log in some failure cases to make the result screen better for these infra failures?)

          Show
          michaelneale Michael Neale added a comment - Vivek Pandey request for comment: do you think this is possible/doable? (ie to fall back to grabbing a tail of the log in some failure cases to make the result screen better for these infra failures?)
          Hide
          teilo James Nord added a comment - - edited

          I mentioned to Michael Neale privatley my concern about just tailing the log so will add it here.

          There is no way to know how log the stack trace is in the log. it may be severel exceptions deep (nested) or have suppressions.

          In these cases the upper pipeline has caugth the error and logged it - to me it should also record this exception somewhere and allow retreival for it so that BO can check if there was an overall exception and if so display it without having to just tail some logs in the off chance.

          //cc Andrew Bayer / Jesse Glick - not sure if it is possible to change pipeline to store this (or if it is already stored somewhere and we just don;t know about it)

          Show
          teilo James Nord added a comment - - edited I mentioned to Michael Neale privatley my concern about just tailing the log so will add it here. There is no way to know how log the stack trace is in the log. it may be severel exceptions deep (nested) or have suppressions. In these cases the upper pipeline has caugth the error and logged it - to me it should also record this exception somewhere and allow retreival for it so that BO can check if there was an overall exception and if so display it without having to just tail some logs in the off chance. //cc Andrew Bayer / Jesse Glick - not sure if it is possible to change pipeline to store this (or if it is already stored somewhere and we just don;t know about it)
          Hide
          kshultz Karl Shultz added a comment -

          Testing Notes:
          Once implemented, unit tests should be included as part of the PR.

          Show
          kshultz Karl Shultz added a comment - Testing Notes: Once implemented, unit tests should be included as part of the PR.
          Hide
          vivek Vivek Pandey added a comment -

          Michael Neale Tailing Jenkins log might not show the right error as it might be buried deep in the main jenkins log. Maybe if we have a way to associated failed FlowNode (stage/step/parallel-branch) with other system wide exception, such as failed stage and then associated exception stack trace in that stage/step log.

          Sam Van Oort is your log optimization touches on these areas or any suggestion on what blueocean can use from pipeline infra? 

          Show
          vivek Vivek Pandey added a comment - Michael Neale  Tailing Jenkins log might not show the right error as it might be buried deep in the main jenkins log. Maybe if we have a way to associated failed FlowNode (stage/step/parallel-branch) with other system wide exception, such as failed stage and then associated exception stack trace in that stage/step log. Sam Van Oort is your log optimization touches on these areas or any suggestion on what blueocean can use from pipeline infra? 
          Hide
          vivek Vivek Pandey added a comment -

          Michael Neale https://issues.jenkins-ci.org/browse/JENKINS-48074 fixes cases where pipeline script failed to start due to error (that is no step or stages got to run). Cases such as handling intermediate step failed due to infra error are not handled by JENKINS-48074. I think we should keep it open.

          Right fix should really come from upstream pipeline library where pipeline step contextual errors are made available thru FlowNode or executor so that in case of failure we can display it in context of failed step. cc: Sam Van Oort.

          Show
          vivek Vivek Pandey added a comment - Michael Neale https://issues.jenkins-ci.org/browse/JENKINS-48074 fixes cases where pipeline script failed to start due to error (that is no step or stages got to run). Cases such as handling intermediate step failed due to infra error are not handled by JENKINS-48074 . I think we should keep it open. Right fix should really come from upstream pipeline library where pipeline step contextual errors are made available thru FlowNode or executor so that in case of failure we can display it in context of failed step. cc: Sam Van Oort .
          Hide
          pete312 Pete Moore added a comment -

          This seems to be pretty long standing request. Is it not just possible for now to have a link to full log saying failure detected see full log rather than saying nothing? 

          Show
          pete312 Pete Moore added a comment - This seems to be pretty long standing request. Is it not just possible for now to have a link to full log saying failure detected see full log rather than saying nothing? 

            People

            • Assignee:
              Unassigned
              Reporter:
              michaelneale Michael Neale
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: