Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42863

Docker pipeline steps reported as successful even when failing

    Details

    • Similar Issues:
    • Sprint:
      Blue Ocean 1.0-rc3

      Description

      Problem
      On this jenkins file it sometimes fails to do a docker pull. This results in a step that fails from time to time. This ends up as an exception that is caught and prints out the error here. The problem is the actual failing step is reported as successful, which isn't helpful when highlighting where the problem is. 

      What should happen
      The step that threw the exception should be marked as a failure.

      Screenshot

       

       

       

       

       

       

        Attachments

          Activity

          Hide
          michaelneale Michael Neale added a comment -

          right, this is a docker pipeline thing in that case - (unless it also is using try/catch). 

          Show
          michaelneale Michael Neale added a comment - right, this is a docker pipeline thing in that case - (unless it also is using try/catch). 
          Hide
          jglick Jesse Glick added a comment -

          Afraid I do not follow the discussion here or why someone thinks this is an issue in docker-workflow. Please provide steps to reproduce from scratch without using the stage view or Blue Ocean.

          Show
          jglick Jesse Glick added a comment - Afraid I do not follow the discussion here or why someone thinks this is an issue in docker-workflow . Please provide steps to reproduce from scratch without using the stage view or Blue Ocean.
          Hide
          rtyler R. Tyler Croy added a comment -

          Jesse Glick, my comment has a simple Jenkinsfile which reproduces this issue.

          Somewhere inside the Docker Pipeline interactions a failure is occurring and a step is not being marked as a failure although the Pipeline itself is. I believe a simple reproduction case would be to cause docker inspect to fail, such as when dockerd is not running.

          The reason this relates to Blue Ocean is that visualization and identifying the failing step in a Pipeline is a key feature of Blue Ocean, and the underlying issue here prevents that feature from working correctly.

          Also, I don't think it's polite to simply close tickets if you're not understanding the issue. Re-assigning to the reporter and asking for more information is preferrable IMHO.

          Show
          rtyler R. Tyler Croy added a comment - Jesse Glick , my comment has a simple Jenkinsfile which reproduces this issue. Somewhere inside the Docker Pipeline interactions a failure is occurring and a step is not being marked as a failure although the Pipeline itself is. I believe a simple reproduction case would be to cause docker inspect to fail, such as when dockerd is not running. The reason this relates to Blue Ocean is that visualization and identifying the failing step in a Pipeline is a key feature of Blue Ocean, and the underlying issue here prevents that feature from working correctly. Also, I don't think it's polite to simply close tickets if you're not understanding the issue. Re-assigning to the reporter and asking for more information is preferrable IMHO.
          Hide
          jglick Jesse Glick added a comment -

          Closing an Incomplete means exactly that—not enough information for plugin maintainer to evaluate or fix. If you have new information, simply reopen to indicate that.

          Tried the linked Jenkinsfile. Failed earlier, as

           

          java.io.IOException: Failed to extract ws.tar.gz
          	at hudson.FilePath.readFromTar(FilePath.java:2289)
          	at hudson.FilePath.access$400(FilePath.java:195)
          	at hudson.FilePath$5.invoke(FilePath.java:538)
          	at hudson.FilePath$5.invoke(FilePath.java:536)
          	at hudson.FilePath.act(FilePath.java:996)
          	at hudson.FilePath.act(FilePath.java:974)
          	at hudson.FilePath.untar(FilePath.java:536)
          	at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:122)
          	at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64)
          	at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:54)
          	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
          	at hudson.security.ACL.impersonate(ACL.java:260)
          	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          	at java.lang.Thread.run(Thread.java:748)
          Caused by: java.io.FileNotFoundException: …/jobs/JENKINS-42863/workspace@2/.git/objects/pack/pack-04f91a10d9cec17362e55f3db8bf8cb42a1af750.idx (Permission denied)
          	at java.io.FileOutputStream.open0(Native Method)
          	at java.io.FileOutputStream.open(FileOutputStream.java:270)
          	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
          	at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
          	at hudson.util.IOUtils.copy(IOUtils.java:38)
          	at hudson.FilePath.readFromTar(FilePath.java:2279)
          	... 17 more
          

          which Blue Ocean displayed naturally.

          Do you have minimal steps to reproduce? Something that exercises solely the supposedly faulty part of docker-workflow?

          Show
          jglick Jesse Glick added a comment - Closing an Incomplete means exactly that—not enough information for plugin maintainer to evaluate or fix. If you have new information, simply reopen to indicate that. Tried the linked Jenkinsfile . Failed earlier, as   java.io.IOException: Failed to extract ws.tar.gz at hudson.FilePath.readFromTar(FilePath.java:2289) at hudson.FilePath.access$400(FilePath.java:195) at hudson.FilePath$5.invoke(FilePath.java:538) at hudson.FilePath$5.invoke(FilePath.java:536) at hudson.FilePath.act(FilePath.java:996) at hudson.FilePath.act(FilePath.java:974) at hudson.FilePath.untar(FilePath.java:536) at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:122) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:54) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47) at hudson.security.ACL.impersonate(ACL.java:260) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.FileNotFoundException: …/jobs/JENKINS-42863/workspace@2/.git/objects/pack/pack-04f91a10d9cec17362e55f3db8bf8cb42a1af750.idx (Permission denied) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.<init>(FileOutputStream.java:213) at java.io.FileOutputStream.<init>(FileOutputStream.java:162) at hudson.util.IOUtils.copy(IOUtils.java:38) at hudson.FilePath.readFromTar(FilePath.java:2279) ... 17 more which Blue Ocean displayed naturally. Do you have minimal steps to reproduce? Something that exercises solely the supposedly faulty part of docker-workflow ?
          Hide
          jglick Jesse Glick added a comment -

          To give more background on why I doubt this is in the right component: docker-workflow defines some steps and it defines a docker global variable, but when it comes to docker inspect (as run in this issue) or docker pull the plugin is merely syntactic sugar for running sh steps. If those steps correspond to processes which fail, either

          • the step is setting returnStatus: true, as we are doing for the docker inspect calls, in which case the nonzero exit code does not produce a Groovy exception and the script will continue, so it is correct for the UI to show these in green
          • it is not setting that, as for example the docker pull call, in which case a nonzero exit code should have associated an AbortException with the step via ErrorAction (which you can see most reliably in the Pipeline Steps link), and so if the UI does not display these as failures, then the UI layer (here, Blue Ocean or something probably in workflow-api it calls) is buggy
          Show
          jglick Jesse Glick added a comment - To give more background on why I doubt this is in the right component: docker-workflow defines some steps and it defines a docker global variable, but when it comes to docker inspect  (as run in this issue) or docker pull  the plugin is merely syntactic sugar for running sh steps. If those steps correspond to processes which fail, either the step is setting returnStatus: true , as we are doing for the docker inspect calls, in which case the nonzero exit code does not produce a Groovy exception and the script will continue, so it is correct for the UI to show these in green it is not setting that, as for example the docker pull call, in which case a nonzero exit code should have associated an AbortException with the step via ErrorAction (which you can see most reliably in the Pipeline Steps link), and so if the UI does not display these as failures, then the UI layer (here, Blue Ocean or something probably in workflow-api  it calls) is buggy

            People

            • Assignee:
              Unassigned
              Reporter:
              michaelneale Michael Neale
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: