Details

    • Similar Issues:
    • Released As:
      workflow-cps 2.76

      Description

      On Jenkins 2.176.2 and credentials-binding-plugin 1.19, the following pipeline will intermittently hang:

      pipeline {
          agent any
      
          stages {
              stage('bug') {
                  steps {
                      script {
                          for (int i = 0; i < 100; i++) {
                              withEnv(['A=b']) {
                                  withCredentials([]) {
                                      sh "echo hello ${i}"
                                  }
                              }
                          }
                      }
                  }
              }
          }
      }
      

      The build log looks like this:

      ...
      [Pipeline] withEnv
      [Pipeline] {
      [Pipeline] withCredentials
      

      until the build is aborted.

      I've seen two independent things that both seem to work around the problem:

      • flipping so `withCredentials([])` wraps `withEnv([...])` seems to avoid the hang.
      • ensuring there's always at least one item in the `withCredentials` list parameter seems to avoid the hang.

      We run into this in a pipeline where the parameter to withCredentials varies by projects, and sometimes is an empty list.

        Attachments

          Activity

          Hide
          jons Jon Sten added a comment -

          You're welcome, and thank you Devin Nusbaum for fixing the problem! I started to try to fix the bug, but became very fast confused and disoriented by all the different threads and objects involved, so I gave up . In other words, I appreciate experts like you that can fix these problems!

          I've given the linked experimental release a spin on our staging environment and I'm no longer able to reproduce the issue

          Show
          jons Jon Sten added a comment - You're welcome, and thank you Devin Nusbaum for fixing the problem! I started to try to fix the bug, but became very fast confused and disoriented by all the different threads and objects involved, so I gave up . In other words, I appreciate experts like you that can fix these problems! I've given the linked experimental release a spin on our staging environment and I'm no longer able to reproduce the issue
          Hide
          dnusbaum Devin Nusbaum added a comment -

          A fix for this issue was just released in Pipeline: Groovy Plugin version 2.76.

          Show
          dnusbaum Devin Nusbaum added a comment - A fix for this issue was just released in Pipeline: Groovy Plugin version 2.76.
          Hide
          reinholdfuereder Reinhold Füreder added a comment -

          Devin Nusbaum By any chance could this fix also solve my sporadically experienced hanging pipelines in withDockerRegistry/withDockerServer context (with executor on Jenkins master *cough*)?

          Example thread dump:

              at DSL.withDockerRegistry(not currently scheduled, or running blocks)
              at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:40)
              at DSL.withEnv(Native Method)
              at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:39)
              at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66)
              at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:38)
              at com.acme.php.phpstorm.OfflineInspection.execute(OfflineInspection.groovy:59)
              at acme.executePhpStormInspections(acme.groovy:256)
              at WorkflowScript.run(WorkflowScript:101)
          ...
          

          Please note, however, that it appeared as if the single or first (?) step inside such withDockerRegistry/withDockerServer contexts may sometimes actually be still executed...

          Show
          reinholdfuereder Reinhold Füreder added a comment - Devin Nusbaum By any chance could this fix also solve my sporadically experienced hanging pipelines in withDockerRegistry / withDockerServer context (with executor on Jenkins master *cough*) ? Example thread dump: at DSL.withDockerRegistry(not currently scheduled, or running blocks) at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:40) at DSL.withEnv(Native Method) at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:39) at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66) at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:38) at com.acme.php.phpstorm.OfflineInspection.execute(OfflineInspection.groovy:59) at acme.executePhpStormInspections(acme.groovy:256) at WorkflowScript.run(WorkflowScript:101) ... Please note, however, that it appeared as if the single or first (?) step inside such withDockerRegistry / withDockerServer contexts may sometimes actually be still executed...
          Hide
          dnusbaum Devin Nusbaum added a comment -

          Reinhold Füreder Yes, if you were running Docker Pipeline 1.19 or newer, in which case those steps use GeneralizedNonBlockingStepExecution (see docker-workflow PR 158), then they could have been affected by the bug and should be fixed now.

          Show
          dnusbaum Devin Nusbaum added a comment - Reinhold Füreder Yes, if you were running Docker Pipeline 1.19 or newer, in which case those steps use GeneralizedNonBlockingStepExecution (see docker-workflow PR 158 ), then they could have been affected by the bug and should be fixed now.
          Hide
          reinholdfuereder Reinhold Füreder added a comment -

          Thanks, this is very promising/good news!

          Show
          reinholdfuereder Reinhold Füreder added a comment - Thanks, this is very promising/good news!

            People

            • Assignee:
              dnusbaum Devin Nusbaum
              Reporter:
              jonathanb1 Jonathan B
            • Votes:
              4 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: