Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15066

Long wait between last build step and first post build step

    Details

    • Similar Issues:

      Description

      Were are struggling with our builds that are occasionally jamming in between the last build step and first post-build action. The more builds we are running in parallel the more likely we'll face this issue. We are running Jenkins 1.454 and "execute concurrent builds" option to run multiple instances of the same job. Our master is Windows server 2008 and so are the slaves. All the builds are happening on the slaves, master is just dispatching the jobs. Slaves are virtual machines. We are not running out of memory.

      Any help is appreciated. For example what log recorders we could use to debug this issue? Jenkins default log doesn't provide any further info and neither does the console log.

        Attachments

          Activity

          Hide
          kutzi kutzi added a comment -

          Try to get some thread dumps of the 'jammed' builds (kill -3 or jstack) and post them here!

          Show
          kutzi kutzi added a comment - Try to get some thread dumps of the 'jammed' builds (kill -3 or jstack) and post them here!
          Hide
          strand Markus Strand added a comment - - edited

          jstack from a jammed build

          It is the sawnwp110_image1 slave that is running the jammed job.

          Show
          strand Markus Strand added a comment - - edited jstack from a jammed build It is the sawnwp110_image1 slave that is running the jammed job.
          Hide
          kutzi kutzi added a comment -

          This looks like the thread dump of the master process.
          We also need the thread dump of the VM process on the slave which actually runs the build.

          Show
          kutzi kutzi added a comment - This looks like the thread dump of the master process. We also need the thread dump of the VM process on the slave which actually runs the build.
          Hide
          strand Markus Strand added a comment -

          It seems like the template project plugin is causing this behaviour when running builds in parallel.

          The
          https://github.com/jenkinsci/template-project-plugin/blob/master/src/main/java/hudson/plugins/templateproject/ProxyPublisher.java#L43
          """
          return BuildStepMonitor.STEP;
          """
          Will cause the
          Method perform to block in
          http://sorcerer.jenkins-ci.org/source-view.html?hudson/tasks/BuildStepMonitor.js#34

          Show
          strand Markus Strand added a comment - It seems like the template project plugin is causing this behaviour when running builds in parallel. The https://github.com/jenkinsci/template-project-plugin/blob/master/src/main/java/hudson/plugins/templateproject/ProxyPublisher.java#L43 """ return BuildStepMonitor.STEP; """ Will cause the Method perform to block in http://sorcerer.jenkins-ci.org/source-view.html?hudson/tasks/BuildStepMonitor.js#34
          Hide
          jglick Jesse Glick added a comment -

          Waiting for PR 10 to be merged.

          Show
          jglick Jesse Glick added a comment - Waiting for PR 10 to be merged.
          Hide
          brantone Brantone added a comment -

          This is going many years on, but @strand : did this fix resolve your issues?

          Show
          brantone Brantone added a comment - This is going many years on, but @strand : did this fix resolve your issues?
          Hide
          strand Markus Strand added a comment -

          Yes, that solved this problem.

          Show
          strand Markus Strand added a comment - Yes, that solved this problem.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Brenton B
          Path:
          src/main/java/hudson/plugins/templateproject/ProxyPublisher.java
          http://jenkins-ci.org/commit/template-project-plugin/0685b9995ef788a0df2746100f1353067da425e7
          Log:
          Merge pull request #10 from jglick/BuildStepMonitor

          [FIXED JENKINS-15066] Switch to BuildStepMonitor.NONE for ProxyPublisher

          Compare: https://github.com/jenkinsci/template-project-plugin/compare/e11b170c33ed...0685b9995ef7

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Brenton B Path: src/main/java/hudson/plugins/templateproject/ProxyPublisher.java http://jenkins-ci.org/commit/template-project-plugin/0685b9995ef788a0df2746100f1353067da425e7 Log: Merge pull request #10 from jglick/BuildStepMonitor [FIXED JENKINS-15066] Switch to BuildStepMonitor.NONE for ProxyPublisher Compare: https://github.com/jenkinsci/template-project-plugin/compare/e11b170c33ed...0685b9995ef7

            People

            • Assignee:
              Unassigned
              Reporter:
              strand Markus Strand
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: