Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-14630

Artifact archiving blocks other builds

    Details

    • Similar Issues:

      Description

      Here's what we see happen in our Jenkins instance every once in a while:
      A build (let's call A) will get stuck on the "Archiving Artifacts" step. (The artifacts are no bigger than maybe 10MBs total).
      Each other build of that same job will be blocked waiting for that build A to finish archiving its artifacts. If I cancel the 'stuck' build the other builds finish archiving their artifacts almost immediately.
      We're not building anything, this instance is solely for running Squish tests. The artifacts are the results of those tests. All tests call a single paramaterized job for their build. This build is concurrently running on separate slaves.
      These jobs are all running on Windows slaves. The master is a linux box.
      I'm not able to find any logs that have Archiving Artifact output other than the main console logs. Let me know if there's something I can attach.

        Attachments

          Activity

          Hide
          evernat evernat added a comment -

          Is it reproduced with a recent Jenkins version?

          Show
          evernat evernat added a comment - Is it reproduced with a recent Jenkins version?
          Hide
          vbaskoff Vitalii Baskov added a comment -

          Hi,
          We have similar issue on Jenkins 1.532.2 (Copy Artifact Plugin 1.27)

          Show
          vbaskoff Vitalii Baskov added a comment - Hi, We have similar issue on Jenkins 1.532.2 (Copy Artifact Plugin 1.27)
          Hide
          vbaskoff Vitalii Baskov added a comment - - edited

          In our setup we have upstream matrix-job(6 configs in total) - "jobA", which triggers downstream job - "jobB". "jobB" configured to run 2 parallel builds max, on separate slaves.

          Last build of "jobB", which processing(it takes ~5min for 100 files with total size ~1.6M) "Archiving artifacts" step, blocks artifacts sending(to upstream "jobA") for already completed "jobB" builds

          Show
          vbaskoff Vitalii Baskov added a comment - - edited In our setup we have upstream matrix-job(6 configs in total) - "jobA", which triggers downstream job - "jobB". "jobB" configured to run 2 parallel builds max, on separate slaves. Last build of "jobB", which processing(it takes ~5min for 100 files with total size ~1.6M) "Archiving artifacts" step, blocks artifacts sending(to upstream "jobA") for already completed "jobB" builds
          Hide
          lunatic Sergey Titov added a comment -

          Yes, it is reproducible on Jenkins ver. 1.532.2
          Moreover, I'd say it takes exactly 5 minutes to "archive artifacts", which is suspicious. Maybe there is some timeout?
          I look on the same configuration as Vitaliy has (but 8 configs in total).

          Here is how it looks in logs:

          ==========build #1==========
          09:02:55 BUILD SUCCESSFUL
          09:02:55 Total time: 14 minutes 34 seconds
          09:03:18 Archiving artifacts
          09:08:20 Notifying upstream projects of job completion
          09:08:20 Join notifier cannot find upstream project: SOAP_cover/Version=410,db=mysql
          09:08:20 Finished: SUCCESS
          ===========================

          And few more:
          ==========build #2==========
          09:04:03 Archiving artifacts
          09:09:04 Notifying upstream projects of job completion
          ==========build #3==========
          09:24:38 Archiving artifacts
          09:29:43 Notifying upstream projects of job completion
          ==========build #4==========
          09:24:53 Archiving artifacts
          09:29:53 Notifying upstream projects of job completion
          ==========build #5==========
          09:40:04 Archiving artifacts
          09:45:10 Notifying upstream projects of job completion
          ==========build #6==========
          09:42:20 Archiving artifacts
          09:47:21 Notifying upstream projects of job completion
          ==========build #7==========
          09:57:13 Archiving artifacts
          10:02:15 Notifying upstream projects of job completion
          ==========build #8==========
          09:57:42 Archiving artifacts
          10:02:42 Notifying upstream projects of job completion
          ===========================

          Show
          lunatic Sergey Titov added a comment - Yes, it is reproducible on Jenkins ver. 1.532.2 Moreover, I'd say it takes exactly 5 minutes to "archive artifacts", which is suspicious. Maybe there is some timeout? I look on the same configuration as Vitaliy has (but 8 configs in total). Here is how it looks in logs: ==========build #1========== 09:02:55 BUILD SUCCESSFUL 09:02:55 Total time: 14 minutes 34 seconds 09:03:18 Archiving artifacts 09:08:20 Notifying upstream projects of job completion 09:08:20 Join notifier cannot find upstream project: SOAP_cover/Version=410,db=mysql 09:08:20 Finished: SUCCESS =========================== And few more: ==========build #2========== 09:04:03 Archiving artifacts 09:09:04 Notifying upstream projects of job completion ==========build #3========== 09:24:38 Archiving artifacts 09:29:43 Notifying upstream projects of job completion ==========build #4========== 09:24:53 Archiving artifacts 09:29:53 Notifying upstream projects of job completion ==========build #5========== 09:40:04 Archiving artifacts 09:45:10 Notifying upstream projects of job completion ==========build #6========== 09:42:20 Archiving artifacts 09:47:21 Notifying upstream projects of job completion ==========build #7========== 09:57:13 Archiving artifacts 10:02:15 Notifying upstream projects of job completion ==========build #8========== 09:57:42 Archiving artifacts 10:02:42 Notifying upstream projects of job completion ===========================
          Hide
          danielbeck Daniel Beck added a comment -

          And there is no other post-build step executed (silently), e.g. an email notification etc?

          Show
          danielbeck Daniel Beck added a comment - And there is no other post-build step executed (silently), e.g. an email notification etc?

            People

            • Assignee:
              Unassigned
              Reporter:
              mdorval Michael Dorval
            • Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: