-
Type:
Bug
-
Status: Resolved (View Workflow)
-
Priority:
Blocker
-
Resolution: Fixed
-
Component/s: workflow-api-plugin
-
Labels:
-
Similar Issues:
-
Released As:workflow-api 2.31
The graph here in this description more clearly depicts whats going on. I'm the only one using this jenkins server today...
Around 5pm, i kicked off a pipeine. The pipeline builds a container and then uses that container to run some pytests...
It is during that last "postbuild" phase, the CPU runs real hot. While the pytest shards are running, they are just doing a pytest run and then i capture the junit.xml file.
The reason why this is a problem is that with too many of these running at the same time, Jenkins blips out and we cannot contact the web interface because when the CPU is pegged.. it basically crashes the master.
Here are details about my jenkins:
My jenkins is 100% current:
- Jenkins ver. 2.147
- Please note I did today accept the plugin updates described on https://github.com/jenkinsci/jep/blob/master/jep/210/README.adoc#abstract
- relates to
-
JENKINS-52729 Launcher.ProcStarter.stdout(TaskListener) discards remotability
-
- Resolved
-
-
JENKINS-38381 [JEP-210] Optimize log handling in Pipeline and Durable Task
-
- Resolved
-
- links to
Field | Original Value | New Value |
---|---|---|
Summary | Jenkins Master Host CPU pegs at 50% | Jenkins Master Host CPU pegs during parallel jobs |
Attachment | image-2018-10-14-17-35-11-037.png [ 44776 ] |
Description |
AWS CPU Reports:
!image-2018-10-14-14-48-32-774.png! SSH'ed into the Jenkins Master Host. Using nmon, I see: !image-2018-10-14-14-50-10-820.png! Docker stats on the host shows me: !image-2018-10-14-14-50-53-021.png! |
!image-2018-10-14-17-35-11-037.png|width=671,height=140! |
Description | !image-2018-10-14-17-35-11-037.png|width=671,height=140! |
The graph here in this description more clearly depicts whats going on. I'm the only one using this jenkins server today... you can
!image-2018-10-14-17-35-11-037.png|width=657,height=137! |
Attachment | image-2018-10-14-17-36-52-958.png [ 44777 ] |
Description |
The graph here in this description more clearly depicts whats going on. I'm the only one using this jenkins server today... you can
!image-2018-10-14-17-35-11-037.png|width=657,height=137! |
The graph here in this description more clearly depicts whats going on. I'm the only one using this jenkins server today... you can
!image-2018-10-14-17-35-11-037.png|width=657,height=137! Around 5pm, i kicked off a pipeine. The pipeline builds a container and then uses that container to run some pytests... !image-2018-10-14-17-36-52-958.png|width=329,height=367! It is during that last "postbuild" phase, the CPU runs real hot. While the pytest shards are running, they are just doing a pytest run and then i capture the junit.xml file. The reason why this is a problem is that with too many of these running at the same time, Jenkins blips out and we cannot contact the web interface because when the CPU is pegged.. it basically crashes the master. Here are details about my jenkins: My jenkins is 100% current: * Jenkins ver. 2.147 * Please note I did today accept the plugin updates described on [https://github.com/jenkinsci/jep/blob/master/jep/210/README.adoc#abstract] |
Attachment | image-2018-10-14-14-48-32-774.png [ 44775 ] |
Attachment | image-2018-10-14-14-50-10-820.png [ 44774 ] |
Attachment | image-2018-10-14-14-50-53-021.png [ 44773 ] |
Summary | Jenkins Master Host CPU pegs during parallel jobs | Jenkins Master Host CPU pegs to ~100% during parallel jobs |
Component/s | pipeline [ 21692 ] | |
Component/s | core [ 15593 ] |
Attachment | image-2018-10-15-00-07-35-969.png [ 44780 ] |
Attachment | image-2018-10-15-00-09-21-112.png [ 44781 ] |
Summary | Jenkins Master Host CPU pegs to ~100% during parallel jobs | "Pipeline: Job" plugin causes Jenkins Master CPU to peg to ~100% during pipelines with parallel steps |
Labels | performance regression |
Component/s | workflow-job-plugin [ 21716 ] | |
Component/s | pipeline [ 21692 ] |
Priority | Minor [ 4 ] | Major [ 3 ] |
Link |
This issue relates to |
Attachment | threaddump_jenkins-54073.txt [ 44820 ] |
Remote Link | This issue links to "workflow-api PR 81 (Web Link)" [ 21937 ] |
Assignee | Jesse Glick [ jglick ] |
Status | Open [ 1 ] | In Progress [ 3 ] |
Priority | Major [ 3 ] | Blocker [ 1 ] |
Remote Link | This issue links to "update-center2 PR 247 (Web Link)" [ 21939 ] |
Description |
The graph here in this description more clearly depicts whats going on. I'm the only one using this jenkins server today... you can
!image-2018-10-14-17-35-11-037.png|width=657,height=137! Around 5pm, i kicked off a pipeine. The pipeline builds a container and then uses that container to run some pytests... !image-2018-10-14-17-36-52-958.png|width=329,height=367! It is during that last "postbuild" phase, the CPU runs real hot. While the pytest shards are running, they are just doing a pytest run and then i capture the junit.xml file. The reason why this is a problem is that with too many of these running at the same time, Jenkins blips out and we cannot contact the web interface because when the CPU is pegged.. it basically crashes the master. Here are details about my jenkins: My jenkins is 100% current: * Jenkins ver. 2.147 * Please note I did today accept the plugin updates described on [https://github.com/jenkinsci/jep/blob/master/jep/210/README.adoc#abstract] |
The graph here in this description more clearly depicts whats going on. I'm the only one using this jenkins server today...
!image-2018-10-14-17-35-11-037.png|width=657,height=137! Around 5pm, i kicked off a pipeine. The pipeline builds a container and then uses that container to run some pytests... !image-2018-10-14-17-36-52-958.png|width=329,height=367! It is during that last "postbuild" phase, the CPU runs real hot. While the pytest shards are running, they are just doing a pytest run and then i capture the junit.xml file. The reason why this is a problem is that with too many of these running at the same time, Jenkins blips out and we cannot contact the web interface because when the CPU is pegged.. it basically crashes the master. Here are details about my jenkins: My jenkins is 100% current: * Jenkins ver. 2.147 * Please note I did today accept the plugin updates described on [https://github.com/jenkinsci/jep/blob/master/jep/210/README.adoc#abstract] |
Component/s | durable-task-plugin [ 18622 ] | |
Component/s | workflow-durable-task-step-plugin [ 21715 ] | |
Component/s | workflow-job-plugin [ 21716 ] |
Link |
This issue relates to |
Component/s | workflow-api-plugin [ 21711 ] | |
Component/s | durable-task-plugin [ 18622 ] | |
Component/s | workflow-durable-task-step-plugin [ 21715 ] |
Status | In Progress [ 3 ] | In Review [ 10005 ] |
Status | In Review [ 10005 ] | Resolved [ 5 ] |
Resolution | Fixed [ 1 ] | |
Released As | workflow-api 2.31 |
Hi. It is not analyzable without Jenkins version and thread CPU usage stats.
You can use tools like jvmtop to retrieve the JVM stats