Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39084

PROD: cbwf-controller component is causing Jenkins's CPU to spike

    Details

    • Similar Issues:

      Description

      Within the Jenkin Pipeline View, any build that contains

      cbwf-controller

      , is causing a HTTP GET request to https://example.com/job/project/job/BRANCH_NAME/3/wfapi/changesets?_=1476824899808.

      Each branch of a project on initial load will make a request to that endpoint. That will then cause the Jenkins process spike because it is trying to fulfill all of the requests.
      An example of the element which has that controller is

      <div class="changeset-box cbwf-widget cbwf-controller-applied run-changesets" cbwf-controller="run-changesets" objecturl="/job/ssw/job/theater-client/job/SSW-416/4/wfapi/changesets"><div><div class="run-changesets">
      

      Build #10 will cause a GET request while build #9 wouldn't

        Attachments

          Issue Links

            Activity

            Hide
            svanoort Sam Van Oort added a comment -

            David Chau The cause is a Jenkins core change to fix SECURITY-243, which makes user lookup extremely expensive – this is now fixed, please see JENKINS-35493 and upgrade to Jenkins 2.7.3 or later.

            There's a workaround for people not yet able to update, and documented on the wiki: https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Stage+View+Plugin#PipelineStageViewPlugin-ConfigurableProperties%3A

            Set property com.cloudbees.workflow.rest.external.ChangeSetExt.resolveCommitAuthors to "false" to disable user lookup for changesets. This can be set at runtime to test the impact and should make your problem go away nearly immediately.

            Show
            svanoort Sam Van Oort added a comment - David Chau The cause is a Jenkins core change to fix SECURITY-243, which makes user lookup extremely expensive – this is now fixed, please see JENKINS-35493 and upgrade to Jenkins 2.7.3 or later. There's a workaround for people not yet able to update, and documented on the wiki: https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Stage+View+Plugin#PipelineStageViewPlugin-ConfigurableProperties%3A Set property com.cloudbees.workflow.rest.external.ChangeSetExt.resolveCommitAuthors to "false" to disable user lookup for changesets. This can be set at runtime to test the impact and should make your problem go away nearly immediately.
            Hide
            svanoort Sam Van Oort added a comment -

            Closing as duplicate of existing issue with documented workaround and sustainable fix.

            Show
            svanoort Sam Van Oort added a comment - Closing as duplicate of existing issue with documented workaround and sustainable fix.
            Hide
            dchau David Chau added a comment -

            The work around fixes the error but we are in Jenkins Version 2.7.4

            Show
            dchau David Chau added a comment - The work around fixes the error but we are in Jenkins Version 2.7.4
            Hide
            svanoort Sam Van Oort added a comment -

            David Chou Which would suggest that user lookup is still proving to be a bottleneck in your setup (haven't seen that happen before with the UserDetails cache on though). That's tied to your security realm – setting hudson.model.User.SECURITY_243_FULL_DEFENSE=false may also resolve the issue (and would suggest the details cache needs more work).

            Unfortunately there's not really anything to be done about this from the stage view end of things, aside from using one of the overrides in your Jenkins config file to make it permanent: the problematic API is deeply embedded in Jenkins core, and the override I put in here simply avoids invoking it. No rabbits to pull out from my hat this time, I'm afraid.

            Show
            svanoort Sam Van Oort added a comment - David Chou Which would suggest that user lookup is still proving to be a bottleneck in your setup (haven't seen that happen before with the UserDetails cache on though). That's tied to your security realm – setting hudson.model.User.SECURITY_243_FULL_DEFENSE=false may also resolve the issue (and would suggest the details cache needs more work). Unfortunately there's not really anything to be done about this from the stage view end of things, aside from using one of the overrides in your Jenkins config file to make it permanent: the problematic API is deeply embedded in Jenkins core, and the override I put in here simply avoids invoking it. No rabbits to pull out from my hat this time, I'm afraid.

              People

              • Assignee:
                svanoort Sam Van Oort
                Reporter:
                dchau David Chau
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: