Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39661

Investigate performance of Pipeline /activity and /runs REST endpoint

    Details

    • Epic Link:
    • Sprint:
      arctic, tasman
    • Similar Issues:

      Description

      The /activity and /runs endpoints have been reported to take up to 28 seconds on some Jenkins masters.

      Tracing through the code, it looks like io.jenkins.blueocean.service.embedded.rest.RunContainerImpl#get will load all the runs using hudson.model.Job#getBuilds() for the Job then applies the pagination.

      However, there is a method hudson.model.Job#getBuilds(hudson.model.Fingerprint.RangeSet) that will allow us to fetch a range of data which would be suitable for pagination.

        Attachments

          Issue Links

            Activity

            Hide
            vivek Vivek Pandey added a comment -

            Tom FENNELLY Yeah runs not jobs, thanks!

            Show
            vivek Vivek Pandey added a comment - Tom FENNELLY Yeah runs not jobs , thanks!
            Hide
            michaelneale Michael Neale added a comment - - edited

            ok Vivek Pandey we have opened other tickets for the artifact stuff, which will probably improve things a bit (in your testing you might want to comment out all artifacts and see if it is still slow).

            Based on my calculations looking at the json from https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fpipeline-model-definition-plugin/detail/PR-54/1/artifacts/ (ie the https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fpipeline-model-definition-plugin project) - it takes around 10 seconds to load one runs artifact flat listing (there are 1700 odd artifacts). Multiply that by a page worth of activity (26 items), and you get about 4 minutes latency.

            So this could be the cause of the slowness, at least in the case of Andrew Bayer's pipeline.

            Still need to run a pressure test anyway as suggested by tom's script.

            Show
            michaelneale Michael Neale added a comment - - edited ok Vivek Pandey we have opened other tickets for the artifact stuff, which will probably improve things a bit (in your testing you might want to comment out all artifacts and see if it is still slow). Based on my calculations looking at the json from https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fpipeline-model-definition-plugin/detail/PR-54/1/artifacts/ (ie the https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fpipeline-model-definition-plugin project) - it takes around 10 seconds to load one runs artifact flat listing (there are 1700 odd artifacts). Multiply that by a page worth of activity (26 items), and you get about 4 minutes latency. So this could be the cause of the slowness, at least in the case of Andrew Bayer 's pipeline. Still need to run a pressure test anyway as suggested by tom's script.
            Hide
            vivek Vivek Pandey added a comment -

            Tom FENNELLY I have create 6000 runs of a simple pipeline, fresh load time (on new browser is about 600ms). Maybe its my pipeline is too simple to reproduce 10s of sec initial load time? Its all on my laptop.

            Here is my pipeline:

            stage 'build'
            node{
              echo "Building..."
            }
            stage 'deploy'
            node{
              echo "Deploying"
            }
            

            Can you share your pipeline? Maybe that will help me diagnose. I am proceeding with a fix regardless, however need to validate it so need to reproduce.

            Thanks buddy!

            Show
            vivek Vivek Pandey added a comment - Tom FENNELLY I have create 6000 runs of a simple pipeline, fresh load time (on new browser is about 600ms). Maybe its my pipeline is too simple to reproduce 10s of sec initial load time? Its all on my laptop. Here is my pipeline: stage 'build' node{ echo "Building..." } stage 'deploy' node{ echo "Deploying" } Can you share your pipeline? Maybe that will help me diagnose. I am proceeding with a fix regardless, however need to validate it so need to reproduce. Thanks buddy!
            Hide
            tfennelly Tom FENNELLY added a comment -

            Vivek Pandey Hey Vivek ... afraid I wiped my test env, but it was a simple enough pipeline that just echoed to the log about 100 or 200 times (I forget exactly, but wasn't a lot).

            So something like...

            for (i = 0; i < 200; i++) {
                echo "Vivek is an awesome guy, except that he loves Donald Trump"
            }
            
            Show
            tfennelly Tom FENNELLY added a comment - Vivek Pandey Hey Vivek ... afraid I wiped my test env, but it was a simple enough pipeline that just echoed to the log about 100 or 200 times (I forget exactly, but wasn't a lot). So something like... for (i = 0; i < 200; i++) { echo "Vivek is an awesome guy, except that he loves Donald Trump" }
            Hide
            vivek Vivek Pandey added a comment -

            Tom FENNELLY Ha ha! That test is some evil fantasy of yours

            Ok so test is fine. For some reason I don't see that kind of slow performance, relatively speaking I am getting about 300% improvement with my change. 670ms to ~250ms. I am going to put a PR.

            Show
            vivek Vivek Pandey added a comment - Tom FENNELLY Ha ha! That test is some evil fantasy of yours Ok so test is fine. For some reason I don't see that kind of slow performance, relatively speaking I am getting about 300% improvement with my change. 670ms to ~250ms. I am going to put a PR.

              People

              • Assignee:
                vivek Vivek Pandey
                Reporter:
                jamesdumay James Dumay
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: