Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45628

Poor performance with authenticated user

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Not A Defect
    • Component/s: core
    • Labels:
    • Environment:
    • Similar Issues:

      Description

      Main observation is very poor UI performance when logged into the system. Interestingly, when logged in and browsing anonymously, the system responds immediately through the UI – so it appears to be an issue with logged in users.

      Other observations:

      • Using VisualVM, there appears to be a method run that uses a significant amount of CPU time with every UI interaction: jenkins.model.lazy.AbstractLazyLoadRunMap.search(). Unauthenticated users do not impact this method repeatedly.
      • Load average and overall system resources seem okay

      Things we've tried with (apparently) no effect:

      • Updating Jenkins to 2.60.1-1 (currently at 2.46.2)
      • Switching to alternate authentication mechanisms: LDAP, local UNIX user, etc. (currently Active Directory)
      • Remove user authorization by changing authorization to 'Logged-in users can do anything' (currently 'Matrix-based security')
      • Enabling tcp_tw_recycle and tcp_tw_reuse kernel params (to address large number of processes in TIME_WAIT state)
      • Updating garbage collection flags for the JVM:
        • JAVA_GC_LOGGING="Xloggc:/var/lib/jenkins/gc%t.log -XX:NumberOfGCLogFiles=5 -XX:+UseGCLogFileRotation -XX:GCLogFileSize=20m -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCCause -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC -XX:+PrintAdaptiveSizePolicy"
        • JAVA_GC_TUNING="-XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -XX:+UnlockDiagnosticVMOptions -XX:G1SummarizeRSetStatsPeriod=1 -XX:MetaspaceSize=1g -XX:+AlwaysPreTouch"

        Attachments

          Activity

          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          > jenkins.model.lazy.AbstractLazyLoadRunMap.search()

           

          I would start from disabling the Weather column in the default views. "LastBuild" and other such columns also contribute to these method calls.

          If you have hundreds of jobs on the top level, consider organizing them to Folders

          Show
          oleg_nenashev Oleg Nenashev added a comment - > jenkins.model.lazy.AbstractLazyLoadRunMap.search()   I would start from disabling the Weather column in the default views. "LastBuild" and other such columns also contribute to these method calls. If you have hundreds of jobs on the top level, consider organizing them to Folders
          Hide
          danielbeck Daniel Beck added a comment -

          IIRC Build Pipeline is known to be expensive to memory (loading many builds).

          Notably, this is only a persistent(ish) problem if you have way more builds than fit in RAM, so perhaps there's a mismatch between Jenkins and master server sizing.

          Not a bug in core though.

          Show
          danielbeck Daniel Beck added a comment - IIRC Build Pipeline is known to be expensive to memory (loading many builds). Notably, this is only a persistent(ish) problem if you have way more builds than fit in RAM, so perhaps there's a mismatch between Jenkins and master server sizing. Not a bug in core though.

            People

            • Assignee:
              Unassigned
              Reporter:
              mjmccormac Michael McCormac
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: