Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54181

Performance Plugin - Performance report displayed incorrect when compare Reports with Build number

    Details

    • Similar Issues:

      Description

      Artem Fedorov

      Jenkins ver:2.133

      Performance Plugin:3.11

      Compare reports with build number not working.

      I run build #56 and I waited compare with build #53, check the log in jenkins console, it looks like correct to compare with #53. However when look performance report, it compares with the previous version #55. 

      I would like to confirm whether the number of changes shown in the report is related to the COMPARED_NUMBER in jenkins configuration. And why the performance report is not consistent with jenkins log.

       

        Attachments

        1. BuildHistory.PNG
          BuildHistory.PNG
          96 kB
        2. Capture.PNG
          Capture.PNG
          73 kB
        3. Console #55.PNG
          Console #55.PNG
          68 kB
        4. Console-53.PNG
          Console-53.PNG
          27 kB
        5. Console-55.PNG
          Console-55.PNG
          27 kB
        6. Result #53.png
          Result #53.png
          278 kB
        7. Result #55.png
          Result #55.png
          302 kB
        8. Result #56.png
          Result #56.png
          302 kB
        9. Trend.PNG
          Trend.PNG
          114 kB

          Activity

          christy Liu Wang created issue -
          Hide
          artem_fedorov Artem Fedorov added a comment -

          Sure. It looks like comparison between 55 and 56 build. 
           Could you add performance trend for 55 build? And I want to know build statuses for builds 53-56 

          Show
          artem_fedorov Artem Fedorov added a comment - Sure. It looks like comparison between 55 and 56 build.   Could you add performance trend for 55 build? And I want to know build statuses for builds 53-56 
          christy Liu Wang made changes -
          Field Original Value New Value
          Description [~artem_fedorov]

          Jenkins ver:2.133

          Performance Plugin:3.11

          Compare reports with build number not working.

          Output:
           !Capture.PNG|width=678,height=275!

          I run build #56 and I waited compare with build #53, look through the log in jenkins console, it looks like correct to compare with #53. However when look performance report, it compares with the previous version #55.

          !Result #56.png|width=652,height=341!
          [~artem_fedorov]

          Jenkins ver:2.133

          Performance Plugin:3.11

          Compare reports with build number not working.

          I run build #56 and I waited compare with build #53, check the log in jenkins console, it looks like correct to compare with #53. However when look performance report, it compares with the previous version #55. 

          I would like to confirm whether the number of changes shown in the report is related to the COMPARED_NUMBER in jenkins configuration. And why the performance report is not consistent with jenkins log.

           
          christy Liu Wang made changes -
          Attachment Result #56.png [ 44828 ]
          christy Liu Wang made changes -
          Attachment Capture.PNG [ 44829 ]
          christy Liu Wang made changes -
          Attachment Result #53.png [ 44830 ]
          artem_fedorov Artem Fedorov made changes -
          Assignee Andrey Pokhilko [ undera ] Artem Fedorov [ artem_fedorov ]
          christy Liu Wang made changes -
          Attachment Result #55.png [ 44831 ]
          christy Liu Wang made changes -
          Attachment Trend.PNG [ 44832 ]
          Hide
          christy Liu Wang added a comment -

          Artem Fedorov

          I didn't find performance trend for 55 build, I attached the performance report of 55 build and the performance trend of this pipeline.

          Is there any information I need to provide?

          Show
          christy Liu Wang added a comment - Artem Fedorov I didn't find performance trend for 55 build, I attached the performance report of 55 build and the performance trend of this pipeline. Is there any information I need to provide?
          Hide
          artem_fedorov Artem Fedorov added a comment -

           This image contains build 56 info..
          Also could you please add Console log for 55 build and Build History (from main job page) for 53-56 builds

          Show
          artem_fedorov Artem Fedorov added a comment -  This image contains build 56 info.. Also could you please add Console log for 55 build and Build History (from main job page) for 53-56 builds
          christy Liu Wang made changes -
          Attachment Console #55.PNG [ 44833 ]
          christy Liu Wang made changes -
          Attachment Build history.PNG [ 44834 ]
          christy Liu Wang made changes -
          Attachment Build history.PNG [ 44834 ]
          christy Liu Wang made changes -
          Attachment BuildHistory.PNG [ 44835 ]
          Hide
          christy Liu Wang added a comment -

          55 build is also compared with 53 build, but performance report shows it always comparing with the previous build(#54) based on Average response time. Although the build of 54 is aborted.

          Show
          christy Liu Wang added a comment - 55 build is also compared with 53 build, but performance report shows it always comparing with the previous build(#54) based on Average response time. Although the build of 54 is aborted.
          Hide
          artem_fedorov Artem Fedorov added a comment -

          Sure
          Could you add the last lines from Console log for 53 and 55 build? 
          E.g.: 

          The label "Dummy" caused the build to fail
          Build step 'Publish Performance test result report' changed build result to FAILURE
          Also it will be useful if you add config for Performance Publisher step 

          Show
          artem_fedorov Artem Fedorov added a comment - Sure Could you add the last lines from Console log for 53 and 55 build?  E.g.:  The label "Dummy" caused the build to fail Build step 'Publish Performance test result report' changed build result to FAILURE Also it will be useful if you add config for Performance Publisher step 
          christy Liu Wang made changes -
          Attachment Console-53.PNG [ 44836 ]
          christy Liu Wang made changes -
          Attachment Console-55.PNG [ 44837 ]
          christy Liu Wang made changes -
          Attachment Console-55.PNG [ 44837 ]
          christy Liu Wang made changes -
          Attachment Console-55.PNG [ 44838 ]
          christy Liu Wang made changes -
          Attachment Console-53.PNG [ 44836 ]
          christy Liu Wang made changes -
          Attachment Console-53.PNG [ 44839 ]
          Hide
          christy Liu Wang added a comment -
           if (env.COMPARED_NUMBER == '') {
                  echo "Compare with Previous Successful Number"
                  perfReport compareBuildPrevious: true, modeOfThreshold: true, percentiles: '0,50,90,100', relativeFailedThresholdNegative: 20.0,
          	    relativeFailedThresholdPositive: 15.0, relativeUnstableThresholdNegative: 10.0, relativeUnstableThresholdPositive: 10.0,
          	    sourceDataFiles: 'independence-solution-loadtests-'+version+'.'+buildNum+'/report/result.jtl'
                } else {
                  echo "Compare with Specific Number ${env.COMPARED_NUMBER}"
                  def compareNum = sh(script: "echo ${env.COMPARED_NUMBER}", returnStdout: true).toInteger()
                  perfReport modeOfThreshold: true, nthBuildNumber: compareNum, percentiles: '0,50,90,100', relativeFailedThresholdNegative: 20.0,
                  relativeFailedThresholdPositive: 15.0, relativeUnstableThresholdNegative: 10.0, relativeUnstableThresholdPositive: 10.0,
                  sourceDataFiles: 'independence-solution-loadtests-'+version+'.'+buildNum+'/report/result.jtl'
                }
                
                echo currentBuild.result
                if (currentBuild.result == 'UNSTABLE' || currentBuild.result == 'FAILURE') {
                  emailext body: '${JELLY_SCRIPT, template="html-with-health-and-console"}', subject: "Build failed: ${env.JOB_NAME}", to: '${DEFAULT_RECIPIENTS}', replyTo: 'no-reply@kingland.com', mimeType: 'text/html';
                }
          

          I added the codes that configuration of performance report in jenkinsfile.

          Show
          christy Liu Wang added a comment - if (env.COMPARED_NUMBER == '') { echo "Compare with Previous Successful Number " perfReport compareBuildPrevious: true , modeOfThreshold: true , percentiles: '0,50,90,100' , relativeFailedThresholdNegative: 20.0, relativeFailedThresholdPositive: 15.0, relativeUnstableThresholdNegative: 10.0, relativeUnstableThresholdPositive: 10.0, sourceDataFiles: 'independence-solution-loadtests-' +version+ '.' +buildNum+ '/report/result.jtl' } else { echo "Compare with Specific Number ${env.COMPARED_NUMBER}" def compareNum = sh(script: "echo ${env.COMPARED_NUMBER}" , returnStdout: true ).toInteger() perfReport modeOfThreshold: true , nthBuildNumber: compareNum, percentiles: '0,50,90,100' , relativeFailedThresholdNegative: 20.0, relativeFailedThresholdPositive: 15.0, relativeUnstableThresholdNegative: 10.0, relativeUnstableThresholdPositive: 10.0, sourceDataFiles: 'independence-solution-loadtests-' +version+ '.' +buildNum+ '/report/result.jtl' } echo currentBuild.result if (currentBuild.result == 'UNSTABLE' || currentBuild.result == 'FAILURE' ) { emailext body: '${JELLY_SCRIPT, template= "html-with-health-and-console" }' , subject: "Build failed: ${env.JOB_NAME}" , to: '${DEFAULT_RECIPIENTS}' , replyTo: 'no-reply@kingland.com' , mimeType: 'text/html' ; } I added the codes that configuration of performance report in jenkinsfile.
          Hide
          artem_fedorov Artem Fedorov added a comment -

          OK
          Will try to fix it on this week

          Show
          artem_fedorov Artem Fedorov added a comment - OK Will try to fix it on this week
          Hide
          christy Liu Wang added a comment -

          Thank you very much for your help, looking forward to your good news.

          Show
          christy Liu Wang added a comment - Thank you very much for your help, looking forward to your good news.
          Hide
          christy Liu Wang added a comment -

          Hi Artem Fedorov,

          Is there any update for this issue?

          Show
          christy Liu Wang added a comment - Hi Artem Fedorov , Is there any update for this issue?
          Hide
          artem_fedorov Artem Fedorov added a comment -

          Hi Liu Wang 
          I working on this issue now. I hope that I'll finished it today/tomorrow 

          Show
          artem_fedorov Artem Fedorov added a comment - Hi Liu Wang   I working on this issue now. I hope that I'll finished it today/tomorrow 
          Hide
          artem_fedorov Artem Fedorov added a comment -

          Hi Liu Wang
          In version 3.13 we added `Baseline build` option in Advanced options. If this value is empty or 0 - all Performance trends will be compare with previous build, if not 0 then Performance Trend will have comparison with selected build
          https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/performance/3%EF%BF%BD%EF%BF%BD.13/performance-3%EF%BF%BD%EF%BF%BD.13.hpi 

          Show
          artem_fedorov Artem Fedorov added a comment - Hi Liu Wang In version 3.13 we added `Baseline build` option in Advanced options. If this value is empty or 0 - all Performance trends will be compare with previous build, if not 0 then Performance Trend will have comparison with selected build https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/performance/3%EF%BF%BD%EF%BF%BD.13/performance-3%EF%BF%BD%EF%BF%BD.13.hpi  
          Hide
          artem_fedorov Artem Fedorov added a comment -

          fixed in 3.13

          Show
          artem_fedorov Artem Fedorov added a comment - fixed in 3.13
          artem_fedorov Artem Fedorov made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          christy Liu Wang added a comment -

          Artem Fedorov

          Thanks for your help, I will try to update the plugin when v3.13 is released.

          Show
          christy Liu Wang added a comment - Artem Fedorov Thanks for your help, I will try to update the plugin when v3.13 is released.
          Show
          artem_fedorov Artem Fedorov added a comment - https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/performance/3.13/performance-3.13.hpi

            People

            • Assignee:
              artem_fedorov Artem Fedorov
              Reporter:
              christy Liu Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: