Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19437

Implement load on demand functionality in Cppcheck

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Currently all violations are stored in build.xml file which drastically increase Jenkins startup time, CPU and RAM consumption. Desired implementation will be to store trend graph related data to build.xml and more detailed data (violations found in builds) store in external file which will be parsed as soon as concrete build will be chosen by user.

        Attachments

          Issue Links

            Activity

            Hide
            rvstaveren Ruben van Staveren added a comment -

            Looks like this issue could benefit from the proposed solution there

            Show
            rvstaveren Ruben van Staveren added a comment - Looks like this issue could benefit from the proposed solution there
            Hide
            rvstaveren Ruben van Staveren added a comment -

            I see the same issue with a very large amount of cppcheck data per run. probably this will be on par with less cppcheck data per run, but with many jobs as all build.xml needs to be read into memory. The description mentions the right solution. (as is done with the compiler warnings)

            A similar implementation issue is found within sloccount (JENKINS-4769)

            Show
            rvstaveren Ruben van Staveren added a comment - I see the same issue with a very large amount of cppcheck data per run. probably this will be on par with less cppcheck data per run, but with many jobs as all build.xml needs to be read into memory. The description mentions the right solution. (as is done with the compiler warnings) A similar implementation issue is found within sloccount ( JENKINS-4769 )
            Hide
            mixalturek Michal Turek added a comment -

            I will try to fix the issue the same as in SLOCCount's JENKINS-4769.

            Show
            mixalturek Michal Turek added a comment - I will try to fix the issue the same as in SLOCCount's JENKINS-4769 .
            Show
            mixalturek Michal Turek added a comment - Fixed in https://github.com/mixalturek/cppcheck-plugin/commit/e9122560bfc6944395c4a72a628310516a90bf5e , pull request sent.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Michal Turek
            Path:
            pom.xml
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckBuildAction.java
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckPublisher.java
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckReport.java
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckResult.java
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSourceContainer.java
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckStatistics.java
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSummary.java
            src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/index.jelly
            src/main/resources/tabview/main.jelly
            http://jenkins-ci.org/commit/cppcheck-plugin/e9122560bfc6944395c4a72a628310516a90bf5e
            Log:
            JENKINS-17363 Ludicrously slow load time [with lazyloading]
            JENKINS-19437 Implement load on demand functionality in Cppcheck

            • All details from Cppcheck report is stored to build.xml which is wrong. The file quickly grows with many findings and causes performance and memory issues.
            • Storing of the details to external file cppcheck_details.xml and their lazy loading implemented to fix the issue. File build.xml now contains only small statitistics neccessary for trend graph and build summary.
            • There is no need to store CppcheckReport during the build at all, its content is fully dupplicated in CppcheckSourceContainer. CppcheckStatistics has nearly identical interface so it can be used instead of CppcheckReport most of time.
            • Incorrect uses of @SuppressWarnings("unused") removed to solve compiler warnings.
            • Cppcheck tool version is stored to build.xml together with CppcheckStatistics now. This fixes a bug in the details page, the information was expected in CppcheckReport but its field is transient so the information was never stored/loaded and displayed.
            • Content of tabview/main.jelly moved dirrectly to CppcheckResult/index.jelly, include removed. This was probably only a historical location.
            • New developer added to pom.xml.
            • Several TODOs unrelated to this commit added to the code, they will be fixed in future.
            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michal Turek Path: pom.xml src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckBuildAction.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckPublisher.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckReport.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckResult.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSourceContainer.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckStatistics.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSummary.java src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/index.jelly src/main/resources/tabview/main.jelly http://jenkins-ci.org/commit/cppcheck-plugin/e9122560bfc6944395c4a72a628310516a90bf5e Log: JENKINS-17363 Ludicrously slow load time [with lazyloading] JENKINS-19437 Implement load on demand functionality in Cppcheck All details from Cppcheck report is stored to build.xml which is wrong. The file quickly grows with many findings and causes performance and memory issues. Storing of the details to external file cppcheck_details.xml and their lazy loading implemented to fix the issue. File build.xml now contains only small statitistics neccessary for trend graph and build summary. There is no need to store CppcheckReport during the build at all, its content is fully dupplicated in CppcheckSourceContainer. CppcheckStatistics has nearly identical interface so it can be used instead of CppcheckReport most of time. Incorrect uses of @SuppressWarnings("unused") removed to solve compiler warnings. Cppcheck tool version is stored to build.xml together with CppcheckStatistics now. This fixes a bug in the details page, the information was expected in CppcheckReport but its field is transient so the information was never stored/loaded and displayed. Content of tabview/main.jelly moved dirrectly to CppcheckResult/index.jelly, include removed. This was probably only a historical location. New developer added to pom.xml. Several TODOs unrelated to this commit added to the code, they will be fixed in future.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Michal Turek
            Path:
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckResult.java
            src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSourceContainer.java
            src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/index.jelly
            src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/summary.jelly
            http://jenkins-ci.org/commit/cppcheck-plugin/5db2c791e4ceb93f5cafce525ddbde5047aa0b97
            Log:
            JENKINS-17363 Ludicrously slow load time [with lazyloading]
            JENKINS-19437 Implement load on demand functionality in Cppcheck

            • Lazy loaded details are stored in CppcheckResult member variable and never released by Java garbace collector since a reference to them exists forever. All build.xml files are loaded during Jenkins startup and never released, CppcheckResult objects are part of them. This is kind of a memory leak. The update will cause slight lower performance but releasing of the memory is much more important for long run tasks/daemons/servers.
            • Method lazyLoadSourceContainer() updated to return the data instead of their caching in a member variable.
            • If cached CppcheckSourceContainer object can't be loaded, an empty one will be created. Different constructor used to remove unnecessary exception catches. Recently added related if removed from CppcheckSourceContainer.
            • Condition moved from index.jelly to summary.jelly to be able to reuse the data and remove one unnecessary loading (performance).
            • Construction "key = +key" has no meaning, updated to "+key" to resolve a warning.
            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michal Turek Path: src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckResult.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSourceContainer.java src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/index.jelly src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/summary.jelly http://jenkins-ci.org/commit/cppcheck-plugin/5db2c791e4ceb93f5cafce525ddbde5047aa0b97 Log: JENKINS-17363 Ludicrously slow load time [with lazyloading] JENKINS-19437 Implement load on demand functionality in Cppcheck Lazy loaded details are stored in CppcheckResult member variable and never released by Java garbace collector since a reference to them exists forever. All build.xml files are loaded during Jenkins startup and never released, CppcheckResult objects are part of them. This is kind of a memory leak. The update will cause slight lower performance but releasing of the memory is much more important for long run tasks/daemons/servers. Method lazyLoadSourceContainer() updated to return the data instead of their caching in a member variable. If cached CppcheckSourceContainer object can't be loaded, an empty one will be created. Different constructor used to remove unnecessary exception catches. Recently added related if removed from CppcheckSourceContainer. Condition moved from index.jelly to summary.jelly to be able to reuse the data and remove one unnecessary loading (performance). Construction "key = + key" has no meaning, updated to " +key" to resolve a warning.
            Hide
            mixalturek Michal Turek added a comment -

            Fixed, will be released in version 1.15.

            Show
            mixalturek Michal Turek added a comment - Fixed, will be released in version 1.15.

              People

              • Assignee:
                mixalturek Michal Turek
                Reporter:
                exotic Jan Magur
              • Votes:
                2 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: