Details

    • Type: Improvement
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: sloccount-plugin
    • Labels:
      None
    • Environment:
      Platform: All, OS: All
    • Similar Issues:

      Description

      A couple of weeks after we started using the sloccount plugin our hudson started
      to crash due to out of memory exceptions. This seems to happen due to a very
      high number of instances of hudson.plugins.sloccount.model.File
      It do seem that we get one such instance per file, per build, per job.

      Secondly our build.xml files (per build, per job) have grown a lot.

      We dont uses / needs the ability to browse the size of individual files.
      Would it be possibel to have the sloccount plugin configurable, so only the
      SLOCCountTrend graph and the delta information on the build page was available
      This corresponds to the amount of information available at the 'Languages' tab
      on the 'SLOCCount Results' page

      Alternatively the plugin could read the detailed info from the sloccount.sc file
      if a user clicked the files or folder tabs. (Lazy reading)
      That would ofcourse require that the sloccount.sc was kept in the builds folder
      as the violations plugin does with the violations.xml file

        Attachments

          Issue Links

            Activity

            Hide
            rvstaveren Ruben van Staveren added a comment -

            build.xml is not the place to log the per run full sloccount data. If there is a need, only some summary could be included (as with the compiler warnings). The sloccount data should log its per run to a different xml file which is only consulted when drilling down into a run result.

            https://groups.google.com/forum/#!topic/jenkinsci-users/z66arr7QN1M

            Show
            rvstaveren Ruben van Staveren added a comment - build.xml is not the place to log the per run full sloccount data. If there is a need, only some summary could be included (as with the compiler warnings). The sloccount data should log its per run to a different xml file which is only consulted when drilling down into a run result. https://groups.google.com/forum/#!topic/jenkinsci-users/z66arr7QN1M
            Hide
            rvstaveren Ruben van Staveren added a comment -

            A similar implementation issue is found in JENKINS-19437

            Show
            rvstaveren Ruben van Staveren added a comment - A similar implementation issue is found in JENKINS-19437
            Show
            mixalturek Michal Turek added a comment - Fixed in https://github.com/mixalturek/sloccount-plugin/commit/284e6ea145aaa458078b6d16972dcc305a742836 .
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Michal Turek
            Path:
            src/main/java/hudson/plugins/sloccount/ReportSummary.java
            src/main/java/hudson/plugins/sloccount/SloccountBuildAction.java
            src/main/java/hudson/plugins/sloccount/SloccountChartBuilder.java
            src/main/java/hudson/plugins/sloccount/SloccountPublisher.java
            src/main/java/hudson/plugins/sloccount/SloccountResult.java
            src/main/java/hudson/plugins/sloccount/model/SloccountLanguageStatistics.java
            src/main/java/hudson/plugins/sloccount/model/SloccountParser.java
            src/main/java/hudson/plugins/sloccount/model/SloccountPublisherReport.java
            src/main/java/hudson/plugins/sloccount/model/SloccountReport.java
            src/main/java/hudson/plugins/sloccount/model/SloccountReportInterface.java
            http://jenkins-ci.org/commit/sloccount-plugin/284e6ea145aaa458078b6d16972dcc305a742836
            Log:
            JENKINS-4769 Memory consumption is huge

            • New data format introduced, it should be backward compatible.
            • Only statistics per language are stored in the publisher.
            • The legacy structures are still stored too but with no data (they are null). This is to be able to load the legacy data that were stored using old version of the plugin.
            • The report data can loaded from the original files after they are needed.
            • Trend graph updated to use the new language statistics.
            • Report summary updated to use the new language statistics.
            • The plugin now consumes much less memory and is much faster most of the time.
            • Only statistics are used during results publication at build time, displaying trend graph and displaying report summary. Only small statistics are deserialized.
            • Complete report is loaded only when report details page is displayed.
            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michal Turek Path: src/main/java/hudson/plugins/sloccount/ReportSummary.java src/main/java/hudson/plugins/sloccount/SloccountBuildAction.java src/main/java/hudson/plugins/sloccount/SloccountChartBuilder.java src/main/java/hudson/plugins/sloccount/SloccountPublisher.java src/main/java/hudson/plugins/sloccount/SloccountResult.java src/main/java/hudson/plugins/sloccount/model/SloccountLanguageStatistics.java src/main/java/hudson/plugins/sloccount/model/SloccountParser.java src/main/java/hudson/plugins/sloccount/model/SloccountPublisherReport.java src/main/java/hudson/plugins/sloccount/model/SloccountReport.java src/main/java/hudson/plugins/sloccount/model/SloccountReportInterface.java http://jenkins-ci.org/commit/sloccount-plugin/284e6ea145aaa458078b6d16972dcc305a742836 Log: JENKINS-4769 Memory consumption is huge New data format introduced, it should be backward compatible. Only statistics per language are stored in the publisher. The legacy structures are still stored too but with no data (they are null). This is to be able to load the legacy data that were stored using old version of the plugin. The report data can loaded from the original files after they are needed. Trend graph updated to use the new language statistics. Report summary updated to use the new language statistics. The plugin now consumes much less memory and is much faster most of the time. Only statistics are used during results publication at build time, displaying trend graph and displaying report summary. Only small statistics are deserialized. Complete report is loaded only when report details page is displayed.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Michal Turek
            Path:
            src/main/resources/hudson/plugins/sloccount/SloccountResult/files.jelly
            src/main/resources/hudson/plugins/sloccount/SloccountResult/folders.jelly
            src/main/resources/hudson/plugins/sloccount/SloccountResult/languages.jelly
            http://jenkins-ci.org/commit/sloccount-plugin/e4050c883f0604d14fb3f6dd8ea9bffe119ef858
            Log:
            JENKINS-4769 Memory consumption is huge

            • Report is cached at report details pages.
            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michal Turek Path: src/main/resources/hudson/plugins/sloccount/SloccountResult/files.jelly src/main/resources/hudson/plugins/sloccount/SloccountResult/folders.jelly src/main/resources/hudson/plugins/sloccount/SloccountResult/languages.jelly http://jenkins-ci.org/commit/sloccount-plugin/e4050c883f0604d14fb3f6dd8ea9bffe119ef858 Log: JENKINS-4769 Memory consumption is huge Report is cached at report details pages.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Michal Turek
            Path:
            src/main/java/hudson/plugins/sloccount/ReportSummary.java
            http://jenkins-ci.org/commit/sloccount-plugin/b7c0907a8c738ec0cc1ed03d2bc3acdb15828f04
            Log:
            JENKINS-4769 Memory consumption is huge

            • Previous commit added a regression, thousands in numbers are no longer separated by commas in report summary.
            • Displaying of all numbers wrapped to StringUtil.grouping() to fix the regression.
            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michal Turek Path: src/main/java/hudson/plugins/sloccount/ReportSummary.java http://jenkins-ci.org/commit/sloccount-plugin/b7c0907a8c738ec0cc1ed03d2bc3acdb15828f04 Log: JENKINS-4769 Memory consumption is huge Previous commit added a regression, thousands in numbers are no longer separated by commas in report summary. Displaying of all numbers wrapped to StringUtil.grouping() to fix the regression.

              People

              • Assignee:
                mixalturek Michal Turek
                Reporter:
                worsaae worsaae
              • Votes:
                2 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: