Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-4934

Plot data is retained for deleted builds

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: plot-plugin
    • Labels:
      None
    • Environment:
      Platform: All, OS: All
    • Similar Issues:

      Description

      If one specific build is deleted the data point from the plot should be deleted.
      Currently it is not.

      E.g. My plot shows data points for builds 1 through 28. I have decided that
      build 25 is bad and is falsely skewing the plot. I delete build 25. The plot
      data point for build 25 should also be deleted.

        Attachments

          Issue Links

            Activity

            Hide
            tonylampada Tony Lampada added a comment -

            Developers, heads up!
            A new offer has been placed on FreedomSponsors for this issue.
            Please see http://www.freedomsponsors.org/core/issue/23/plot-data-is-retained-for-deleted-builds

            Voters and watchers, feel free to chip in.

            Show
            tonylampada Tony Lampada added a comment - Developers, heads up! A new offer has been placed on FreedomSponsors for this issue. Please see http://www.freedomsponsors.org/core/issue/23/plot-data-is-retained-for-deleted-builds Voters and watchers, feel free to chip in.
            Hide
            mc1arke Michael Clarke added a comment -

            How do you see this working for builds that are auto-deleted by Jenkins as part of it's normal cleanup (e.g. if a user has set builds to only be retained for 5 days, or only the last 5 builds are to be kept)? Would you expect these removed build to be automatically removed from the plot too?

            Show
            mc1arke Michael Clarke added a comment - How do you see this working for builds that are auto-deleted by Jenkins as part of it's normal cleanup (e.g. if a user has set builds to only be retained for 5 days, or only the last 5 builds are to be kept)? Would you expect these removed build to be automatically removed from the plot too?
            Hide
            drdt Don Ross added a comment -

            Absolutely. Or it should have its own setting for build retention. Either way, you should be able to trim old plot data. I can't see a way to do that.

            Show
            drdt Don Ross added a comment - Absolutely. Or it should have its own setting for build retention. Either way, you should be able to trim old plot data. I can't see a way to do that.
            Hide
            kowalcj0 Kowal Cj0 added a comment -

            As a temporary solution I wrote this bash script that deletes the build data you want to get rid off.
            Once unwanted data is gone, simply refresh the page with the plot(s). In case graph is still showing that deleted data, go to the Jenkins->Manage Jenkins and hit the "Reload Configuration from Disk" then it will work for sure

            #!/usr/bin/env bash
            #
            # @author: kowalcj0
            # @date:   2013-09-25
            # @usage: 
            #       cd to the build folder, then:
            #       example 1: delete data from build #1
            #       ./rmbuilds.sh 1
            #       example 2: delete data from builds #4,5,7,20
            #       ./rmbuilds.sh 4 5 7 20
            
            die() { echo "$@" 1>&2 ; exit 1; }
            
            [ $# -gt 1 ] || die "this script requirers at least build number!"
            
            # some global variables
            sed_reg_pattern=''
            sed_table_pattern=''
            semicolon=""
            
            # get the first argument
            # will use it to check when we should add "|" (pipe) to the pattern
            first=$1
            
            # generate SED line matching patterns based on the number of passed arguments
            for arg in $@ ; do
                # don't add semicolon before first pattern
                # of if there's only one argument passed
                if [ "${arg}" = "${first}" ] ; then
                    semicolon=""
                else
                    semicolon=";"
                fi;
                sed_reg_pattern=${sed_reg_pattern}"${semicolon}/\"${arg}\",/d"
                sed_table_pattern=${sed_table_pattern}"${semicolon}/^\"${arg}\",/d"
            done;
            
            echo "Generated SED pattern from the provided parameters: '${sed_reg_pattern}'"
            echo "Generated SED pattern from the provided parameters for files starting with table: '${sed_table_pattern}'"
            
            # iterate through all csv files which filename doesn't start with "table_"
            for f in `find * -maxdepth 1 -name "*.csv" | grep -v "^table"` ; do
                echo "Processing file: ${f}"
                # use SED to remove all matching lines
                sed -i.bak "${sed_reg_pattern}" ${f}
            done;
            
            # iterate through all csv files which filename starts with "table_"
            for f in `find * -maxdepth 1 -name "*.csv" | grep "^table"` ; do
                echo "Processing file: ${f}"
                # use SED to remove all matching lines
                sed -i.bak "${sed_table_pattern}" ${f}
            done;
            
            Show
            kowalcj0 Kowal Cj0 added a comment - As a temporary solution I wrote this bash script that deletes the build data you want to get rid off. Once unwanted data is gone, simply refresh the page with the plot(s). In case graph is still showing that deleted data, go to the Jenkins->Manage Jenkins and hit the "Reload Configuration from Disk" then it will work for sure #!/usr/bin/env bash # # @author: kowalcj0 # @date: 2013-09-25 # @usage: # cd to the build folder, then: # example 1: delete data from build #1 # ./rmbuilds.sh 1 # example 2: delete data from builds #4,5,7,20 # ./rmbuilds.sh 4 5 7 20 die() { echo "$@" 1>&2 ; exit 1; } [ $# -gt 1 ] || die "this script requirers at least build number!" # some global variables sed_reg_pattern='' sed_table_pattern='' semicolon="" # get the first argument # will use it to check when we should add "|" (pipe) to the pattern first=$1 # generate SED line matching patterns based on the number of passed arguments for arg in $@ ; do # don't add semicolon before first pattern # of if there's only one argument passed if [ "${arg}" = "${first}" ] ; then semicolon="" else semicolon=";" fi; sed_reg_pattern=${sed_reg_pattern}"${semicolon}/\"${arg}\",/d" sed_table_pattern=${sed_table_pattern}"${semicolon}/^\"${arg}\",/d" done; echo "Generated SED pattern from the provided parameters: '${sed_reg_pattern}'" echo "Generated SED pattern from the provided parameters for files starting with table: '${sed_table_pattern}'" # iterate through all csv files which filename doesn't start with "table_" for f in `find * -maxdepth 1 -name "*.csv" | grep -v "^table"` ; do echo "Processing file: ${f}" # use SED to remove all matching lines sed -i.bak "${sed_reg_pattern}" ${f} done; # iterate through all csv files which filename starts with "table_" for f in `find * -maxdepth 1 -name "*.csv" | grep "^table"` ; do echo "Processing file: ${f}" # use SED to remove all matching lines sed -i.bak "${sed_table_pattern}" ${f} done;
            Hide
            ericbn Eric Nielsen added a comment -

            Fixed it! Here is a patch for version 1.6-SNAPSHOT:

             
            diff --git a/src/main/java/hudson/plugins/plot/Plot.java b/src/main/java/hudson/plugins/plot/Plot.java
            index cb23db2..4ba5b77 100644
            --- a/src/main/java/hudson/plugins/plot/Plot.java
            +++ b/src/main/java/hudson/plugins/plot/Plot.java
            @@ -542,7 +542,8 @@
                         int buildNum;
                         try {
                             buildNum = Integer.valueOf(record[2]);
            -                if (buildNum > getRightBuildNum()) {
            +                if (project.getBuildByNumber(buildNum) == null
            +                        || buildNum > getRightBuildNum()) {
                                 continue; // skip this record
                             }
                         } catch (NumberFormatException nfe) {
            @@ -732,7 +733,9 @@
                         writer.writeNext(header2);
                         // write each entry of rawPlotData to a new line in the CSV file
                         for (String[] entry : rawPlotData) {
            -                writer.writeNext(entry);
            +                if (project.getBuildByNumber(Integer.parseInt(entry[2])) != null) {
            +                    writer.writeNext(entry);
            +                }
                         }
                     } catch (IOException ioe) {
                         //ignore
            

            The code for this plugin needs massive refactoring! I tried to fix the issue with minimal changes in code structure.

            Show
            ericbn Eric Nielsen added a comment - Fixed it! Here is a patch for version 1.6-SNAPSHOT: diff --git a/src/main/java/hudson/plugins/plot/Plot.java b/src/main/java/hudson/plugins/plot/Plot.java index cb23db2..4ba5b77 100644 --- a/src/main/java/hudson/plugins/plot/Plot.java +++ b/src/main/java/hudson/plugins/plot/Plot.java @@ -542,7 +542,8 @@ int buildNum; try { buildNum = Integer .valueOf(record[2]); - if (buildNum > getRightBuildNum()) { + if (project.getBuildByNumber(buildNum) == null + || buildNum > getRightBuildNum()) { continue ; // skip this record } } catch (NumberFormatException nfe) { @@ -732,7 +733,9 @@ writer.writeNext(header2); // write each entry of rawPlotData to a new line in the CSV file for ( String [] entry : rawPlotData) { - writer.writeNext(entry); + if (project.getBuildByNumber( Integer .parseInt(entry[2])) != null ) { + writer.writeNext(entry); + } } } catch (IOException ioe) { //ignore The code for this plugin needs massive refactoring! I tried to fix the issue with minimal changes in code structure.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Eric Nielsen
            Path:
            src/main/java/hudson/plugins/plot/Plot.java
            http://jenkins-ci.org/commit/plot-plugin/7731e712763908405afdc361d652bee5956083dc
            Log:
            [FIXED JENKINS-4934] Do not plot or save plot data for deleted builds.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Eric Nielsen Path: src/main/java/hudson/plugins/plot/Plot.java http://jenkins-ci.org/commit/plot-plugin/7731e712763908405afdc361d652bee5956083dc Log: [FIXED JENKINS-4934] Do not plot or save plot data for deleted builds.
            Hide
            frozen_man Brian Smith added a comment - - edited

            This is interesting... I have the exact opposite problem . It is currently deleting data from my plots even though I have told it to use the data from the last 300 builds...

            If you set the plot to store the last 300 data points I want it to display the last 300 data points. I don't care if the build the data is built off of is removed.

            I don't want to waste the space storing the build but I do want to keep the plot data as I am trying to view trends over time.

            Maybe this should be made an option instead of just doing it one way or the other??

            There are also many better ways to handle data that skews the plot (like setting the axis scale) than deleting it. In general terms for my use it is the data that is skewing the plot that I am looking for.

            Show
            frozen_man Brian Smith added a comment - - edited This is interesting... I have the exact opposite problem . It is currently deleting data from my plots even though I have told it to use the data from the last 300 builds... If you set the plot to store the last 300 data points I want it to display the last 300 data points. I don't care if the build the data is built off of is removed. I don't want to waste the space storing the build but I do want to keep the plot data as I am trying to view trends over time. Maybe this should be made an option instead of just doing it one way or the other?? There are also many better ways to handle data that skews the plot (like setting the axis scale) than deleting it. In general terms for my use it is the data that is skewing the plot that I am looking for.
            Show
            olivergondza Oliver Gondža added a comment - Brian Smith : I agree, https://issues.jenkins-ci.org/browse/JENKINS-22575

              People

              • Assignee:
                ericbn Eric Nielsen
                Reporter:
                glennhalstead glennhalstead
              • Votes:
                6 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: