Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15358

Plotting of CSV does not work correctly, data out of sync with graph

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: plot-plugin
    • Labels:
      None
    • Environment:
      OSX Mountain Lion, Firefox browser
    • Similar Issues:

      Description

      I created a csv file with mysqlslap benchmark tool and wanted to plot the AVG, MIN, MAX values. I experimented with the settings of the plugin and I am experiencing some problems to get the correct graph. It seems like there is some syncing problem between the data used for the graph and graph showed (see attachment)

      The csv file is produced in a shell command build like this:
      echo "LABEL1, LOAD_TYPE, AVG , MIN , MAX, NR_CLIENTS, AVG_NRC_QUERY" > header.txt
      mysqlslap -uroot -a --csv=results.csv --concurrency=100 --number-of-queries=10000
      cat header.txt results.csv > report.csv

        Attachments

          Activity

          Hide
          ericbn Eric Nielsen added a comment -

          I fixed this making it read the *.csv file with plot data instead of the table_*.csv file, which was redundant and only contained data from the first series.

          Here's the patch for version 1.6-SNAPSHOT:

          diff --git a/src/main/java/hudson/plugins/plot/PlotReport.java b/src/main/java/hudson/plugins/plot/PlotReport.java
          index 73d9b6b..1cb18fd 100644
          --- a/src/main/java/hudson/plugins/plot/PlotReport.java
          +++ b/src/main/java/hudson/plugins/plot/PlotReport.java
          @@ -18,6 +18,7 @@ import org.kohsuke.stapler.StaplerRequest;
           import org.kohsuke.stapler.StaplerResponse;
           
           import au.com.bytecode.opencsv.CSVReader;
          +import org.apache.commons.lang.StringUtils;
           
           /**
            * Represents a plot report for a single group of plots.
          @@ -96,19 +97,71 @@ public class PlotReport {
           	
           	// called from PlotReport/index.jelly
               public ArrayList getTable(int i) {
          -    	ArrayList tableData = new ArrayList<String[]>();
          +    	ArrayList<ArrayList<String>> tableData = new ArrayList<ArrayList<String>>();
               	
               	Plot plot = getPlot(""+i);
               	
                   // load existing csv file
          -        File tableFile = new File(project.getRootDir(), "table_"+plot.getCsvFileName());
          -        if (!tableFile.exists()) {
          +        File plotFile = new File(project.getRootDir(), plot.getCsvFileName());
          +        if (!plotFile.exists()) {
                       return tableData;
                   }
                   CSVReader reader = null;
                   try {
          -            reader = new CSVReader(new FileReader(tableFile));
          -            tableData = (ArrayList)reader.readAll();
          +            reader = new CSVReader(new FileReader(plotFile));
          +            // throw away 2 header lines
          +            reader.readNext(); reader.readNext();
          +            // array containing header titles
          +            ArrayList<String> header = new ArrayList<String>();
          +            header.add(Messages.Plot_Build() + " #");
          +            tableData.add(header);
          +            String[] nextLine;
          +            while ((nextLine = reader.readNext()) != null) {
          +                String buildNumber = nextLine[2];
          +                if (project.getBuildByNumber(Integer.parseInt(buildNumber)) == null) {
          +                    continue;
          +                }
          +                String seriesLabel = nextLine[1];
          +                // index of the column where the value should be located
          +                int index = header.lastIndexOf(seriesLabel);
          +                if (index <= 0) {
          +                    // add header label
          +                    index = header.size();
          +                    header.add(seriesLabel);
          +                }
          +                ArrayList<String> tableRow = null;
          +                for (int j = 1; j < tableData.size(); j++) {
          +                    ArrayList<String> r = tableData.get(j);
          +                    if (StringUtils.equals(r.get(0), buildNumber)) {
          +                        // found table row corresponding to the build number
          +                        tableRow = r;
          +                        break;
          +                    }
          +                }
          +                // table row corresponding to the build number not found
          +                if (tableRow == null) {
          +                    // create table row with build number at first column
          +                    tableRow = new ArrayList<String>();
          +                    tableRow.add(buildNumber);
          +                    tableData.add(tableRow);
          +                }
          +                // set value at index column
          +                String value = nextLine[0];
          +                if (index < tableRow.size()) {
          +                    tableRow.set(index, value);
          +                } else {
          +                    for (int j = tableRow.size(); j < index; j++) {
          +                        tableRow.add(StringUtils.EMPTY);
          +                    }
          +                    tableRow.add(value);
          +                }
          +            }
          +            int lastColumn = tableData.get(0).size();
          +            for (ArrayList<String> tableRow : tableData) {
          +                for (int j = tableRow.size(); j < lastColumn; j++) {
          +                    tableRow.add(StringUtils.EMPTY);
          +                }
          +            }
                   } catch (IOException ioe) {
                       //ignore
                   } finally {
          
          Show
          ericbn Eric Nielsen added a comment - I fixed this making it read the *.csv file with plot data instead of the table_*.csv file, which was redundant and only contained data from the first series. Here's the patch for version 1.6-SNAPSHOT: diff --git a/src/main/java/hudson/plugins/plot/PlotReport.java b/src/main/java/hudson/plugins/plot/PlotReport.java index 73d9b6b..1cb18fd 100644 --- a/src/main/java/hudson/plugins/plot/PlotReport.java +++ b/src/main/java/hudson/plugins/plot/PlotReport.java @@ -18,6 +18,7 @@ import org.kohsuke.stapler.StaplerRequest; import org.kohsuke.stapler.StaplerResponse; import au.com.bytecode.opencsv.CSVReader; + import org.apache.commons.lang.StringUtils; /** * Represents a plot report for a single group of plots. @@ -96,19 +97,71 @@ public class PlotReport { // called from PlotReport/index.jelly public ArrayList getTable( int i) { - ArrayList tableData = new ArrayList< String []>(); + ArrayList<ArrayList< String >> tableData = new ArrayList<ArrayList< String >>(); Plot plot = getPlot(""+i); // load existing csv file - File tableFile = new File(project.getRootDir(), "table_" +plot.getCsvFileName()); - if (!tableFile.exists()) { + File plotFile = new File(project.getRootDir(), plot.getCsvFileName()); + if (!plotFile.exists()) { return tableData; } CSVReader reader = null ; try { - reader = new CSVReader( new FileReader(tableFile)); - tableData = (ArrayList)reader.readAll(); + reader = new CSVReader( new FileReader(plotFile)); + // throw away 2 header lines + reader.readNext(); reader.readNext(); + // array containing header titles + ArrayList< String > header = new ArrayList< String >(); + header.add(Messages.Plot_Build() + " #" ); + tableData.add(header); + String [] nextLine; + while ((nextLine = reader.readNext()) != null ) { + String buildNumber = nextLine[2]; + if (project.getBuildByNumber( Integer .parseInt(buildNumber)) == null ) { + continue ; + } + String seriesLabel = nextLine[1]; + // index of the column where the value should be located + int index = header.lastIndexOf(seriesLabel); + if (index <= 0) { + // add header label + index = header.size(); + header.add(seriesLabel); + } + ArrayList< String > tableRow = null ; + for ( int j = 1; j < tableData.size(); j++) { + ArrayList< String > r = tableData.get(j); + if (StringUtils.equals(r.get(0), buildNumber)) { + // found table row corresponding to the build number + tableRow = r; + break ; + } + } + // table row corresponding to the build number not found + if (tableRow == null ) { + // create table row with build number at first column + tableRow = new ArrayList< String >(); + tableRow.add(buildNumber); + tableData.add(tableRow); + } + // set value at index column + String value = nextLine[0]; + if (index < tableRow.size()) { + tableRow.set(index, value); + } else { + for ( int j = tableRow.size(); j < index; j++) { + tableRow.add(StringUtils.EMPTY); + } + tableRow.add(value); + } + } + int lastColumn = tableData.get(0).size(); + for (ArrayList< String > tableRow : tableData) { + for ( int j = tableRow.size(); j < lastColumn; j++) { + tableRow.add(StringUtils.EMPTY); + } + } } catch (IOException ioe) { //ignore } finally {
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Eric Nielsen
          Path:
          src/main/java/hudson/plugins/plot/Plot.java
          src/main/java/hudson/plugins/plot/PlotReport.java
          http://jenkins-ci.org/commit/plot-plugin/3899ab76ce0242dd21beb3a8a217c1c62818cc05
          Log:
          [FIXED JENKINS-15358] Not using redundant table_*.csv file anymore

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Eric Nielsen Path: src/main/java/hudson/plugins/plot/Plot.java src/main/java/hudson/plugins/plot/PlotReport.java http://jenkins-ci.org/commit/plot-plugin/3899ab76ce0242dd21beb3a8a217c1c62818cc05 Log: [FIXED JENKINS-15358] Not using redundant table_*.csv file anymore

            People

            • Assignee:
              ericbn Eric Nielsen
              Reporter:
              tooh Peter Florijn
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: