Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20505

gradle build failed due to multiline build parameter

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: gradle-plugin
    • Labels:
      None
    • Environment:
      Any
    • Similar Issues:

      Description

      I added a Build parameter of type text which contains multiline text for use by another plugin. For some reason, gradle plugindecided it needed to pass it gradle using -D options, but because the value was multiline the command is not valid

        Attachments

          Activity

          Hide
          f_ritaly FRANCOIS RITALY added a comment -

          Hello Gregory,

          I have the exact same issue with jenkins 1.532.1 & gradle-plugin 1.23. I need to create a job where one of the parameters is a text one (in order to specify release notes).

          The text parameter has the following value:

          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          

          When the Gradle builder runs, it fails with the following error which shows that Gradle is confused by the multi-line argument:

          Started by user anonymous
          Building in workspace /Users/francois_ritaly/git/gradle-plugin/work/jobs/TEST/workspace
          [Gradle] - Launching build.
          [workspace] $ gradle "-DRELEASE_NOTES=- Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes" clean tasks
          Projects loaded
          
          FAILURE: Could not determine which tasks to execute.
          
          * What went wrong:
          Task '-DRELEASE_NOTES=- Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes' not found in root project 'workspace'.
          
          * Try:
          Run gradle tasks to get a list of available tasks.
          
          BUILD FAILED
          
          Total time: 4.048 secs
          Build step 'Invoke Gradle script' changed build result to FAILURE
          Build step 'Invoke Gradle script' marked build as failure
          Finished: FAILURE
          

          I did some local tests and found that (on Mac OS), the command line should be changed from

          gradle "-DRELEASE_NOTES=- Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes
          - Dummy release notes" clean tasks
          

          to

          gradle "-DRELEASE_NOTES=- Dummy release notes \
          - Dummy release notes \
          - Dummy release notes \
          - Dummy release notes \
          - Dummy release notes \
          - Dummy release notes" clean tasks
          

          Here's the command output when run outside jenkins:

          [15:59:39 francois_ritaly@fritaly-mac:~/releasable-project/trunk]$ gradle "-DRELEASE_NOTES=- Dummy release notes \
          > - Dummy release notes \
          > - Dummy release notes \
          > - Dummy release notes \
          > - Dummy release notes \
          > - Dummy release notes" clean tasks
          :clean UP-TO-DATE
          :tasks
          (log truncated)
          

          I tried changing the gradle plugin to have the plugin generate the command line above. For that, I updated the method hudson.plugins.gradle.Gradle.fixParameters(Map<String, String>) to handle multi-line parameter values.

          Initial implementation:

              private Map<String, String> fixParameters(Map<String, String> parmas) {
              	Map<String, String> result = new HashMap<String, String>();
                  for (Map.Entry<String, String> entry : parmas.entrySet()) {
                      String value = entry.getValue();
                      if (isValue2Escape(value)) {
                          result.put(entry.getKey(), "\"" + value + "\"");
                      } else {
                          result.put(entry.getKey(), value);
                      }
                  }
                  return result;
              }
          

          Implementation I tested:

              private Map<String, String> fixParameters(Map<String, String> parmas) {
                  Map<String, String> result = new HashMap<String, String>();
          
                  // TODO should this line separator depend on the launcher type (unix / windows) ?
                  final String lineSeparator = System.getProperty("line.separator");
          
                  for (Map.Entry<String, String> entry : parmas.entrySet()) {
                      final String value = entry.getValue();
          
                      if (!value.contains(lineSeparator)) {
                      	// the initial implementation expected all values to be a single line (no line feed)
                          if (isValue2Escape(value)) {
                              result.put(entry.getKey(), "\"" + value + "\"");
                          } else {
                              result.put(entry.getKey(), value);
                          }
                      } else {
                      	// the value contains line feeds, escape each line and the line feeds
                      	final StringBuilder buffer = new StringBuilder(256);
          
                      	for (String string: value.split(lineSeparator)) {
          	                if (isValue2Escape(string)) {
          	                	buffer.append("\"" + string + "\"");
          	                } else {
          	                	buffer.append(string);
          	                }
          
          	                // escape line feeds
          	                buffer.append("\\").append(lineSeparator);
          				}
          
                      	result.put(entry.getKey(), buffer.toString());
                      }
          
                  }
                  return result;
              }
          

          With this second implementation, the plugin properly escapes line feeds (see below) but the jenkins job still fails with the same error:

          Started by user anonymous
          Building in workspace /Users/francois_ritaly/git/gradle-plugin/work/jobs/TEST/workspace
          [Gradle] - Launching build.
          [workspace] $ gradle "-DRELEASE_NOTES=- Dummy release notes\
          - Dummy release notes\
          - Dummy release notes\
          - Dummy release notes\
          - Dummy release notes\
          - Dummy release notes\
          " clean tasks
          Projects loaded
          FAILURE: Could not determine which tasks to execute.
          
          * What went wrong:
          Task '-DRELEASE_NOTES=- Dummy release notes\
          - Dummy release notes\
          - Dummy release notes\
          - Dummy release notes\
          - Dummy release notes\
          - Dummy release notes\
          ' not found in root project 'workspace'.
          
          * Try:
          Run gradle tasks to get a list of available tasks.
          
          BUILD FAILED
          
          Total time: 4.121 secs
          Build step 'Invoke Gradle script' changed build result to FAILURE
          Build step 'Invoke Gradle script' marked build as failure
          Finished: FAILURE
          

          The topic of handling multi-line parameters seems to be a tricky one (because very OS-specific).

          Any idea what could be wrong with the above ? Is the method hudson.plugins.gradle.Gradle.fixParameters(Map<String, String>) the one to be changed for fixing this issue ?

          I also found some code in jenkins itself (method hudson.util.ArgumentListBuilder.addKeyValuePair(String, String, String, boolean)) that deals with those system properties but I'm pretty reluctant to change such a low-level method.

          Thanks for your insight

          Show
          f_ritaly FRANCOIS RITALY added a comment - Hello Gregory, I have the exact same issue with jenkins 1.532.1 & gradle-plugin 1.23. I need to create a job where one of the parameters is a text one (in order to specify release notes). The text parameter has the following value: - Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes When the Gradle builder runs, it fails with the following error which shows that Gradle is confused by the multi-line argument: Started by user anonymous Building in workspace /Users/francois_ritaly/git/gradle-plugin/work/jobs/TEST/workspace [Gradle] - Launching build. [workspace] $ gradle "-DRELEASE_NOTES=- Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes" clean tasks Projects loaded FAILURE: Could not determine which tasks to execute. * What went wrong: Task '-DRELEASE_NOTES=- Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes' not found in root project 'workspace'. * Try: Run gradle tasks to get a list of available tasks. BUILD FAILED Total time: 4.048 secs Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure Finished: FAILURE I did some local tests and found that (on Mac OS), the command line should be changed from gradle "-DRELEASE_NOTES=- Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes - Dummy release notes" clean tasks to gradle "-DRELEASE_NOTES=- Dummy release notes \ - Dummy release notes \ - Dummy release notes \ - Dummy release notes \ - Dummy release notes \ - Dummy release notes" clean tasks Here's the command output when run outside jenkins : [15:59:39 francois_ritaly@fritaly-mac:~/releasable-project/trunk]$ gradle "-DRELEASE_NOTES=- Dummy release notes \ > - Dummy release notes \ > - Dummy release notes \ > - Dummy release notes \ > - Dummy release notes \ > - Dummy release notes" clean tasks :clean UP-TO-DATE :tasks (log truncated) I tried changing the gradle plugin to have the plugin generate the command line above. For that, I updated the method hudson.plugins.gradle.Gradle.fixParameters(Map<String, String>) to handle multi-line parameter values. Initial implementation: private Map< String , String > fixParameters(Map< String , String > parmas) { Map< String , String > result = new HashMap< String , String >(); for (Map.Entry< String , String > entry : parmas.entrySet()) { String value = entry.getValue(); if (isValue2Escape(value)) { result.put(entry.getKey(), "\" " + value + " \""); } else { result.put(entry.getKey(), value); } } return result; } Implementation I tested: private Map< String , String > fixParameters(Map< String , String > parmas) { Map< String , String > result = new HashMap< String , String >(); // TODO should this line separator depend on the launcher type (unix / windows) ? final String lineSeparator = System .getProperty( "line.separator" ); for (Map.Entry< String , String > entry : parmas.entrySet()) { final String value = entry.getValue(); if (!value.contains(lineSeparator)) { // the initial implementation expected all values to be a single line (no line feed) if (isValue2Escape(value)) { result.put(entry.getKey(), "\" " + value + " \""); } else { result.put(entry.getKey(), value); } } else { // the value contains line feeds, escape each line and the line feeds final StringBuilder buffer = new StringBuilder(256); for ( String string: value.split(lineSeparator)) { if (isValue2Escape(string)) { buffer.append( "\" " + string + " \""); } else { buffer.append(string); } // escape line feeds buffer.append( "\\" ).append(lineSeparator); } result.put(entry.getKey(), buffer.toString()); } } return result; } With this second implementation, the plugin properly escapes line feeds (see below) but the jenkins job still fails with the same error: Started by user anonymous Building in workspace /Users/francois_ritaly/git/gradle-plugin/work/jobs/TEST/workspace [Gradle] - Launching build. [workspace] $ gradle "-DRELEASE_NOTES=- Dummy release notes\ - Dummy release notes\ - Dummy release notes\ - Dummy release notes\ - Dummy release notes\ - Dummy release notes\ " clean tasks Projects loaded FAILURE: Could not determine which tasks to execute. * What went wrong: Task '-DRELEASE_NOTES=- Dummy release notes\ - Dummy release notes\ - Dummy release notes\ - Dummy release notes\ - Dummy release notes\ - Dummy release notes\ ' not found in root project 'workspace'. * Try: Run gradle tasks to get a list of available tasks. BUILD FAILED Total time: 4.121 secs Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure Finished: FAILURE The topic of handling multi-line parameters seems to be a tricky one (because very OS-specific). Any idea what could be wrong with the above ? Is the method hudson.plugins.gradle.Gradle.fixParameters(Map<String, String>) the one to be changed for fixing this issue ? I also found some code in jenkins itself (method hudson.util.ArgumentListBuilder.addKeyValuePair(String, String, String, boolean)) that deals with those system properties but I'm pretty reluctant to change such a low-level method. Thanks for your insight
          Hide
          f_ritaly FRANCOIS RITALY added a comment -

          This is the same issue as JENKINS-17871.

          Show
          f_ritaly FRANCOIS RITALY added a comment - This is the same issue as JENKINS-17871 .
          Hide
          f_ritaly FRANCOIS RITALY added a comment - - edited

          In fact this comment is wrong (the approach can't solve the issue).

          Show
          f_ritaly FRANCOIS RITALY added a comment - - edited In fact this comment is wrong (the approach can't solve the issue).
          Hide
          wallacen60 Ian Wallace-Hoyt added a comment -

          Hi Francois,

          I just discovered that the plugin passes build parameters to gradle as -D options. I'm using parameters to control behavior completely un-related to gradle, so was surprised by this. In fact, I've stopped using the plugin for the time being. It wasn't causing any problems per se, but I didn't want anyone inadvertently affect the gradle invocation in the future by adding a new parameter.

          I'm sure there is a good reason build parameters are treated this way, but the use-case isn't obvious to me. Would you mind explaining the rationale behind this?

          Show
          wallacen60 Ian Wallace-Hoyt added a comment - Hi Francois, I just discovered that the plugin passes build parameters to gradle as -D options. I'm using parameters to control behavior completely un-related to gradle, so was surprised by this. In fact, I've stopped using the plugin for the time being. It wasn't causing any problems per se, but I didn't want anyone inadvertently affect the gradle invocation in the future by adding a new parameter. I'm sure there is a good reason build parameters are treated this way, but the use-case isn't obvious to me. Would you mind explaining the rationale behind this?
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Stefan Wolf
          Path:
          README.md
          src/test/groovy/hudson/plugins/gradle/GradlePluginIntegrationTest.groovy
          http://jenkins-ci.org/commit/gradle-plugin/db9a28079bf437dd9f7c0a8cbd121e9a0c441d46
          Log:
          Add a test for multiline parameters

          [FIXED JENKINS-20505]

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stefan Wolf Path: README.md src/test/groovy/hudson/plugins/gradle/GradlePluginIntegrationTest.groovy http://jenkins-ci.org/commit/gradle-plugin/db9a28079bf437dd9f7c0a8cbd121e9a0c441d46 Log: Add a test for multiline parameters [FIXED JENKINS-20505]
          Hide
          wolfs Stefan Wolf added a comment -

          1.27 has been released.

          Show
          wolfs Stefan Wolf added a comment - 1.27 has been released.

            People

            • Assignee:
              wolfs Stefan Wolf
              Reporter:
              dalewking Dale King
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: