Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7501

POST'ing a config.xml to create a new job via the XML API results in newlines being removed from XML elements containing them (ant build steps, winbatch steps, etc)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Not A Defect
    • Component/s: core
    • Labels:
      None
    • Environment:
      I'm using 1.374 right now, but can easily update to a new version since I'm using yum to manage my installation.
    • Similar Issues:

      Description

      I've found that with my bash clone job script [1] that when I POST a config.xml file that I've first downloaded from hudson using curl, that any ant build properties (or targets) that are separated with a newline have the newline removed during the upload/import process. For targets, this isn't as big of a deal because they can be separated by a space, but for ant properties (the -D arguments), this is a big deal because they need to be separated by a newline. I've confirmed that the downloaded file has the newline, so it seems the newline is being lost when the upload/import happens.

      [1] http://wiki.jenkins-ci.org/display/JENKINS/Bash+Job+Clone+script

        Attachments

          Activity

          Hide
          bogdaniosif bogdaniosif added a comment - - edited

          I updated the subject because the problem is more general and not specific only to ant build steps. I'm running Jenkins v1.418 and I can observe this problem with both Windows batch steps' and SetEnv plugin's configurations.

          I think there's a strong chance that when a config.xml is posted via the XML API and while the XML is read internally by Jenkins all newlines found in any element text are being stripped by the XML reader.

          I found a workaround but it is pretty ugly. What has to be done to get newlines through is to persist newlines in element text from config.xml by using the char reference for LF.

          So instead of having config.xml like this:

          <project>
          .............
            <builders>
              <hudson.tasks.BatchFile>
                <command>@ECHO OFF
          ECHO Line 1
          ECHO Line 2</command>
              </hudson.tasks.BatchFile>
            </builders>
          .............
          </project>
          

          which, when posted via the XML API, would create a job with the following undesirable content for the batch script step:

          @ECHO OFFECHO Line 1ECHO Line 2
          

          you must have config.xml like this:

          <project>
          .............
            <builders>
              <hudson.tasks.BatchFile>
                <command>@ECHO OFF&#xA;ECHO Line 1&#xA;ECHO Line 2</command>
              </hudson.tasks.BatchFile>
            </builders>
          .............
          </project>
          

          which, when posted via the XML API, would create a job with the following expected content for the batch script step:

          @ECHO OFF
          ECHO Line 1
          ECHO Line 2
          

          On Windows machines you can use the small tool I attached to this issue, LF2CharRefXmlEncoder, to prepare config.xml files before posting them to Jenkins. It takes a single parameter, the path to a config.xml and goes through the text of all its element nodes. Wherever a newline is found (be it CRLF/Windows or LF/Linux) it will be replaced with the char reference for LF. Finally, the config.xml given as a parameter is overwritten with its processed version.

          LF2CharRefXmlEncoder may work under Linux too, via Mono, because it runs on .NET Framework 2.0.

          Show
          bogdaniosif bogdaniosif added a comment - - edited I updated the subject because the problem is more general and not specific only to ant build steps. I'm running Jenkins v1.418 and I can observe this problem with both Windows batch steps' and SetEnv plugin's configurations. I think there's a strong chance that when a config.xml is posted via the XML API and while the XML is read internally by Jenkins all newlines found in any element text are being stripped by the XML reader. I found a workaround but it is pretty ugly. What has to be done to get newlines through is to persist newlines in element text from config.xml by using the char reference for LF. So instead of having config.xml like this: <project> ............. <builders> <hudson.tasks.BatchFile> <command> @ECHO OFF ECHO Line 1 ECHO Line 2 </command> </hudson.tasks.BatchFile> </builders> ............. </project> which, when posted via the XML API, would create a job with the following undesirable content for the batch script step: @ECHO OFFECHO Line 1ECHO Line 2 you must have config.xml like this: <project> ............. <builders> <hudson.tasks.BatchFile> <command> @ECHO OFF&#xA;ECHO Line 1&#xA;ECHO Line 2 </command> </hudson.tasks.BatchFile> </builders> ............. </project> which, when posted via the XML API, would create a job with the following expected content for the batch script step: @ECHO OFF ECHO Line 1 ECHO Line 2 On Windows machines you can use the small tool I attached to this issue, LF2CharRefXmlEncoder, to prepare config.xml files before posting them to Jenkins. It takes a single parameter, the path to a config.xml and goes through the text of all its element nodes. Wherever a newline is found (be it CRLF/Windows or LF/Linux) it will be replaced with the char reference for LF. Finally, the config.xml given as a parameter is overwritten with its processed version. LF2CharRefXmlEncoder may work under Linux too, via Mono, because it runs on .NET Framework 2.0.
          Hide
          mika Michael Prokop added a comment -

          This issue is still present with Jenkins ver. 1.439, I just stumbled upon it. JFYI

          Show
          mika Michael Prokop added a comment - This issue is still present with Jenkins ver. 1.439, I just stumbled upon it. JFYI
          Hide
          bmwiedemann Bernhard Wiedemann added a comment -

          still present in Jenkins 1.461
          It is breaking backup/restore for us
          because we have multi-line properties to set multiple shell variables

          Show
          bmwiedemann Bernhard Wiedemann added a comment - still present in Jenkins 1.461 It is breaking backup/restore for us because we have multi-line properties to set multiple shell variables
          Hide
          jglick Jesse Glick added a comment -

          I suspect the use of javax.xml.transform.Transformer is to blame; this has been there since the POST support was added in 2008 (eabb943).

          Show
          jglick Jesse Glick added a comment - I suspect the use of javax.xml.transform.Transformer is to blame; this has been there since the POST support was added in 2008 (eabb943).
          Hide
          jglick Jesse Glick added a comment - - edited

          Confirmed that this is broken in Jenkins dev for both creating and modifying jobs, though they seem to use different code paths:

          curl -d @config.xml -H 'Content-Type: text/xml' 'http://localhost:8080/createItem?name=new'
          curl -d @config.xml http://localhost:8080/job/existing/config.xml
          
          Show
          jglick Jesse Glick added a comment - - edited Confirmed that this is broken in Jenkins dev for both creating and modifying jobs, though they seem to use different code paths: curl -d @config.xml -H 'Content-Type: text/xml' 'http://localhost:8080/createItem?name=new' curl -d @config.xml http://localhost:8080/job/existing/config.xml
          Hide
          jglick Jesse Glick added a comment -

          In fact Jenkins is working fine. The problem is in your script: you need to use --data-binary rather than --data.

          Show
          jglick Jesse Glick added a comment - In fact Jenkins is working fine. The problem is in your script: you need to use --data-binary rather than --data .

            People

            • Assignee:
              Unassigned
              Reporter:
              latchkey latchkey
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: