Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48648

Violation Comments to Bitbucket Server plugin does not add comments to files with non-ASCII names on Windows

    Details

    • Similar Issues:

      Description

      I deliberately changed three files so that the build system generates warnings that the Violation Comments to Bitbucket Server Plugin should report as comments to a pull request:

      • mobiilirajapinta/json-dateandtime.md
      • mobiilirajapinta/json-nimeämiskäytäntö.md
      • sanasto.md

      The plugin does add comments about the other two files but not about "mobiilirajapinta/json-nimeämiskäytäntö.md", whose name contains non-ASCII characters. This happens on Windows.

      The console output says:

      Found 3 violations from ViolationConfig [pattern=.*\\docfx\.log\.xml, reporter=DocFX, parser=JCREPORT].

      Jenkins loggers say:

      joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi
      
      Found 1 reports:
      
      joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi
      
      C:\JenkTst\agent1\workspace\n-mobiilin-dokumentit_PR-16-XBYT654LGQHOZLTISQHSND3OR52VOZRVTZYYQSSUQPHLAN2AU6IQ\obj\docfx.log.xml
      
      joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi
      
      Found 3 violations:
      
      joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi
      
      DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) mobiilirajapinta/json-nimeämiskäytäntö.md 7 -> 7
      
      joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi
      
      DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) sanasto.md 63 -> 63
      
      joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi
      
      DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) mobiilirajapinta/json-dateandtime.md 18 -> 18
      
      joulukuuta 20, 2017 12:58:13 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator 
      
      3 violations.
      
      joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments
      
      Asking BitbucketServerCommentsProvider to comment:
      
      joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments
      
      DocFX INFO InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save) ChangedFile [filename=sanasto.md, specifics=[]] 63 
      
      joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments
      
      DocFX INFO InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save) ChangedFile [filename=mobiilirajapinta/json-dateandtime.md, specifics=[]] 18 
      

      That is, the plugin did not realize that the pull request included a change to mobiilirajapinta/json-nimeämiskäytäntö.md, so it did not report any violations in that file. However, the web UI of Bitbucket Server shows that the pull request includes such a change, and so does the REST API.

      I'm guessing the problem is in BitbucketServerInvoker.invokeUrl:

            reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
      

      This calls public InputStreamReader(InputStream in), which uses the default charset of Java, likely Windows-1252 on Windows; but the JSON HTTP response from the server seems to be always UTF-8.

      I'm filing this under the violation-comments-to-github-plugin component because there is no violation-comments-to-stash-plugin component and both plugins have the same author.

        Attachments

          Activity

          Hide
          kon Kalle Niemitalo added a comment -

          I tried to work around this by mangling the file names in the violations log file before the plugin parses it: convert the strings to byte arrays as UTF-8, convert back to strings as Windows-1252, and write to the log file as UTF-8 again. This made the plugin generate a comment about mobiilirajapinta/json-nimeämiskäytäntö.md too, thereby proving that the problem is caused by the encoding. However, this is not an acceptable workaround because it causes the plugin to send the wrongly encoded file name to Bitbucket Server when it is publishing the comment. Bitbucket Server then displays the comment but does not display any context from the diff.

          Show
          kon Kalle Niemitalo added a comment - I tried to work around this by mangling the file names in the violations log file before the plugin parses it: convert the strings to byte arrays as UTF-8, convert back to strings as Windows-1252, and write to the log file as UTF-8 again. This made the plugin generate a comment about mobiilirajapinta/json-nimeämiskäytäntö.md too, thereby proving that the problem is caused by the encoding. However, this is not an acceptable workaround because it causes the plugin to send the wrongly encoded file name to Bitbucket Server when it is publishing the comment. Bitbucket Server then displays the comment but does not display any context from the diff.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Tomas Bjerre
          Path:
          CHANGELOG.md
          pom.xml
          http://jenkins-ci.org/commit/violation-comments-to-stash-plugin/e64417e9dc0123363c44f9278f9dc762b1ac23db
          Log:
          Using UTF-8 in Bitbucket client JENKINS-48648

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Tomas Bjerre Path: CHANGELOG.md pom.xml http://jenkins-ci.org/commit/violation-comments-to-stash-plugin/e64417e9dc0123363c44f9278f9dc762b1ac23db Log: Using UTF-8 in Bitbucket client JENKINS-48648
          Hide
          tomasbjerre Tomas Bjerre added a comment -

          Hopefully this change fixed it:

          https://github.com/tomasbjerre/violation-comments-to-bitbucket-server-lib/commit/3fd97cc3246284bcb3b4dddd5833143ac59cd9c4

           

          I am releasing that now in 1.59. Open issue again if not working!

          Show
          tomasbjerre Tomas Bjerre added a comment - Hopefully this change fixed it: https://github.com/tomasbjerre/violation-comments-to-bitbucket-server-lib/commit/3fd97cc3246284bcb3b4dddd5833143ac59cd9c4   I am releasing that now in 1.59. Open issue again if not working!
          Hide
          kon Kalle Niemitalo added a comment -

          Thank you for fixing it. Violation Comments to Bitbucket Server Plugin 1.59 works OK.

          Show
          kon Kalle Niemitalo added a comment - Thank you for fixing it. Violation Comments to Bitbucket Server Plugin 1.59 works OK.

            People

            • Assignee:
              tomasbjerre Tomas Bjerre
              Reporter:
              kon Kalle Niemitalo
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: