Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-16716

FILE token does not evaluate a path that contains an environment variable

    Details

    • Similar Issues:

      Description

      When I try the following in the Default Content of the Editable E-mail Notification:

      ${FILE,path="$MY_FILE_PATH"}

      The contents of the file specified by the path do not get displayed. I have MY_FILE_PATH defined as a property in a properties file that is injected into the environment in the "Prepare an environment for the run" option where the full path of the property file is specified.

      The property file defines: MY_FILE_PATH=custom_info.txt

      where "custom_info.txt" is a file in the workspace of the job that contains some text that I would like to display as part of the e-mail.

      What is strange is that when I receive the e-mail, this appears:

      ${FILE,path="custom_info.txt"}

      So the email-ext plugin evaluates the environment variable MY_FILE_PATH, but the FILE token does not evaluate it as a path.

      The ability to have the FILE token evaluate a path that contains an environment variable would be useful for displaying different information depending on which file path is stored in the environment variable.

      I am running Jenkins 1.480.2.

        Attachments

          Activity

          Hide
          vwagner Valerie Wagner added a comment -

          Does the $FILE token work at all? I'm trying to use it with literal paths and I get the same result as you once your environment variable has been substituted. Perhaps it's not the variable replacement that is broken but the file inclusion mechanism itself.

          If I put in something like:

          ${FILE,path="custom_info.txt"}

          That literal string appears in my email. I've tried all sorts of combinations of relative & absolute paths and I can't get any other result.

          Show
          vwagner Valerie Wagner added a comment - Does the $FILE token work at all? I'm trying to use it with literal paths and I get the same result as you once your environment variable has been substituted. Perhaps it's not the variable replacement that is broken but the file inclusion mechanism itself. If I put in something like: ${FILE,path="custom_info.txt"} That literal string appears in my email. I've tried all sorts of combinations of relative & absolute paths and I can't get any other result.
          Hide
          slide_o_mix Alex Earl added a comment -

          It only works with workspace relative paths. Also, you need a space between the , and the path.

          Show
          slide_o_mix Alex Earl added a comment - It only works with workspace relative paths. Also, you need a space between the , and the path.
          Hide
          vwagner Valerie Wagner added a comment -

          I've tried it with & without a space. And with relative paths. Neither works for me.

          Show
          vwagner Valerie Wagner added a comment - I've tried it with & without a space. And with relative paths. Neither works for me.
          Hide
          slide_o_mix Alex Earl added a comment -

          Hmmm, I'll give it a shot locally.

          Show
          slide_o_mix Alex Earl added a comment - Hmmm, I'll give it a shot locally.
          Hide
          vwagner Valerie Wagner added a comment -

          I don't want to hijack this issue if mine is different. But perhaps adding a space above will solve kavita's original problem. As far as I can tell, this is exactly what my syntax should be:

          ${FILE, path="sw/java/libs/testutils/build/testng-changes.html"}

          I've triple checked that the path is right & the file exists. Incidentally, this now gives me an exception:

          ERROR: Could not send email as a part of the post-build publishers.
          java.lang.NullPointerException
          at java.util.regex.Matcher.quoteReplacement(Matcher.java:655)
          at hudson.plugins.emailext.plugins.ContentBuilder$Tokenizer.appendReplacement(ContentBuilder.java:211)
          at hudson.plugins.emailext.plugins.ContentBuilder.replaceTokensWithContent(ContentBuilder.java:112)
          at hudson.plugins.emailext.plugins.ContentBuilder.transformText(ContentBuilder.java:82)
          at hudson.plugins.emailext.ExtendedEmailPublisher.getContent(ExtendedEmailPublisher.java:482)
          at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:314)
          at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:259)
          at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:251)
          at hudson.plugins.emailext.ExtendedEmailPublisher.perform(ExtendedEmailPublisher.java:211)
          at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
          at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:810)
          at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:785)
          at hudson.model.Build$BuildExecution.cleanUp(Build.java:192)
          at hudson.model.Run.execute(Run.java:1601)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:236)

          Perhaps I should open a separate issue?

          Thanks for your help!

          Show
          vwagner Valerie Wagner added a comment - I don't want to hijack this issue if mine is different. But perhaps adding a space above will solve kavita's original problem. As far as I can tell, this is exactly what my syntax should be: ${FILE, path="sw/java/libs/testutils/build/testng-changes.html"} I've triple checked that the path is right & the file exists. Incidentally, this now gives me an exception: ERROR: Could not send email as a part of the post-build publishers. java.lang.NullPointerException at java.util.regex.Matcher.quoteReplacement(Matcher.java:655) at hudson.plugins.emailext.plugins.ContentBuilder$Tokenizer.appendReplacement(ContentBuilder.java:211) at hudson.plugins.emailext.plugins.ContentBuilder.replaceTokensWithContent(ContentBuilder.java:112) at hudson.plugins.emailext.plugins.ContentBuilder.transformText(ContentBuilder.java:82) at hudson.plugins.emailext.ExtendedEmailPublisher.getContent(ExtendedEmailPublisher.java:482) at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:314) at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:259) at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:251) at hudson.plugins.emailext.ExtendedEmailPublisher.perform(ExtendedEmailPublisher.java:211) at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:810) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:785) at hudson.model.Build$BuildExecution.cleanUp(Build.java:192) at hudson.model.Run.execute(Run.java:1601) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236) Perhaps I should open a separate issue? Thanks for your help!
          Hide
          slide_o_mix Alex Earl added a comment -

          That file exists relative to the workspace root? Also, what version of the plugin are you using? There was a version that threw an exception like you are seeing if the file didn't exist. I fixed that, but I don't remember what version it was.

          Show
          slide_o_mix Alex Earl added a comment - That file exists relative to the workspace root? Also, what version of the plugin are you using? There was a version that threw an exception like you are seeing if the file didn't exist. I fixed that, but I don't remember what version it was.
          Hide
          vwagner Valerie Wagner added a comment -

          Yes, the file definitely exists. I'm using 2.21. Actually I thought I upgraded to 2.25 earlier today in an attempt to see if it would fix it, but perhaps I need to restart Jenkins to get that to take effect. I'll give it a try.

          Show
          vwagner Valerie Wagner added a comment - Yes, the file definitely exists. I'm using 2.21. Actually I thought I upgraded to 2.25 earlier today in an attempt to see if it would fix it, but perhaps I need to restart Jenkins to get that to take effect. I'll give it a try.
          Hide
          slide_o_mix Alex Earl added a comment -

          Yes, when you upgrade plugins, you need to restart Jenkins. 2.21 would definitely throw the exception for a file it couldn't find.

          Show
          slide_o_mix Alex Earl added a comment - Yes, when you upgrade plugins, you need to restart Jenkins. 2.21 would definitely throw the exception for a file it couldn't find.
          Hide
          slide_o_mix Alex Earl added a comment -

          There is a test case in place for the build for this token, so I know the token itself works.

          Show
          slide_o_mix Alex Earl added a comment - There is a test case in place for the build for this token, so I know the token itself works.
          Hide
          vwagner Valerie Wagner added a comment -

          Restarting Jenkins seems to have done the trick! That and getting the right combination of spaces and double-quotes. thanks for all your help!

          Show
          vwagner Valerie Wagner added a comment - Restarting Jenkins seems to have done the trick! That and getting the right combination of spaces and double-quotes. thanks for all your help!
          Hide
          trincodog Kavita Tiwari added a comment -

          I think Valerie's issue was different from mine as the FILE token itself works fine for me.

          What does not work is the evaluation of an environment variable in the "path" for the FILE token. I am running email-ext - 2.24.1.

          Show
          trincodog Kavita Tiwari added a comment - I think Valerie's issue was different from mine as the FILE token itself works fine for me. What does not work is the evaluation of an environment variable in the "path" for the FILE token. I am running email-ext - 2.24.1.
          Hide
          slide_o_mix Alex Earl added a comment -

          Kavita, I agree. I need to determine how best to do what you want. One workaround I might propose for now would be to generate a file with the ${FILE, path="real_path_here"} when you know the env variable works to specify the path to the file. Then use that generated file in the email template, the FILE token allows for nested content, so it should read the file, then replace the FILE token inside with the content you are looking for. Let me know if you need more information.

          Show
          slide_o_mix Alex Earl added a comment - Kavita, I agree. I need to determine how best to do what you want. One workaround I might propose for now would be to generate a file with the ${FILE, path="real_path_here"} when you know the env variable works to specify the path to the file. Then use that generated file in the email template, the FILE token allows for nested content, so it should read the file, then replace the FILE token inside with the content you are looking for. Let me know if you need more information.
          Hide
          trincodog Kavita Tiwari added a comment -

          Alex, thanks for the suggestion. I tried out the workaround and you are right that the FILE token allows for nested content. However, when the nested content is a FILE token that uses an environment variable in the "path", it does not evaluate the environment variable. So the same problem occurs. Nesting does not rectify the problem. I think the issue is inherent in the FILE token interpretation itself.

          Show
          trincodog Kavita Tiwari added a comment - Alex, thanks for the suggestion. I tried out the workaround and you are right that the FILE token allows for nested content. However, when the nested content is a FILE token that uses an environment variable in the "path", it does not evaluate the environment variable. So the same problem occurs. Nesting does not rectify the problem. I think the issue is inherent in the FILE token interpretation itself.
          Hide
          slide_o_mix Alex Earl added a comment -

          That's not what I was trying to recommend, let me try and rephrase.

          1) Create a build step that creates a file in the workspace like this:

          echo ${FILE, path="%PATH_TO_FILE%"} > staticname.txt

          2) Use the following as the token in the email

          ${FILE, path="staticname.txt"}

          The staticname.txt does not use a variable since you know the name of the file you are creating.

          Show
          slide_o_mix Alex Earl added a comment - That's not what I was trying to recommend, let me try and rephrase. 1) Create a build step that creates a file in the workspace like this: echo ${FILE, path="%PATH_TO_FILE%"} > staticname.txt 2) Use the following as the token in the email ${FILE, path="staticname.txt"} The staticname.txt does not use a variable since you know the name of the file you are creating.
          Hide
          trincodog Kavita Tiwari added a comment - - edited

          Hi Alex. I tried Step 1) that you've listed above but I get a "bad substitution" shell error and the build is marked as failed as a result. I don't think I can use the FILE token outside of the editable e-mail notification, should I be able to? Here is my syntax:

          echo ${FILE, path="$MY_FILE_PATH"} > staticname.txt

          where "MY_FILE_PATH" is an environment variable that I've specified in a property file that I've defined as part of "Prepare an environment for the run".

          Show
          trincodog Kavita Tiwari added a comment - - edited Hi Alex. I tried Step 1) that you've listed above but I get a "bad substitution" shell error and the build is marked as failed as a result. I don't think I can use the FILE token outside of the editable e-mail notification, should I be able to? Here is my syntax: echo ${FILE, path="$MY_FILE_PATH"} > staticname.txt where "MY_FILE_PATH" is an environment variable that I've specified in a property file that I've defined as part of "Prepare an environment for the run".
          Hide
          slide_o_mix Alex Earl added a comment -

          Depending on what OS you are running on, you may need to escape the writing of the token to the file. I don't what the token to be replaced, I just want you to write it out to the file when you know the path to the file you want included in the email. The idea is to write the content you want ${FILE, path="custom_info.txt"}, when custom_info.txt is in your MY_FILE_PATH variable. You would want to escape the first $ in the token while writing it out to the file.

          Build step:
          echo \${FILE, path="$MY_FILE_PATH"} > staticname.txt

          Email content:
          ${FILE, path="staticname.txt"}

          Show
          slide_o_mix Alex Earl added a comment - Depending on what OS you are running on, you may need to escape the writing of the token to the file. I don't what the token to be replaced, I just want you to write it out to the file when you know the path to the file you want included in the email. The idea is to write the content you want ${FILE, path="custom_info.txt"}, when custom_info.txt is in your MY_FILE_PATH variable. You would want to escape the first $ in the token while writing it out to the file. Build step: echo \${FILE, path="$MY_FILE_PATH"} > staticname.txt Email content: ${FILE, path="staticname.txt"}
          Hide
          trincodog Kavita Tiwari added a comment - - edited

          Ah I see. When I escaped the first $ in the token, this is what gets displayed in the e-mail:

          ${FILE, path=custom_info.txt}

          So even though MY_FILE_PATH is properly evaluated to be "custom_info.txt", it looks like the nested functionality isn't working for me because it isn't evaluating "${FILE, path=custom_info.txt}" to display the contents of custom_info.txt.

          The contents of staticname.txt is also:

          ${FILE, path=custom_info.txt}

          Show
          trincodog Kavita Tiwari added a comment - - edited Ah I see. When I escaped the first $ in the token, this is what gets displayed in the e-mail: ${FILE, path=custom_info.txt} So even though MY_FILE_PATH is properly evaluated to be "custom_info.txt", it looks like the nested functionality isn't working for me because it isn't evaluating "${FILE, path=custom_info.txt}" to display the contents of custom_info.txt. The contents of staticname.txt is also: ${FILE, path=custom_info.txt}
          Hide
          slide_o_mix Alex Earl added a comment -

          You may need more escaping for the ", play around with it.

          Show
          slide_o_mix Alex Earl added a comment - You may need more escaping for the ", play around with it.
          Hide
          trincodog Kavita Tiwari added a comment -

          Success! Thanks for the workaround Alex! I will use it . Here's the build step that worked for me (I'm running on Linux with the default shell):

          echo "\${FILE,path=\"$MY_FILE_PATH\"}" > staticname.txt

          I still think it would be nice to have the FILE token evaluate the environment variable specified in its "path" itself as that would avoid the extra build step, but for now, this workaround does what I want it to do.

          Thanks again for your quick response and help!

          Show
          trincodog Kavita Tiwari added a comment - Success! Thanks for the workaround Alex! I will use it . Here's the build step that worked for me (I'm running on Linux with the default shell): echo "\${FILE,path=\"$MY_FILE_PATH\"}" > staticname.txt I still think it would be nice to have the FILE token evaluate the environment variable specified in its "path" itself as that would avoid the extra build step, but for now, this workaround does what I want it to do. Thanks again for your quick response and help!
          Hide
          slide_o_mix Alex Earl added a comment -

          I'll leave this open and see if its something that I can add. It becomes less of a priority with the workaround, but I can see where it would be useful.

          Show
          slide_o_mix Alex Earl added a comment - I'll leave this open and see if its something that I can add. It becomes less of a priority with the workaround, but I can see where it would be useful.
          Hide
          trincodog Kavita Tiwari added a comment -

          Agreed. Actually in my implementation, I need unique files to be written into the workspace as I want my job to support concurrent builds. So having "staticname.txt" isn't ideal as it could get overwritten if two jobs are running at the same time. That's why I wanted to use an environment variable to specify the path in the FILE token where each instance of the running job would define its own unique value for the path in an environment variable. I will use this workaround for now until a fix is in place.

          Show
          trincodog Kavita Tiwari added a comment - Agreed. Actually in my implementation, I need unique files to be written into the workspace as I want my job to support concurrent builds. So having "staticname.txt" isn't ideal as it could get overwritten if two jobs are running at the same time. That's why I wanted to use an environment variable to specify the path in the FILE token where each instance of the running job would define its own unique value for the path in an environment variable. I will use this workaround for now until a fix is in place.
          Hide
          trincodog Kavita Tiwari added a comment -

          The PROPFILE token also does not evaluate an environment variable when specified as part of the "file" variable for that token. So if it gets fixed for the FILE token, it would also be useful to have in the PROPFILE token or any token that allows you to specify a file path (FILE and PROPFILE tokens are the two that come to mind).

          Show
          trincodog Kavita Tiwari added a comment - The PROPFILE token also does not evaluate an environment variable when specified as part of the "file" variable for that token. So if it gets fixed for the FILE token, it would also be useful to have in the PROPFILE token or any token that allows you to specify a file path (FILE and PROPFILE tokens are the two that come to mind).
          Hide
          slide_o_mix Alex Earl added a comment - - edited

          Propfile is not from email-ext, its from token-macro. Which means you would need to have someone fix it for that plugin too.

          Show
          slide_o_mix Alex Earl added a comment - - edited Propfile is not from email-ext, its from token-macro. Which means you would need to have someone fix it for that plugin too.
          Hide
          slide_o_mix Alex Earl added a comment - - edited

          Also, you would be able to do this with a groovy template much easier, especially in the next version where I make it easier to use content tokens in the groovy template. See https://issues.jenkins-ci.org/browse/JENKINS-16916

          Show
          slide_o_mix Alex Earl added a comment - - edited Also, you would be able to do this with a groovy template much easier, especially in the next version where I make it easier to use content tokens in the groovy template. See https://issues.jenkins-ci.org/browse/JENKINS-16916
          Hide
          trincodog Kavita Tiwari added a comment -

          Thanks for clarifying about PROPFILE. As I am no longer using PROPFILE, I'll hold off on raising a ticket with the token-macro plug-in for PROPFILE not evaluating a path that contain an environment variable.

          Yes you are right. I could probably do what I have described above in this ticket with a groovy template and have the groovy do the job of evaluating the environment variable. But at present I've only been using plain-text e-mails. Though I do like the suggestion of using a groovy template, and it would be a good workaround for this issue. Do you know when the next version of email-ext with your changes will be made available?

          Show
          trincodog Kavita Tiwari added a comment - Thanks for clarifying about PROPFILE. As I am no longer using PROPFILE, I'll hold off on raising a ticket with the token-macro plug-in for PROPFILE not evaluating a path that contain an environment variable. Yes you are right. I could probably do what I have described above in this ticket with a groovy template and have the groovy do the job of evaluating the environment variable. But at present I've only been using plain-text e-mails. Though I do like the suggestion of using a groovy template, and it would be a good workaround for this issue. Do you know when the next version of email-ext with your changes will be made available?
          Hide
          slide_o_mix Alex Earl added a comment -

          I haven't implemented anything in regards to this ticket, so I am not sure what version it will get into.

          Show
          slide_o_mix Alex Earl added a comment - I haven't implemented anything in regards to this ticket, so I am not sure what version it will get into.
          Hide
          trincodog Kavita Tiwari added a comment -

          I meant when the new version of email-ext with your fix for this other ticket https://issues.jenkins-ci.org/browse/JENKINS-16916 will be available to upgrade to?

          Show
          trincodog Kavita Tiwari added a comment - I meant when the new version of email-ext with your fix for this other ticket https://issues.jenkins-ci.org/browse/JENKINS-16916 will be available to upgrade to?
          Hide
          slide_o_mix Alex Earl added a comment -

          It's available now (2.27)

          Show
          slide_o_mix Alex Earl added a comment - It's available now (2.27)
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Alex Earl
          Path:
          pom.xml
          src/main/java/hudson/plugins/emailext/plugins/content/WorkspaceFileContent.java
          http://jenkins-ci.org/commit/email-ext-plugin/7b5441e68a00255b8a916015d53f87055ffad185
          Log:
          Fixed JENKINS-16716

          Added an expansion of the environment variables

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Alex Earl Path: pom.xml src/main/java/hudson/plugins/emailext/plugins/content/WorkspaceFileContent.java http://jenkins-ci.org/commit/email-ext-plugin/7b5441e68a00255b8a916015d53f87055ffad185 Log: Fixed JENKINS-16716 Added an expansion of the environment variables
          Hide
          slide_o_mix Alex Earl added a comment -

          Added expansion of path parameter.

          Show
          slide_o_mix Alex Earl added a comment - Added expansion of path parameter.
          Hide
          avirosenfeld Avi Rosenfeld added a comment -

          How do I use the "expansion of path parameter" ?

          I tried ${FILE, path="$MY_FILE_PATH"} , it still doesn't work.

          Show
          avirosenfeld Avi Rosenfeld added a comment - How do I use the "expansion of path parameter" ? I tried ${FILE, path="$MY_FILE_PATH"} , it still doesn't work.
          Hide
          slide_o_mix Alex Earl added a comment -

          I think this may have been removed because it was causing issues.

          Show
          slide_o_mix Alex Earl added a comment - I think this may have been removed because it was causing issues.
          Hide
          avirosenfeld Avi Rosenfeld added a comment -

          Thanks Alex for the update.

          So currently the only way to do it is the workaround of coping the $MY_FILE_PATH to a constant name file.
          Right?

          Show
          avirosenfeld Avi Rosenfeld added a comment - Thanks Alex for the update. So currently the only way to do it is the workaround of coping the $MY_FILE_PATH to a constant name file. Right?
          Hide
          slide_o_mix Alex Earl added a comment -

          It would also be easy to do via groovy template, which is my recommended method.

          Show
          slide_o_mix Alex Earl added a comment - It would also be easy to do via groovy template, which is my recommended method.

            People

            • Assignee:
              slide_o_mix Alex Earl
              Reporter:
              trincodog Kavita Tiwari
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: