Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13466

Custom environment variables not available when build started by an SCM change

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Postponed
    • Component/s: core, envinject-plugin
    • Labels:
      None
    • Environment:
      not applicable

      Description

      Hi there,

      I may have found a sneaky bug. Custom environment variables can be defined by the user into the general & nodes config:

      When a build is triggered by an SCM change, those variables are not available for SCM plugins.

      Let's say that I have a field in my SCM plugin configured with this value: ${JOB_NAME}_${HOSTNAME}_${NODE_TYPE} If the build is triggered by an SCM change I get this error:

      Started by an SCM change
      Building remotely on vmo426
      [ClearCase] ### Begin source code retrieval ###
      cleartool error: Illegal characters found in view name : ${NODE_TYPE}. An environment variable may have not been resolved.
      Finished: FAILURE
      

      Here is the code snipet from my plugin that does that:

      @Override
      public boolean checkout(AbstractBuild build, Launcher launcher, FilePath workspace, 
              BuildListener listener, File changelogFile) throws IOException, InterruptedException
      {
          try {          
              logger.log("### Begin source code retrieval ###");
      
              String resolvedViewName = build.getEnvironment(listener).expand(this.viewName);
              
              Pattern pattern = Pattern.compile("(\\$\\{.+?\\})");
              Matcher matcher = pattern.matcher(resolvedViewName);
              if (matcher.find()) {
                  String message = "Illegal characters found in view name : %s. " +
                          "An environment variable may have not been resolved.";
                  throw new ClearToolError(String.format(message, matcher.group()));
              }
      

      I removed ${NODE_TYPE} from my SCM configuration to let the build continue. And the user custom environment variables seem to be injected into the build environment after the SCM checkout.

      Started by an SCM change
      Building remotely on vmo426
      [ClearCase] ### Begin source code retrieval ###
      ...
      [ClearCase] === End source code retrieval ===
      [polling_linux] $ /bin/sh -xe /tmp/hudson7589128551511062241.sh
      + env
      ...
      NODE_TYPE=SLAVE
      ...
      

      Could this be fixed?

        Activity

        Hide
        gbois Gregory Boissinot added a comment -

        I think environment variables defined in node properties are not available in AbstractBuild.getEnvironment(TaskListener) method.
        I think it doesn't work in your use case because you have maybe created your own launcher.
        Therefore, the following code is not available

        for (NodeProperty nodeProperty: Hudson.getInstance().getGlobalNodeProperties()) {
        Environment environment = nodeProperty.setUp(AbstractBuild.this, l, listener);
        if (environment != null)

        { buildEnvironments.add(environment); }
        }

        for (NodeProperty nodeProperty: Computer.currentComputer().getNode().getNodeProperties()) {
        Environment environment = nodeProperty.setUp(AbstractBuild.this, l, listener);
        if (environment != null) { buildEnvironments.add(environment); }

        }

        located in AbstractBuild#createLauncher() method

        A better alternative is to use the great EnvInject plugin
        With it, in your SCM code, just get the EnvInjectAction and its captured environment variables (used in my forked SCM clearcase plugin).

        Show
        gbois Gregory Boissinot added a comment - I think environment variables defined in node properties are not available in AbstractBuild.getEnvironment(TaskListener) method. I think it doesn't work in your use case because you have maybe created your own launcher. Therefore, the following code is not available for (NodeProperty nodeProperty: Hudson.getInstance().getGlobalNodeProperties()) { Environment environment = nodeProperty.setUp(AbstractBuild.this, l, listener); if (environment != null) { buildEnvironments.add(environment); } } for (NodeProperty nodeProperty: Computer.currentComputer().getNode().getNodeProperties()) { Environment environment = nodeProperty.setUp(AbstractBuild.this, l, listener); if (environment != null) { buildEnvironments.add(environment); } } located in AbstractBuild#createLauncher() method A better alternative is to use the great EnvInject plugin With it, in your SCM code, just get the EnvInjectAction and its captured environment variables (used in my forked SCM clearcase plugin).
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Gregory Boissinot
        Path:
        src/main/java/org/jenkinsci/lib/envinject/service/EnvVarsResolver.java
        http://jenkins-ci.org/commit/envinject-lib/dfd27a41e20b9694c50511d001c1b171f477946e
        Log:
        Add support of JENKINS-13466

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Gregory Boissinot Path: src/main/java/org/jenkinsci/lib/envinject/service/EnvVarsResolver.java http://jenkins-ci.org/commit/envinject-lib/dfd27a41e20b9694c50511d001c1b171f477946e Log: Add support of JENKINS-13466
        Hide
        gbois Gregory Boissinot added a comment -

        The issue is not a bug but a design issue.
        EnvironmentVariablesNodeProperty variables are computed at the launcher creation and it is not possible to compute them in another way.

        At the moment, my proposal solution is to add a dependency to envinject-lib and use
        EnvVarsResolver#resolveEnVars(build, this.viewName)

        This method uses the variables injected by the EnvInject plugin if it is installed. If the plugin is not installed, all environment variables are computed (and EnvironmentVariablesNodeProperty are used).

        The only incovenience is to tie your code with this library.
        Hopefully, this solution should be suitable for you.

        Show
        gbois Gregory Boissinot added a comment - The issue is not a bug but a design issue. EnvironmentVariablesNodeProperty variables are computed at the launcher creation and it is not possible to compute them in another way. At the moment, my proposal solution is to add a dependency to envinject-lib and use EnvVarsResolver#resolveEnVars(build, this.viewName) This method uses the variables injected by the EnvInject plugin if it is installed. If the plugin is not installed, all environment variables are computed (and EnvironmentVariablesNodeProperty are used). The only incovenience is to tie your code with this library. Hopefully, this solution should be suitable for you.
        Hide
        gbois Gregory Boissinot added a comment -

        Additionally, if you just install the envinject-plugin and active it, it has to work.

        Show
        gbois Gregory Boissinot added a comment - Additionally, if you just install the envinject-plugin and active it, it has to work.
        Hide
        robinjarry Robin Jarry added a comment -

        So if I add a dependency to envinject-lib and change my code it will be enough?

        People will not have to install/use the EnvInject plugin to make it work? Most of them will have it installed (as it is a kick ass plugin, but this is to make sure )

        Robin

        Show
        robinjarry Robin Jarry added a comment - So if I add a dependency to envinject-lib and change my code it will be enough? People will not have to install/use the EnvInject plugin to make it work? Most of them will have it installed (as it is a kick ass plugin, but this is to make sure ) Robin
        Hide
        gbois Gregory Boissinot added a comment -

        As I said, it is an alternative solution, waiting for a Jenkins core refactoring.
        The problem is a design Jenkins core issue.
        For summary, there are two temporary solutions:
        1) Install the EnvInject and use it, all environment variables will be populated.
        2) Or depend on the envinject-lib library, modify your code to use EnvVarsResolver#resolveEnVars(build, this.viewName)
        This latest solution enables you to not depend of the EnvInject plugin (installed or not, activated or not).
        The only drawback is to have an additional dependency. It is due to a Jenkins core problem.

        Show
        gbois Gregory Boissinot added a comment - As I said, it is an alternative solution, waiting for a Jenkins core refactoring. The problem is a design Jenkins core issue. For summary, there are two temporary solutions: 1) Install the EnvInject and use it, all environment variables will be populated. 2) Or depend on the envinject-lib library, modify your code to use EnvVarsResolver#resolveEnVars(build, this.viewName) This latest solution enables you to not depend of the EnvInject plugin (installed or not, activated or not). The only drawback is to have an additional dependency. It is due to a Jenkins core problem.
        Hide
        gbois Gregory Boissinot added a comment - - edited

        Any suggestions?

        Show
        gbois Gregory Boissinot added a comment - - edited Any suggestions?

          People

          • Assignee:
            gbois Gregory Boissinot
            Reporter:
            robinjarry Robin Jarry
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: