Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48562

Inline Environment Variables not replaced in config file

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I am having an issue pulling in inline environment variables in a Pipeline script. I have a Freestyle Project I want to convert to pipeline.

      I am wanting to create a settings.php file for Drupal or Wordpress.

      Test file

      ${ENV, var="X_DB_USER"}
      ${ENV, var="X_SITENAME"}
      ${ENV, var="DOMAIN_NAME"}
      

      Pipeline:

       

       

      pipeline {
      environment {
      dbName = "test-${env.BUILD_NUMBER}"
      X_SITENAME = "${dbName}"
      X_DB_USER = "mp"
      DOMAIN_NAME = "https://www.example.com"
      }
      agent {
      node {
      label "ww7"
      }
      }
      stages {
      stage("init") {
      steps {
      configFileProvider(
      [
      configFile(
      fileId: 'drupal7-test', 
      replaceTokens: true, 
      targetLocation: "/var/www/${dbName}/www/local.settings.php")
      ]) {
      // some block
      }
      }
      }
      } 
      }
      
      
      

      File output

       

       

      <empty>

       

      If I add a global environment variable it will write to the file, but I am unable to overwrite the values.

       

      X_DB_USER global placeholder
      X_SITENAME global placeholder
      DOMAIN_NAME global placeholder
      

       

       

      Using the web interface to add a file using job parameters I am able to use a file like:

       

      <?php
      $$databases = array (
      'default' =>
      array (
      'default' =>
      array (
      'database' => '${DATABASE_NAME}',
      'username' => '${DATABASE_USER}',
      'password' => '${DATABASE_USER}',
      'host' => 'localhost',
      'port' => '',
      'driver' => 'mysql',
      'prefix' => '',
      // 'pdo' => array(ATTR_TIMEOUT => 2.0, MYSQL_ATTR_COMPRESS => 1, MYSQL_ATTR_USE_BUFFERED_QUERY => 1),
      ),
      ),
      );
      

       

      How can I use dynamic variables pulled from a Jenkinsfile?

       

        Attachments

          Issue Links

            Activity

            Hide
            imod Dominik Bartholdi added a comment -

            Hey Andrew Bayer I'm using TokenMacro to expand all variables, this works fine if the pipeline job is parameterized itself, but now it seems that variables provided by the declarative pipeline are not expanded by the TokenMacro, is this correct? what can I do?

             

            https://github.com/jenkinsci/config-file-provider-plugin/blob/master/src/main/java/org/jenkinsci/lib/configprovider/model/ConfigFileManager.java#L118 

            Show
            imod Dominik Bartholdi added a comment - Hey  Andrew Bayer I'm using  TokenMacro to expand all variables, this works fine if the pipeline job is parameterized itself, but now it seems that variables provided by the declarative pipeline are not expanded by the TokenMacro , is this correct? what can I do?   https://github.com/jenkinsci/config-file-provider-plugin/blob/master/src/main/java/org/jenkinsci/lib/configprovider/model/ConfigFileManager.java#L118  
            Hide
            abayer Andrew Bayer added a comment -

            First, give it a try with just a Scripted Pipeline with the variables in withEnv - Declarative's environment just maps to withEnv behind the scenes. And then if it turns out that doesn't work either, we need to figure that out. =)

            Show
            abayer Andrew Bayer added a comment - First, give it a try with just a Scripted Pipeline with the variables in withEnv - Declarative's environment just maps to withEnv behind the scenes. And then if it turns out that doesn't work either, we need to figure that out. =)
            Hide
            imod Dominik Bartholdi added a comment - - edited

            thanks Andrew Bayer!

            I now tried the following:

            node{
               withEnv(['dbName = "test-$\{env.BUILD_NUMBER}"', 'X_SITENAME = "HELLO"', 'X_DB_USER = "mp"', 'DOMAIN_NAME = "https://www.example.com"']) {
                  configFileProvider([configFile(fileId: 'settings_php', replaceTokens: true, targetLocation: "$\{dbName}/www/local.settings.php")]) {
                     sh "cat ${env.dbName}/www/local.settings.php"
                  }
               }
             }

             

            node{
               withEnv(['dbName = "test-$\{env.BUILD_NUMBER}"', 'X_SITENAME = "HELLO"', 'X_DB_USER = "mp"', 'DOMAIN_NAME = "https://www.example.com"']) {
                  configFileProvider([configFile(fileId: 'settings_php', replaceTokens: true, targetLocation: "$\{dbName}/www/local.settings.php")]) {
                     sh "cat ${dbName}/www/local.settings.php"
                  }
               }
             }

             

            ...unfortunate it did not work - it already fails at this:
            sh "cat ${env.dbName}/www/local.settings.php"
            because it can't find "dbName"

            tested on:

            • jenkins 2.73.4.1 (DEV@cloud)
            • pipeline 2.4
            Show
            imod Dominik Bartholdi added a comment - - edited thanks Andrew Bayer ! I now tried the following: node{ withEnv([ 'dbName = "test-$\{env.BUILD_NUMBER}" ' , 'X_SITENAME = "HELLO" ' , 'X_DB_USER = "mp" ' , 'DOMAIN_NAME = "https: //www.example.com" ' ]) { configFileProvider([configFile(fileId: 'settings_php' , replaceTokens: true , targetLocation: "$\{dbName}/www/local.settings.php" )]) { sh "cat ${env.dbName}/www/local.settings.php" } } }   node{ withEnv([ 'dbName = "test-$\{env.BUILD_NUMBER}" ' , 'X_SITENAME = "HELLO" ' , 'X_DB_USER = "mp" ' , 'DOMAIN_NAME = "https: //www.example.com" ' ]) { configFileProvider([configFile(fileId: 'settings_php' , replaceTokens: true , targetLocation: "$\{dbName}/www/local.settings.php" )]) { sh "cat ${dbName}/www/local.settings.php" } } }   ...unfortunate it did not work - it already fails at this: sh "cat ${env.dbName}/www/local.settings.php" because it can't find "dbName" tested on: jenkins 2.73.4.1 (DEV@cloud) pipeline 2.4
            Hide
            michaelpporter Michael Porter added a comment -

            I noticed your first example you are not calling `env.dbName` but `dbName` is that a type-o or does it not work still?

            node{
               withEnv(['dbName = "test-$\{env.BUILD_NUMBER}"', 'X_SITENAME = "HELLO"', 'X_DB_USER = "mp"', 'DOMAIN_NAME = "https://www.example.com"']) {
                  configFileProvider([configFile(fileId: 'settings_php', replaceTokens: true, targetLocation: "$\{dbName}/www/local.settings.php")]) {
                     sh "cat ${env.dbName}/www/local.settings.php"
                  }
               }
             }
            

            I have been calling a Job to build the file, but it would be nice to get away from this and us full pipeline.

                build job: 'global-builds/drupal-build-site-test',
                    parameters: [
                        string(name: 'SITE_NAME', value: "${siteName}"),
                        string(name: 'DATABASE_NAME', value: "${siteName.replaceAll('-','_')}"),
                        string(name: 'DATABASE_USER', value: "${dbUser}"),
                        string(name: 'DOMAIN_NAME', value: "${domain}"),
                        [$class: 'LabelParameterValue', name: 'node', label: "${buildLabel}"]]
            
            Show
            michaelpporter Michael Porter added a comment - I noticed your first example you are not calling `env.dbName` but `dbName` is that a type-o or does it not work still? node{ withEnv([ 'dbName = "test-$\{env.BUILD_NUMBER}" ' , 'X_SITENAME = "HELLO" ' , 'X_DB_USER = "mp" ' , 'DOMAIN_NAME = "https: //www.example.com" ' ]) { configFileProvider([configFile(fileId: 'settings_php' , replaceTokens: true , targetLocation: "$\{dbName}/www/local.settings.php" )]) { sh "cat ${env.dbName}/www/local.settings.php" } } } I have been calling a Job to build the file, but it would be nice to get away from this and us full pipeline. build job: 'global-builds/drupal-build-site-test' , parameters: [ string(name: 'SITE_NAME' , value: "${siteName}" ), string(name: 'DATABASE_NAME' , value: "${siteName.replaceAll( '-' , '_' )}" ), string(name: 'DATABASE_USER' , value: "${dbUser}" ), string(name: 'DOMAIN_NAME' , value: "${domain}" ), [$class: 'LabelParameterValue' , name: 'node' , label: "${buildLabel}" ]]
            Hide
            michaelpporter Michael Porter added a comment -

            A bit of a hack, but I found a pipeline workaround. by using `envsubst`

            node{
                configFileProvider([configFile(fileId: 'drupal7-test', 
                    replaceTokens: false, targetLocation: "web/settings.template.php")]) {
                     sh """cat web/settings.template.php
                     SITE_NAME=my-db-master DATABASE_NAME=my_db_master DATABASE_USER=my_db envsubst < web/settings.template.php > web/settings.local.php
                     cat web/settings.local.php
                     """
                  }
             }
            
            Show
            michaelpporter Michael Porter added a comment - A bit of a hack, but I found a pipeline workaround. by using `envsubst` node{ configFileProvider([configFile(fileId: 'drupal7-test' , replaceTokens: false , targetLocation: "web/settings.template.php" )]) { sh """cat web/settings.template.php SITE_NAME=my-db-master DATABASE_NAME=my_db_master DATABASE_USER=my_db envsubst < web/settings.template.php > web/settings.local.php cat web/settings.local.php """ } }
            Hide
            imod Dominik Bartholdi added a comment -

            resolved with 3.6.1

            Show
            imod Dominik Bartholdi added a comment - resolved with 3.6.1

              People

              • Assignee:
                domi Dominik Bartholdi
                Reporter:
                michaelpporter Michael Porter
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: