Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45076

Support Declarative Pipeline method to return the file path (similar to tool() and credential() methods)

    Details

    • Similar Issues:

      Description

      The scripted syntax step configFileProvider is workable, but it would be nice to directly set env variables in the declarative environment{} section or in the withEnv() step. 

      environment {
        FOO_CONFIG_FILE = configFile('default-foo-settings')
      }

      and

      withEnv(["FOO_CONFIG_FILE=${configFile 'default-foo-settings'}"]) {
        ...
      }

       

        Attachments

          Activity

          Hide
          imod Dominik Bartholdi added a comment -

          This is a really cool idea, but I'm afraid its probably not possible to implement or at least I have no idea how. The issues is, that the location for temp files can be different on each slave and can only be detected once the job is running on the respective node. The environment declaration in pipeline on the other hand is read/executed a lot earlier and therefore has no access to the node.

           

          Andrew Bayer do you agree or do you have an idea how to define an environment variable that needs some information like a the tmp file location of a node?

          Show
          imod Dominik Bartholdi added a comment - This is a really cool idea, but I'm afraid its probably not possible to implement or at least I have no idea how. The issues is, that the location for temp files can be different on each slave and can only be detected once the job is running on the respective node. The environment declaration in pipeline on the other hand is read/executed a lot earlier and therefore has no access to the node.   Andrew Bayer do you agree or do you have an idea how to define an environment variable that needs some information like a the tmp file location of a node?
          Hide
          abayer Andrew Bayer added a comment -

          Hmmm...lemme think on this. I'm not a huge fan of how we handled credentials - it's hardcoded into Declarative, so adding something like this isn't as smooth as it should be. There should really be an extension point for contributing methods like this. That said, the withEnv block is evaluated after entering the node block (both at the top-level and per-stage), so this should be theoretically possible.

          Actually, it might even be possible to just call a step to set the environment variable - now that I think about it, credentials is weird because it's actually got to translate on the backend into a withCredentials block, not a simple FOO=returnFromStep() declaration. So if there was a step that did the config file provider magic and returned the path, that should work...

          Show
          abayer Andrew Bayer added a comment - Hmmm...lemme think on this. I'm not a huge fan of how we handled credentials - it's hardcoded into Declarative, so adding something like this isn't as smooth as it should be. There should really be an extension point for contributing methods like this. That said, the withEnv block is evaluated after entering the node block (both at the top-level and per-stage), so this should be theoretically possible. Actually, it might even be possible to just call a step to set the environment variable - now that I think about it, credentials is weird because it's actually got to translate on the backend into a withCredentials block, not a simple FOO=returnFromStep() declaration. So if there was a step that did the config file provider magic and returned the path, that should work...
          Hide
          imod Dominik Bartholdi added a comment - - edited

          cool!

          So you'r saying that if I would create a step like configFile('default-foo-settings') that returns FOO, then this would already work? or is it just an idea on how you could improve declarative pipeline?

          But even if this would already work, how could I clean up the tmp files after the build? They do potentially contain stuff that should not stay around... and just letting the user taking care of it is just not a good idea - I think. 

           

          personally, I would very much love prefere the first idea (without the withEnv)...

          Show
          imod Dominik Bartholdi added a comment - - edited cool! So you'r saying that if I would create a step like configFile('default-foo-settings') that returns FOO , then this would already work? or is it just an idea on how you could improve declarative pipeline? But even if this would already work, how could I clean up the tmp files after the build? They do potentially contain stuff that should not stay around... and just letting the user taking care of it is just not a good idea - I think.    personally, I would very much love prefere the first idea (without the withEnv )...
          Hide
          abayer Andrew Bayer added a comment -

          It should work. Especially once https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/174 lands.

          But yeah, cleanup is a good point. It probably still makes more sense to be a block, meaning I need to make that an extension point. =)

          Show
          abayer Andrew Bayer added a comment - It should work. Especially once https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/174 lands. But yeah, cleanup is a good point. It probably still makes more sense to be a block, meaning I need to make that an extension point. =)
          Hide
          abayer Andrew Bayer added a comment -

          Hrm, for the first time, I'm wishing that options allowed wrapper steps that required a FilePath. That'd solve this without needing an extension point for use in environment...let me think on that. The wrapper steps now are executed inside the top-level node block, so it could theoretically work, though I'd want to add validation logic to make sure you didn't have agent none at the top-level if you had a FilePath-requiring wrapper step...

          Show
          abayer Andrew Bayer added a comment - Hrm, for the first time, I'm wishing that options allowed wrapper steps that required a FilePath . That'd solve this without needing an extension point for use in environment ...let me think on that. The wrapper steps now are executed inside the top-level node block, so it could theoretically work, though I'd want to add validation logic to make sure you didn't have agent none at the top-level if you had a FilePath -requiring wrapper step...
          Hide
          jg_lgc Justin Georgeson added a comment -

          In my experience so far, both use cases (environment{} and withEnv()) are necessary in order to support parallel steps. The declarative environment {} in the stage will only resolve a single node-local path as you pointed out, and the parallel node steps are declared with scripted syntax, so withEnv is required to get the node-local paths inside the parallel node steps.

          Show
          jg_lgc Justin Georgeson added a comment - In my experience so far, both use cases ( environment{} and  withEnv() ) are necessary in order to support parallel steps. The declarative environment {}  in the stage will only resolve a single node-local path as you pointed out, and the parallel  node steps are declared with scripted syntax, so  withEnv is required to get the node-local paths inside the parallel  node steps.

            People

            • Assignee:
              domi Dominik Bartholdi
              Reporter:
              jg_lgc Justin Georgeson
            • Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated: