Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62127

Dynamic Axis-Values for declarative Pipeline

    Details

    • Type: New Feature
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Labels:
      None
    • Environment:
    • Similar Issues:

      Description

      Hi there,

      I was asked to open a Ticket here, because it seems like the feature is interesting for many peoples.

      It would be nice to have the possibility to set the dynamically set the values of the "axis".

      Use-Case:
      I want to compile multiple plugins with the same Jenkins-pipeline. To do so I wrote a "JenkinsLibrary" with an interface the plugins can use for parameters. One of this parameters is the axis-value for compiler.

      • e.g. I want to build Plugin "X" for gcc7 on armv8 and Plugin Y for vc15 x64.
        So I have a Jenkins-Library with the logic of the build parameters.
      • And each plugin calls the library with the build-parameters. (e.g. compiler and arch )

      Example:
      a "JenkinsLibrary" with a file my_library.groovy

      def call(Map i_options)
      {
          def axis_1_values = "axis_1_value_A"
          // later: def axis_1_values = i_options.get("axis_1_values")
          pipeline
          {
              agent any
              stages { stage("stage A") { matrix {
                  axes {
                      axis {
                          name "axis_1"
                          values "${axis_1_values}" 
                      }
                      axis {
                          name "axis_2"
                          values "axis_2_value_A", "axis_2_value_B"
                      }
                  }
                  stages {
                      stage("another stage") { steps {
                          echo "hello world from ${axis_1} && ${axis_2}"
                      } }
                  }
              } } }
          }
      }
      

      and each plugin contains a Jenkinsfile like that:

      @Library("JenkinsLibrary") _
      
      def options = [
          axis_1_values: "a_axis_value"
      ]
      
      my_library(options)
      

      We want to keep the Jenkinsfiles of the plugins as short as possible to keep the maintenance as low as possible.

       

      If you run the pipeline above you get the error:

      [...] Expected string literal but got "${axis_1_values}"
      

      Complete log:

          19: Expected string literal but got "${axis_1_values}" @ line 19, column 28.
                             values "${axis_1_values}" 
                                    ^
      
      1 error
      
          at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
          at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
          at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
          at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
          at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
          at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
          at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
          at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
          at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:761)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:718)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
          at org.jenkinsci.plugins.workflow.cps.global.UserDefinedGlobalVariable.getValue(UserDefinedGlobalVariable.java:57)
          at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
          at sun.reflect.GeneratedMethodAccessor729.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
          at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
          at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
          at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
          at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
          at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
          at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
          at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:160)
          at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
          at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
          at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142)
          at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
          at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:162)
          at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
          at WorkflowScript.run(WorkflowScript:21)
          at ___cps.transform___(Native Method)
          at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
          at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
          at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
          at sun.reflect.GeneratedMethodAccessor500.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
          at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:39)
          at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
          at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:28)
          at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55)
          at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16)
          at com.cloudbees.groovy.cps.Next.step(Next.java:83)
          at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
          at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
          at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
          at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
          at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
          at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
          at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
          at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
          at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
          at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Finished: FAILURE
      

      I tried many ways for of setting the variables in the pipeline:

      // already tried the following alternatives ...
      //    def axis_1_values = 'axis_1_value_A'
      //    def axis_1_values = '''axis_1_value_A'''
      //    def axis_1_values = ["axis_1_value_A", "axis_2_value_A"]
          pipeline
          {
      [...]
      
      [...]
      axis {
          name "axis_1"
          // values "axis_1_value_A" // <- of course this works ...
          // but I want to read it from a variable
          values "${axis_1_values}" 
          // I already tried more variants
          // values ${axis_1_values}
          // values axis_1_values
      [...]
      }
      

      It would be nice if this will be possible.

        Attachments

          Activity

          Hide
          conf Alexey Shein added a comment -

          I've got bitten by this as well. In my use case, I just want to parallelize my tests to N downstream jobs, I use the following Jenkinsfile:

          // arrayList of stringified indices, like ["0", "1", "2", ...]
          def ciIndices(ciNodesTotal) {
            (0..ciNodesTotal-1).toArray().collect { it.toString() }
          }
          
          def ciNodesTotal = 3
          pipeline {
            agent any
          
            stages {
              stage('Test') {
                matrix {
                  axes {
                    axis {
                      name 'Worker'
                      values(*ciIndices(ciNodesTotal)) // doesn't work, throws Expected string literal but got ${*} @ line 32, column 20.
                    } //axis
                  } //axes
          
                  stages {
                    stage('Worker test') {
                      steps {
                        echo "Do build - ${Worker}"
                      } //steps
                    } //stage
                  } //stages
                } //matrix
              } //stage
            }
          }

          If I specify values literally, then everything is working great.

          Show
          conf Alexey Shein added a comment - I've got bitten by this as well. In my use case, I just want to parallelize my tests to N downstream jobs, I use the following Jenkinsfile: // arrayList of stringified indices, like [ "0" , "1" , "2" , ...] def ciIndices(ciNodesTotal) { (0..ciNodesTotal-1).toArray().collect { it.toString() } } def ciNodesTotal = 3 pipeline { agent any stages { stage( 'Test' ) { matrix { axes { axis { name 'Worker' values(*ciIndices(ciNodesTotal)) // doesn't work, throws Expected string literal but got ${*} @ line 32, column 20. } //axis } //axes stages { stage( 'Worker test' ) { steps { echo "Do build - ${Worker}" } //steps } //stage } //stages } //matrix } //stage } } If I specify values literally, then everything is working great.

            People

            • Assignee:
              bitwiseman Liam Newman
              Reporter:
              tharilya tharilya
            • Votes:
              17 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated: