Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47528

MissingPropertyException from SandboxInterceptor during job-dsl-plugin job update

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: matrix-project-plugin
    • Labels:
    • Environment:
      Jenkins 2.46.3
      job-dsl-plugin 1.57
      Matrix (multi-configuration) Jenkins job being updated by job-dsl-plugin
    • Similar Issues:

      Description

      Consider a matrix job with an axis "axisOne" and and an axis "axisTwo", and a combination filter of axisOne.endsWith("bar").
      Now use the job-dsl-plugin to update that job so that it now has an axis "axisThree" and no combination filter at all.

      The result is a half-updated job with the original (unmodified) combination filter, and an exception (and failed job-ds-plugin build step) saying:

      groovy.lang.MissingPropertyException: No such property: axisOne for class: groovy.lang.Binding
      at groovy.lang.Binding.getVariable(Binding.java:63)
      at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:224)
      at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:241)
      at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:238)
      at org.kohsuke.groovy.sandbox.impl.Checker$checkedGetProperty.callStatic(Unknown Source)
      at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
      at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
      at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:230)
      at Script1.run(Script1.groovy:1)
      at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.run(GroovySandbox.java:141)
      at hudson.matrix.FilterScript.evaluate(FilterScript.java:45)
      at hudson.matrix.FilterScript.apply(FilterScript.java:85)
      at hudson.matrix.Combination.evalGroovyExpression(Combination.java:101)
      at hudson.matrix.Combination.evalGroovyExpression(Combination.java:91)
      at hudson.matrix.MatrixProject.rebuildConfigurations(MatrixProject.java:646)
      at hudson.matrix.MatrixProject.onLoad(MatrixProject.java:509)
      at hudson.model.AbstractItem$1.call(AbstractItem.java:674)
      at hudson.model.AbstractItem$1.call(AbstractItem.java:672)
      at hudson.model.Items.whileUpdatingByXml(Items.java:135)
      at hudson.model.AbstractItem.updateByXml(AbstractItem.java:672)
      at javaposse.jobdsl.plugin.JenkinsJobManagement.updateExistingItem(JenkinsJobManagement.java:497)
      at javaposse.jobdsl.plugin.JenkinsJobManagement.createOrUpdateConfig(JenkinsJobManagement.java:149)
      at javaposse.jobdsl.dsl.JobManagement$createOrUpdateConfig$5.call(Unknown Source)
      at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
      at javaposse.jobdsl.dsl.JobManagement$createOrUpdateConfig$5.call(Unknown Source)
      at javaposse.jobdsl.plugin.InterruptibleJobManagement.createOrUpdateConfig(InterruptibleJobManagement.groovy:37)
      at javaposse.jobdsl.dsl.JobManagement$createOrUpdateConfig$5.call(Unknown Source)
      at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
      at javaposse.jobdsl.dsl.JobManagement$createOrUpdateConfig$5.call(Unknown Source)
      at javaposse.jobdsl.dsl.DslScriptLoader$_extractGeneratedJobs_closure4.doCall(DslScriptLoader.groovy:183)
      at sun.reflect.GeneratedMethodAccessor20003.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
      at java.lang.reflect.Method.invoke(Method.java:620)
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
      at groovy.lang.Closure.call(Closure.java:414)
      at groovy.lang.Closure.call(Closure.java:430)
      at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)
      at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2015)
      at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2056)
      at org.codehaus.groovy.runtime.dgm$162.invoke(Unknown Source)
      at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
      at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
      at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
      at javaposse.jobdsl.dsl.DslScriptLoader.extractGeneratedJobs(DslScriptLoader.groovy:176)
      at javaposse.jobdsl.dsl.DslScriptLoader.this$2$extractGeneratedJobs(DslScriptLoader.groovy)
      at sun.reflect.GeneratedMethodAccessor21493.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
      at java.lang.reflect.Method.invoke(Method.java:620)
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384)
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
      at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
      at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:174)
      at javaposse.jobdsl.dsl.DslScriptLoader$_runScripts_closure1.doCall(DslScriptLoader.groovy:61)
      at sun.reflect.GeneratedMethodAccessor21276.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
      at java.lang.reflect.Method.invoke(Method.java:620)
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
      at groovy.lang.Closure.call(Closure.java:414)
      at groovy.lang.Closure.call(Closure.java:430)
      at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)
      at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2015)
      at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2068)
      at org.codehaus.groovy.runtime.dgm$164.invoke(Unknown Source)
      at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
      at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
      at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
      at javaposse.jobdsl.dsl.DslScriptLoader.runScripts(DslScriptLoader.groovy:42)
      at javaposse.jobdsl.plugin.ExecuteDslScripts.perform(ExecuteDslScripts.java:316)
      at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
      at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
      at hudson.model.Build$BuildExecution.build(Build.java:206)
      at hudson.model.Build$BuildExecution.doRun(Build.java:163)
      at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
      at hudson.model.Run.execute(Run.java:1728)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
      at hudson.model.ResourceController.execute(ResourceController.java:98)
      at hudson.model.Executor.run(Executor.java:405)

      What should happen instead is that the MatrixProject's validation code should be deferred until after the job-dsl-plugin has finished updating the job (which would have removed the combination filter).
      i.e. validation should be done on the final product, not on a mixture of the old and new job.

      (also, personally, I think that an invalid combination filter should merely be a warning - users can manually enter invalid filters so we should allow this via the job-dsl-plugin too)

      The workaround was to manually go in to the problem job (which took some effort to find as that DSL build step defined dozens of jobs) and remove the combination filter, "save" the new job and then re-run the Jenkins job that contained the DSL build step, at which point everything went green again.

        Attachments

          Issue Links

            Activity

            Hide
            daspilker Daniel Spilker added a comment -

            Post a minimal script that reproduces the problem.

            Show
            daspilker Daniel Spilker added a comment - Post a minimal script that reproduces the problem.
            Hide
            pjdarton pjdarton added a comment -

            OK. I've added a note to my taskboard (at work) to do this. Won't be this week (), but hopefully next...

            Show
            pjdarton pjdarton added a comment - OK. I've added a note to my taskboard (at work) to do this. Won't be this week (), but hopefully next...
            Hide
            pjdarton pjdarton added a comment -

            To reproduce, create a new Jenkins job with two "Process Job DSLs" build steps:

            1. For the first build step, tell it to "Use the provided DSL script" and give it the script
              matrixJob('Jenkins-47528-example') {
                axes {
                  text('axisOne','onefoo','onebar')
                  text('axisTwo','twofoo','twobar')
                }
                combinationFilter('axisOne.endsWith("bar")')
              }
              
            2. For the second build step, tell it to "Use the provided DSL script" and give it the script
              matrixJob('Jenkins-47528-example') {
                axes {
                  text('axisThree','threefoo','threebar')
                }
              }
              

            When the job is run, the first DSL build step completes as expected, but the second DSL build step fails with the exception (described in the main issue text). Also, the Jenkins-47528-example job is left with the combinationFilter from the first build step and the axisThree axis from the second.

            One possible solution would be to change the DSL plugin's handling of matrix jobs so that, when it starts processing a matrix job, the first thing that the DSL plugin does is to remove any existing combinationFilter (regardless of whether or not the combinationFilter gets set by the DSL code later) and then defers setting it to the value specified by the DSL until right at the end of all other processing of that job. This will ensure that the groovy validation code will only have something to validate once everything else is already "in place".

            Note: I've also tested a similar scenario with touchStoneFilter and found that this works as it should, i.e. no exceptions thrown. It's even possible to define a job with an invalid touchStoneFilter without any exceptions thrown. However, as both of these expressions are groovy code, it's plausible that the Matrix plugin code will later be changed to treat them equally, and thus the touchStoneFilter may be subjected to the same kind of "extreme" validation. It would be wise to ensure that the DSL plugin treats touchStoneFilter with the same level of caution as is currently required with combinationFilter.

            Show
            pjdarton pjdarton added a comment - To reproduce, create a new Jenkins job with two "Process Job DSLs" build steps: For the first build step, tell it to "Use the provided DSL script" and give it the script matrixJob( 'Jenkins-47528-example' ) { axes { text( 'axisOne' , 'onefoo' , 'onebar' ) text( 'axisTwo' , 'twofoo' , 'twobar' ) } combinationFilter( 'axisOne.endsWith( "bar" )' ) } For the second build step, tell it to "Use the provided DSL script" and give it the script matrixJob( 'Jenkins-47528-example' ) { axes { text( 'axisThree' , 'threefoo' , 'threebar' ) } } When the job is run, the first DSL build step completes as expected, but the second DSL build step fails with the exception (described in the main issue text). Also, the Jenkins-47528-example job is left with the combinationFilter from the first build step and the axisThree axis from the second. One possible solution would be to change the DSL plugin's handling of matrix jobs so that, when it starts processing a matrix job, the first thing that the DSL plugin does is to remove any existing combinationFilter (regardless of whether or not the combinationFilter gets set by the DSL code later) and then defers setting it to the value specified by the DSL until right at the end of all other processing of that job. This will ensure that the groovy validation code will only have something to validate once everything else is already "in place". Note: I've also tested a similar scenario with touchStoneFilter and found that this works as it should, i.e. no exceptions thrown. It's even possible to define a job with an invalid touchStoneFilter without any exceptions thrown. However, as both of these expressions are groovy code, it's plausible that the Matrix plugin code will later be changed to treat them equally, and thus the touchStoneFilter may be subjected to the same kind of "extreme" validation. It would be wise to ensure that the DSL plugin treats touchStoneFilter with the same level of caution as is currently required with combinationFilter .
            Hide
            daspilker Daniel Spilker added a comment -

            The Matrix Project plugin does not seem to support the AbstractItem.updateByXml API call. See the excerpt from the stack trace below. This also affects other components using this API, e.g. the update-job CLI command or the REST API. So this should be fixed in Matrix Project plugin.

            groovy.lang.MissingPropertyException: No such property: axisOne for class: groovy.lang.Binding
            at groovy.lang.Binding.getVariable(Binding.java:63)
            at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:224)
            at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:241)
            at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:238)
            at org.kohsuke.groovy.sandbox.impl.Checker$checkedGetProperty.callStatic(Unknown Source)
            at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
            at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
            at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:230)
            at Script1.run(Script1.groovy:1)
            at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.run(GroovySandbox.java:141)
            at hudson.matrix.FilterScript.evaluate(FilterScript.java:45)
            at hudson.matrix.FilterScript.apply(FilterScript.java:85)
            at hudson.matrix.Combination.evalGroovyExpression(Combination.java:101)
            at hudson.matrix.Combination.evalGroovyExpression(Combination.java:91)
            at hudson.matrix.MatrixProject.rebuildConfigurations(MatrixProject.java:646)
            at hudson.matrix.MatrixProject.onLoad(MatrixProject.java:509)
            at hudson.model.AbstractItem$1.call(AbstractItem.java:674)
            at hudson.model.AbstractItem$1.call(AbstractItem.java:672)
            at hudson.model.Items.whileUpdatingByXml(Items.java:135)
            at hudson.model.AbstractItem.updateByXml(AbstractItem.java:672)
            
            Show
            daspilker Daniel Spilker added a comment - The Matrix Project plugin does not seem to support the AbstractItem.updateByXml API call. See the excerpt from the stack trace below. This also affects other components using this API, e.g. the update-job CLI command or the REST API. So this should be fixed in Matrix Project plugin. groovy.lang.MissingPropertyException: No such property: axisOne for class: groovy.lang.Binding at groovy.lang.Binding.getVariable(Binding.java:63) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:224) at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:241) at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:238) at org.kohsuke.groovy.sandbox.impl.Checker$checkedGetProperty.callStatic(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:230) at Script1.run(Script1.groovy:1) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.run(GroovySandbox.java:141) at hudson.matrix.FilterScript.evaluate(FilterScript.java:45) at hudson.matrix.FilterScript.apply(FilterScript.java:85) at hudson.matrix.Combination.evalGroovyExpression(Combination.java:101) at hudson.matrix.Combination.evalGroovyExpression(Combination.java:91) at hudson.matrix.MatrixProject.rebuildConfigurations(MatrixProject.java:646) at hudson.matrix.MatrixProject.onLoad(MatrixProject.java:509) at hudson.model.AbstractItem$1.call(AbstractItem.java:674) at hudson.model.AbstractItem$1.call(AbstractItem.java:672) at hudson.model.Items.whileUpdatingByXml(Items.java:135) at hudson.model.AbstractItem.updateByXml(AbstractItem.java:672)
            Hide
            daspilker Daniel Spilker added a comment -

            The combination filter is not removed due to an issue in config XML processing in Jenkins. See JENKINS-21017. This only affects XML elements directly below the root element. Elements that are removed in config XML are not set to null in the runtime config.

            As a workaround you can try to set the combination filter to an empty string.

            Show
            daspilker Daniel Spilker added a comment - The combination filter is not removed due to an issue in config XML processing in Jenkins. See JENKINS-21017 . This only affects XML elements directly below the root element. Elements that are removed in config XML are not set to null in the runtime config. As a workaround you can try to set the combination filter to an empty string.
            Hide
            jglick Jesse Glick added a comment -

            Verified that the primary problem is a duplicate of JENKINS-21017.

            an invalid combination filter should merely be a warning

            Leaving open in matrix-project for this.

            Show
            jglick Jesse Glick added a comment - Verified that the primary problem is a duplicate of JENKINS-21017 . an invalid combination filter should merely be a warning Leaving open in matrix-project for this.

              People

              • Assignee:
                Unassigned
                Reporter:
                pjdarton pjdarton
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: