Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30519

Declarative job properties in multibranch

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Since a multibranch project creates all of its subprojects (WorkflowJob), any customizations you may want to make on them must be defined at the multibranch level. So far this has been handled with BranchProperty, but that is undesirable in most cases for several reasons:

      1. It defeats the idea of code-as-configuration: you are back to keeping important job configuration in $JENKINS_HOME/jobs/*/config.xml, rather than your own SCM.
      2. Adjusting properties on a per-branch basis is awkward. You can use NamedExceptionsBranchPropertyStrategy, but this is extra work to configure if you have a lot of branches, and again cannot be adjusted outside the Jenkins configuration UI, or set right away for newly created branches (unless you follow a strict naming convention).
      3. While we can hardcode support for certain aspects of static job configuration, as we do with WorkflowParameterDefinitionBranchProperty, BuildRetentionBranchProperty, and RateLimitBranchProperty¹, this is not extensible to JobProperty implementations defined by other plugins.

      Kohsuke Kawaguchi came up with a novel idea to solve this for workflow-multibranch: add a Step which simply (re-)sets the List<JobProperty> associated with the job it is run on. You need to run one build to get the properties set, but that is usually not onerous since multibranch will always trigger an initial build for every branch anyway, and few properties really need to be set in the short interval between the new project being created and the first build getting as far as this step. Jenkinsfile can feel free to hardcode certain properties, or dynamically include some, or configure some according to branch name, etc. While it may sound weird at first blush to have a dynamic script update static properties, this is actually analogous to how Workflow supports SCM polling and triggers without the need for static SCM configuration.

      So then we need to:

      1. define the new step
      2. delete WorkflowParameterDefinitionBranchProperty (subsumed by the new system)
      3. retrofit various fields of WorkflowJob to be configured instead by JobProperty
        1. buildDiscarder (suppress the existing BuildRetentionBranchProperty)
        2. triggers (note that SCMTrigger, GitHubTrigger, etc. would be superfluous in a multibranch project, since SCMSource covers that need, but TimerTrigger and miscellaneous plugin contributions may be wanted)
        3. quietPeriod (if anyone cares)
        4. concurrentBuild
        5. authToken²

      TBD whether the literate plugin would want to follow suit, or stick with the existing BranchProperty-driven approach.

      Some "closed-class" BranchProperty implementations will still be needed, such as UntrustedBranchProperty or a Workflow equivalent TBD.

      ¹Probably this should never have been a BranchProperty at all; it has no apparent logical dependency on multibranch. Really it should be a separate plugin with a general-purpose JobProperty.

      ²Trickier since it is normally a per-Job secret. Fine for it to be the same for all children of a WorkflowMultiBranchProject, but should of course not be hardcoded in the script. Could use StringCredentials, though this will not work well with automatic repository scanning: you would not want to manually create a token for each repository. Nicer would be to keep a HMACConfidentialKey, with the per-project token somehow displayed to anyone with Item.BUILD.

        Attachments

          Issue Links

            Activity

            jglick Jesse Glick created issue -
            Hide
            jglick Jesse Glick added a comment -

            JENKINS-29922 is not strictly required, but would make the result look better.

            Show
            jglick Jesse Glick added a comment - JENKINS-29922 is not strictly required, but would make the result look better.
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Link This issue depends on JENKINS-29922 [ JENKINS-29922 ]
            jglick Jesse Glick made changes -
            Labels multibranch
            jglick Jesse Glick made changes -
            Link This issue is blocking JENKINS-30206 [ JENKINS-30206 ]
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Link This issue depends on JENKINS-28510 [ JENKINS-28510 ]
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-29880 [ JENKINS-29880 ]
            jglick Jesse Glick made changes -
            Link This issue depends on JENKINS-29711 [ JENKINS-29711 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "PR 209 (Web Link)" [ 13153 ]
            Hide
            stephenconnolly Stephen Connolly added a comment -

            I need to think over it some more, but I suspect this is closer to the "right" way to do things... may need minor tweaks along the way

            Show
            stephenconnolly Stephen Connolly added a comment - I need to think over it some more, but I suspect this is closer to the "right" way to do things... may need minor tweaks along the way
            Hide
            jglick Jesse Glick added a comment -

            Pending JENKINS-27295, using ParametersDefinitionProperty is not ideal since the first build (or the first build to request this job property) will not be started with any parameters, so even default parameter values will not be honored: you need to manually fall back to some default.

            Show
            jglick Jesse Glick added a comment - Pending JENKINS-27295 , using ParametersDefinitionProperty is not ideal since the first build (or the first build to request this job property) will not be started with any parameters, so even default parameter values will not be honored: you need to manually fall back to some default.
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-27295 [ JENKINS-27295 ]
            Hide
            jglick Jesse Glick added a comment -

            1, 2, and 3.1 done; enough for now, can file follow-up issues if the need arises for 3.2–5.

            Show
            jglick Jesse Glick added a comment - 1, 2, and 3.1 done; enough for now, can file follow-up issues if the need arises for 3.2–5.
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-31737 [ JENKINS-31737 ]
            Hide
            abayer Andrew Bayer added a comment -

            So, before I file a JIRA, just want to make sure - triggers are still not possible, right? So there's no way to currently have a Multibranch branch Jenkinsfile that gets executed @daily, etc?

            Show
            abayer Andrew Bayer added a comment - So, before I file a JIRA, just want to make sure - triggers are still not possible, right? So there's no way to currently have a Multibranch branch Jenkinsfile that gets executed @daily , etc?
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            I suspect that you can create an external periodic FreeStyle project, which would trigger the desired branches (dirty hack)

            Show
            oleg_nenashev Oleg Nenashev added a comment - I suspect that you can create an external periodic FreeStyle project, which would trigger the desired branches (dirty hack)
            jglick Jesse Glick made changes -
            Link This issue is blocking JENKINS-34005 [ JENKINS-34005 ]
            Hide
            jglick Jesse Glick added a comment -

            I filed JENKINS-34005 for it.

            Show
            jglick Jesse Glick added a comment - I filed JENKINS-34005 for it.
            jglick Jesse Glick made changes -
            Link This issue is blocking JENKINS-34547 [ JENKINS-34547 ]
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-35310 [ JENKINS-35310 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 165630 ] JNJira + In-Review [ 197771 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-37220 [ JENKINS-37220 ]
            abayer Andrew Bayer made changes -
            Component/s pipeline-general [ 21692 ]
            abayer Andrew Bayer made changes -
            Component/s workflow-plugin [ 18820 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-33020 [ JENKINS-33020 ]
            Hide
            tknerr Torben Knerr added a comment -

            Jesse Glick Stephen Connolly desperately looking for a way to configure the build retention strategy on a multibranch pipeline job level. It looks like this JIRA issue directly addresses that, but it is totally unclear to me how to actually make use of it.

            Is this possible right now to define job properties on a multibranch level?

            I see there is a "Properties" section in the UI of the multibranch pipeline job configuration, but there is no way to add items via the UI.

            Then there is the "Property Strategy" under "Branch Sources", which also sounds plausible, but the only property you can add here is "Suppress automatic SCM triggering", but none of the others available in https://github.com/jenkinsci/branch-api-plugin/tree/master/src/main/java/jenkins/branch *Property.java

            Any advice how to deal with this?

            P.S.: I'd love to do that via JobDSL, but as I don't see any way to do that via the UI I doubt it's possible via JobDSL...

            Show
            tknerr Torben Knerr added a comment - Jesse Glick Stephen Connolly desperately looking for a way to configure the build retention strategy on a multibranch pipeline job level. It looks like this JIRA issue directly addresses that, but it is totally unclear to me how to actually make use of it. Is this possible right now to define job properties on a multibranch level? I see there is a "Properties" section in the UI of the multibranch pipeline job configuration, but there is no way to add items via the UI. Then there is the "Property Strategy" under "Branch Sources", which also sounds plausible, but the only property you can add here is "Suppress automatic SCM triggering", but none of the others available in https://github.com/jenkinsci/branch-api-plugin/tree/master/src/main/java/jenkins/branch  *Property.java Any advice how to deal with this? P.S.: I'd love to do that via JobDSL, but as I don't see any way to do that via the UI I doubt it's possible via JobDSL...
            Hide
            tknerr Torben Knerr added a comment -

            I finally found a way to do that via the auto-generated JobDSL. After generating the Jobs these properties are also visible in the UI, so it seems it's simply a UI bug that prevented me from setting the properties via the "Property Strategy" under "Branch Sources".

            The JobDSL API viewer shows the generated JobDSL here:
            http://your-jenkins:8080/plugin/job-dsl/api-viewer/index.html#path/multibranchPipelineJob-branchSources-branchSource

            Example JobDSL snippet for a multibranchPipelineJob with BuildRetentionBranchProperty configured:
            https://gist.github.com/tknerr/c79a514db4bdbfb4956aaf0ee53836c8 

             

             

            Show
            tknerr Torben Knerr added a comment - I finally found a way to do that via the auto-generated JobDSL. After generating the Jobs these properties are also visible in the UI, so it seems it's simply a UI bug that prevented me from setting the properties via the "Property Strategy" under "Branch Sources". The JobDSL API viewer shows the generated JobDSL here: http://your-jenkins:8080/plugin/job-dsl/api-viewer/index.html#path/multibranchPipelineJob-branchSources-branchSource Example JobDSL snippet for a multibranchPipelineJob with BuildRetentionBranchProperty configured: https://gist.github.com/tknerr/c79a514db4bdbfb4956aaf0ee53836c8      
            jglick Jesse Glick made changes -
            Link This issue depends on JENKINS-51227 [ JENKINS-51227 ]

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                jglick Jesse Glick
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: