Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45688

Unable to set extensions for Git SCM Source via DSL

    Details

    • Similar Issues:

      Description

      In the past I used `gitSCMSource` to specify my git configuration with extensions in my Job DSL.
       
      Template I used:

      multibranchPipelineJob('${JOB_NAME}') {
          displayName('${JOB_NAME}')
          description('${JOB_DESCRIPTION}')
          orphanedItemStrategy {
              discardOldItems {
                  numToKeep(7)
              }
          }
          branchSources {
              branchSource {
                  source {
                      gitSCMSource {
                          id('')
                          remoteName('origin')
                          rawRefSpecs('')
                          ignoreOnPushNotifications(false)
                          remote('${JOB_GIT_URL}')
                          credentialsId('${GIT_CREDENTIALS_ID}')
                          includes('*')
                          excludes('')
                          extensions {
                              wipeWorkspace()
                              preBuildMerge {
                                  options {
                                      mergeRemote('origin')
                                      mergeTarget('${GIT_BRANCH}')
                                      mergeStrategy('default')
                                      fastForwardMode('FF')
                                  }
                              }
                          }
                      }
                  }
              }
          }
      }

      Now after updating to latest Jenkins/Plugins this no longer works. I looked at using the `git` source instead of `gitSCMSource` but it does not provide a way to set the extensions.

      In the new git-plugin 3.4.0 extensions are deprecated and are now traits. Not sure if this has anything to do with the issue.

        Attachments

          Issue Links

            Activity

            arty13 Art V created issue -
            Hide
            markewaite Mark Waite added a comment - - edited

            I suspect the fact that it worked before was not an intended use of the GitSCMSource API. Stephen Connolly will have more insights on the expectation of the use of the extensions from GitSCMSource.

            Does the technique in the official documentation work? It seems to be setting extensions using a git node in the DSL, without using GitSCMSource.

            Does the technique described in "use unsupported plugins with job DSL" still work? That technique does not seem to require GitSCMSource.

            A similar technique is described in "How to use Jenkins JobDSL to set 'Check out to specific local branch' in Git Plugin" , in "sparse checkout with jenkins dsl plugin" and in "use Jenkins DSL to specify a git executable in github SCM".

            I'm prone to close this as "Won't fix", but will wait for verification from Stephen Connolly

            Show
            markewaite Mark Waite added a comment - - edited I suspect the fact that it worked before was not an intended use of the GitSCMSource API. Stephen Connolly will have more insights on the expectation of the use of the extensions from GitSCMSource. Does the technique in the official documentation work? It seems to be setting extensions using a git node in the DSL, without using GitSCMSource. Does the technique described in " use unsupported plugins with job DSL " still work? That technique does not seem to require GitSCMSource. A similar technique is described in " How to use Jenkins JobDSL to set 'Check out to specific local branch' in Git Plugin " , in " sparse checkout with jenkins dsl plugin " and in " use Jenkins DSL to specify a git executable in github SCM ". I'm prone to close this as "Won't fix", but will wait for verification from Stephen Connolly
            Hide
            stephenconnolly Stephen Connolly added a comment -

            So my understanding is that you need to use the techniques for using unsupported plugins with job DSL to configure traits... at least until we get around to adding the @Symbol annotations (JENKINS-45504)... of course that is currently blocked on JENKINS-45503 because for reasons documented in those issues.

            Having said all that, we did retain the setter for extensions (just marked deprecated is all) and that setter will create the matching traits, so the only thing I can think is that you may be hitting the ref trimming... in which case all you need to do is add the trait that allows specifying an additional refspec to fetch and ensure that the master branch refspec is always fetched from the remote so that the merge extension / trait can work...

            Mark Waite I think we need to hear back on how this "no longer works" to determine if:

            • The jobs are not getting configured correctly; or
            • The configured jobs are failing because of the missing refspec for origin; or
            • Something else
            Show
            stephenconnolly Stephen Connolly added a comment - So my understanding is that you need to use the techniques for using unsupported plugins with job DSL to configure traits... at least until we get around to adding the @Symbol annotations ( JENKINS-45504 )... of course that is currently blocked on JENKINS-45503 because for reasons documented in those issues. Having said all that, we did retain the setter for extensions (just marked deprecated is all) and that setter will create the matching traits, so the only thing I can think is that you may be hitting the ref trimming... in which case all you need to do is add the trait that allows specifying an additional refspec to fetch and ensure that the master branch refspec is always fetched from the remote so that the merge extension / trait can work... Mark Waite I think we need to hear back on how this "no longer works" to determine if: The jobs are not getting configured correctly; or The configured jobs are failing because of the missing refspec for origin; or Something else
            arty13 Art V made changes -
            Field Original Value New Value
            Attachment Screen Shot 2017-07-21 at 09.36.26.png [ 38988 ]
            Attachment Screen Shot 2017-07-21 at 09.36.06.png [ 38989 ]
            Hide
            arty13 Art V added a comment -

            Using this example:

            multibranchPipelineJob('AMQ-Docker-Branch') {
              displayName('AMQ-Docker-Branch')
              description('AMQ Docker Container Branch Builds')
              orphanedItemStrategy {
                discardOldItems {
                  numToKeep(7)
                }
              }
              branchSources {
                git {
                  id('')
                  ignoreOnPushNotifications(false)
                  remote('git@git.com/amq-docker.git')
                  credentialsId('SSHKeyCredentials')
                  includes('*')
                  excludes('master')
                  extensions {
                    wipeWorkspace()
                    preBuildMerge {
                      options {
                        mergeRemote('origin')
                        mergeTarget('master')
                        mergeStrategy('default')
                        fastForwardMode('FF')
                      }
                    }
                  }
                }
              }
            }
            

            I get this error:

            ERROR: (script, line 17) No signature of method: javaposse.jobdsl.dsl.helpers.workflow.GitBranchSourceContext.extensions() is applicable for argument types: (script$_run_closure1$_closure3$_closure5$_closure6) values: [script$_run_closure1$_closure3$_closure5$_closure6@5218bc64]
            Finished: FAILURE
            

            If I try to do the unsupported approach - I am using this sample, but it does not create the extensions.

            multibranchPipelineJob('CI-TEST-MB') {
              displayName('CI-TEST-MB')
              description('Test Multibranch Job')
              orphanedItemStrategy {
                discardOldItems {
                  numToKeep(7)
                }
              }
            branchSources {
              git {
                id('')
                ignoreOnPushNotifications(false)
                remote('git@git.com/test.git')
                credentialsId('test')
                includes('*')
                excludes('master')
                configure { node -> //the GitSCM node is passed in
            
                    // Add the WipeWorkspace functionality
                    node / 'extensions' << 'hudson.plugins.git.extensions.impl.WipeWorkspace'  {
                    }
                  }
                }
              }
            }
            

             See attachements for result..

             

            Show
            arty13 Art V added a comment - Using this example: multibranchPipelineJob( 'AMQ-Docker-Branch' ) { displayName( 'AMQ-Docker-Branch' ) description( 'AMQ Docker Container Branch Builds' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { git { id('') ignoreOnPushNotifications( false ) remote( 'git@git.com/amq-docker.git' ) credentialsId( 'SSHKeyCredentials' ) includes( '*' ) excludes( 'master' ) extensions { wipeWorkspace() preBuildMerge { options { mergeRemote( 'origin' ) mergeTarget( 'master' ) mergeStrategy( ' default ' ) fastForwardMode( 'FF' ) } } } } } } I get this error: ERROR: (script, line 17) No signature of method: javaposse.jobdsl.dsl.helpers.workflow.GitBranchSourceContext.extensions() is applicable for argument types: (script$_run_closure1$_closure3$_closure5$_closure6) values: [script$_run_closure1$_closure3$_closure5$_closure6@5218bc64] Finished: FAILURE If I try to do the unsupported approach - I am using this sample, but it does not create the extensions. multibranchPipelineJob( 'CI-TEST-MB' ) { displayName( 'CI-TEST-MB' ) description( 'Test Multibranch Job' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { git { id('') ignoreOnPushNotifications( false ) remote( 'git@git.com/test.git' ) credentialsId( 'test' ) includes( '*' ) excludes( 'master' ) configure { node -> //the GitSCM node is passed in // Add the WipeWorkspace functionality node / 'extensions' << 'hudson.plugins.git.extensions.impl.WipeWorkspace' { } } } } }  See attachements for result..  
            Hide
            arty13 Art V added a comment -

            With my latest testing using the configure block - I noticed the node reference is always referencing the root xml node, not the current node. I will continue to test/configure using the job dsl config block and will let you know what I find.

            Show
            arty13 Art V added a comment - With my latest testing using the configure block - I noticed the node reference is always referencing the root xml node, not the current node. I will continue to test/configure using the job dsl config block and will let you know what I find.
            Hide
            arty13 Art V added a comment -

            A hackish workaround using the configure block (bypassing git block entirely)..

            multibranchPipelineJob('CI-TEST-MB') {
              displayName('CI-TEST-MB')
              description('Test Multibranch Job')
              orphanedItemStrategy {
                discardOldItems {
                  numToKeep(7)
                }
              }
              branchSources {
                configure { node ->
                  node / sources(class: 'jenkins.branch.MultiBranchProject$BranchSourceList') / data / 'jenkins.branch.BranchSource' / source(class: 'jenkins.plugins.git.GitSCMSource') {
                    remote 'git@git.com/test.git'
                    credentialsId 'test'
                    includes '*'
                    excludes 'master'
                    ignoreOnPushNotifications 'false'
                    id ''
                    traits {
                      'jenkins.plugins.git.traits.BranchDiscoveryTrait'()
                      'jenkins.plugins.git.traits.WipeWorkspaceTrait'() {
                        extension(class: 'hudson.plugins.git.extensions.impl.WipeWorkspace')
                      }
                      'jenkins.plugins.git.traits.PreBuildMergeTrait'() {
                        extension(class: 'hudson.plugins.git.extensions.impl.PreBuildMerge') {
                          options {
                            mergeRemote 'origin'
                            mergeTarget 'master'
                            mergeStrategy 'default'
                            fastForwardMode 'FF'
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
            

             

            Reason I removed the git {} section entirely was because it would always add a new record of the 'jenkins.branch.BranchSource' section (not updating the existing BranchSource node)

            This might be a separate bug (or misunderstood documentation) If I put the configure block inside the git{} section, the reference 'node' would always point to the root xml element not the current node/git element. 
            Example from documentation..

            job('example') {
                scm {
                    git {
                        remote {
                            url('git@server:account/repo1.git')
                        }
                        configure { node ->
                            // node represents <hudson.plugins.git.GitSCM>
                        }
                    }
                }
            }
            

             

            Show
            arty13 Art V added a comment - A hackish workaround using the configure block (bypassing git block entirely).. multibranchPipelineJob( 'CI-TEST-MB' ) { displayName( 'CI-TEST-MB' ) description( 'Test Multibranch Job' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { configure { node -> node / sources(class: 'jenkins.branch.MultiBranchProject$BranchSourceList' ) / data / 'jenkins.branch.BranchSource' / source(class: 'jenkins.plugins.git.GitSCMSource' ) { remote 'git@git.com/test.git' credentialsId 'test' includes '*' excludes 'master' ignoreOnPushNotifications ' false ' id '' traits { 'jenkins.plugins.git.traits.BranchDiscoveryTrait' () 'jenkins.plugins.git.traits.WipeWorkspaceTrait' () { extension(class: 'hudson.plugins.git.extensions.impl.WipeWorkspace' ) } 'jenkins.plugins.git.traits.PreBuildMergeTrait' () { extension(class: 'hudson.plugins.git.extensions.impl.PreBuildMerge' ) { options { mergeRemote 'origin' mergeTarget 'master' mergeStrategy ' default ' fastForwardMode 'FF' } } } } } } } }   Reason I removed the git {} section entirely was because it would always add a new record of the 'jenkins.branch.BranchSource' section (not updating the existing BranchSource node) This might be a separate bug (or misunderstood documentation) If I put the configure block inside the git{} section, the reference 'node' would always point to the root xml element not the current node/git element.  Example from documentation.. job( 'example' ) { scm { git { remote { url( 'git@server:account/repo1.git' ) } configure { node -> // node represents <hudson.plugins.git.GitSCM> } } } }  
            stephenking Steffen Gebert made changes -
            Link This issue is related to JENKINS-46202 [ JENKINS-46202 ]
            markewaite Mark Waite made changes -
            Assignee Mark Waite [ markewaite ]
            Hide
            daspilker Daniel Spilker added a comment -

            I can not reproduce this with all plugins updated to the latest version. I see multibranchPipelineJob > branchSources > branchSource > source > git > extensions as well as multibranchPipelineJob > branchSources > branchSource > source > git > traits in my embedded API viewer. Can you create a working configuration manually and post the config XML? Then we can figure out what's missing in Job DSL.

            Show
            daspilker Daniel Spilker added a comment - I can not reproduce this with all plugins updated to the latest version. I see multibranchPipelineJob > branchSources > branchSource > source > git > extensions as well as multibranchPipelineJob > branchSources > branchSource > source > git > traits in my embedded API viewer. Can you create a working configuration manually and post the config XML? Then we can figure out what's missing in Job DSL.
            stephenconnolly Stephen Connolly made changes -
            Link This issue duplicates JENKINS-45504 [ JENKINS-45504 ]
            stephenconnolly Stephen Connolly made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Duplicate [ 3 ]
            daspilker Daniel Spilker made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            Hide
            sag47 Sam Gleske added a comment - - edited

            Daniel Spilker I encountered this issue after installing blue ocean. Blue ocean (for some strange reason) requires bit bucket branch source and github branch source plugins to operate. With those plugins installed one faces the issue when trying to configure the git plugin as a branch source.

            When I manually disable those two plugins (breaking blue ocean) I get a working DSL again.

            Show
            sag47 Sam Gleske added a comment - - edited Daniel Spilker I encountered this issue after installing blue ocean. Blue ocean (for some strange reason) requires bit bucket branch source and github branch source plugins to operate. With those plugins installed one faces the issue when trying to configure the git plugin as a branch source. When I manually disable those two plugins (breaking blue ocean) I get a working DSL again.
            Hide
            sag47 Sam Gleske added a comment -

            Here's a link to the DSL used.

            https://github.com/gimp-ci/jenkins-dsl/blob/292cafd41e8c26e200dbe967aba7ec71e120df84/jobs/gimp_multibranch_pipelines.groovy

            Here's a list of plugins installed that broke the the DSL. https://github.com/gimp-ci/jenkins-os-packages/commit/2e9e0e6b5a2e0287268ca70d836f2a7e71fef72b

            In particular I confirmed the following plugins broke the DSL.

            • org.jenkins-ci.plugins:cloudbees-bitbucket-branch-source:2.2.11
            • org.jenkins-ci.plugins:github-branch-source:2.3.4
            Show
            sag47 Sam Gleske added a comment - Here's a link to the DSL used. https://github.com/gimp-ci/jenkins-dsl/blob/292cafd41e8c26e200dbe967aba7ec71e120df84/jobs/gimp_multibranch_pipelines.groovy Here's a list of plugins installed that broke the the DSL. https://github.com/gimp-ci/jenkins-os-packages/commit/2e9e0e6b5a2e0287268ca70d836f2a7e71fef72b In particular I confirmed the following plugins broke the DSL. org.jenkins-ci.plugins:cloudbees-bitbucket-branch-source:2.2.11 org.jenkins-ci.plugins:github-branch-source:2.3.4
            Hide
            sag47 Sam Gleske added a comment -

            This nonsense requiring the configure block is ridiculous.  I've spent the past 5 hours trying to figure it out and I'm definitely not a Job DSL newb.

            Show
            sag47 Sam Gleske added a comment - This nonsense requiring the configure block is ridiculous.  I've spent the past 5 hours trying to figure it out and I'm definitely not a Job DSL newb.
            Hide
            stephenking Steffen Gebert added a comment -

            Sam Gleske I am sure this is not by intention and the maintainers will welcome your pull requests to stop this nonsense.

            Show
            stephenking Steffen Gebert added a comment - Sam Gleske I am sure this is not by intention and the maintainers will welcome your pull requests to stop this nonsense.
            Hide
            sag47 Sam Gleske added a comment - - edited

            Steffen Gebert my comments have added a lot of value to this issue by referencing both source code, examples, and reproduction.  Your comment claims I should stop doing this.  I disagree.

            Edit: I think I initially misread your comment.  In any case, "contribute code" should never be the default retort as a comeback in my opinion.

            Show
            sag47 Sam Gleske added a comment - - edited Steffen Gebert my comments have added a lot of value to this issue by referencing both source code, examples, and reproduction.  Your comment claims I should stop doing this.  I disagree. Edit: I think I initially misread your comment.  In any case, "contribute code" should never be the default retort as a comeback in my opinion.
            Hide
            stephenking Steffen Gebert added a comment -

            No, my point was "please be friendly". Nobody creates "nonsense" by intention.

            Show
            stephenking Steffen Gebert added a comment - No, my point was "please be friendly". Nobody creates "nonsense" by intention.
            sag47 Sam Gleske made changes -
            Labels job-dsl plugin git-plugin job-dsl plugin
            Hide
            sag47 Sam Gleske added a comment -

            Sure, I can put my frustration in check.  My intent wasn't to be hostile nor to imply it was done on purpose (it would be odd to assume that it's on purpose).

            Show
            sag47 Sam Gleske added a comment - Sure, I can put my frustration in check.  My intent wasn't to be hostile nor to imply it was done on purpose (it would be odd to assume that it's on purpose).
            Hide
            sag47 Sam Gleske added a comment -

            Added git-plugin since the git-plugin should define "@Symbol" for its classes to help with the DSL generation to be unique.

            Show
            sag47 Sam Gleske added a comment - Added git-plugin since the git-plugin should define "@Symbol" for its classes to help with the DSL generation to be unique.
            Show
            sag47 Sam Gleske added a comment - https://github.com/jenkinsci/git-plugin/pull/595
            Hide
            stephenking Steffen Gebert added a comment -

            Sam Gleske, thanks for bringing this forward!

            Show
            stephenking Steffen Gebert added a comment - Sam Gleske , thanks for bringing this forward!
            Hide
            sag47 Sam Gleske added a comment -

            You're welcome.

            Show
            sag47 Sam Gleske added a comment - You're welcome.

              People

              • Assignee:
                Unassigned
                Reporter:
                arty13 Art V
              • Votes:
                1 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: