Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43693

multibranchPipelineJob overrides old branch indexing sources

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: job-dsl-plugin
    • Labels:
      None
    • Environment:
      Debian 8
      Jenkins 2.46.1 (as service)

      Plugins (all newest stable):
      Job DSL 1.61
      Pipeline: Multibranch 2.14
    • Similar Issues:

      Description

      When running the multibranchPipelineJob step it seems like it overrides the sources of a Multibranch Pipeline Job instead of just updating it. After running the step, even though nothing has changed the output of the branch indexing will become:

      Checking branch master
            ‘Jenkinsfile’ found
      Met criteria
      Takeover for job-dsl-plugin_multibranch » master by source #1 from source that no longer exists
      Branch reopened: master (b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
      Scheduled build for branch: master
      

       

      To test this, I have installed a brand new Jenkins with the Pipeline plugins and Job DSL plugin (all as the stable version) and made a freestyle job, with a Job DSL step that has the following script:

      multibranchPipelineJob('job-dsl-plugin_multibranch') {
        branchSources {
          git { 
            remote('https://github.com/jenkinsci/job-dsl-plugin.git')
          }
        }
      }
      

       

      After running the Job DSL step and afterwards running branch indexing a "master" job is created with the following log message from branch indexing:

      Checking branch master
            ‘Jenkinsfile’ found
      Met criteria
      Scheduled build for branch: master
      

       

      When running branch indexing again, following log message are shown as expected:

      Checking branch master
            ‘Jenkinsfile’ found
      Met criteria
      No changes detected: master (still at b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
      

       

      However, if I run the Job DSL again and run branch indexing, the first log message (with takeover and so on) are shown. This is not what I would have expected, I would have expected something like a "no changes detected" message.

       

        Attachments

          Activity

          anigif Anders Kielsholm created issue -
          anigif Anders Kielsholm made changes -
          Field Original Value New Value
          Description When running the multibranchPipelineJob step it seems like it overrides the sources of a Multibranch Pipeline Job instead of just updating it. After running the step, even though nothing has changed the output of the branch indexing will become:

           
          {code:java}
          Checking branch master
                ‘Jenkinsfile’ found
          Met criteria
          Takeover for job-dsl-plugin_multibranch » master by source #1 from source that no longer exists
          Branch reopened: master (b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
          Scheduled build for branch: master
          {code}
           

           

          To test this, I have installed a brand new Jenkins with the Pipeline plugins and Job DSL plugin (all as the stable version) and made a freestyle job, with a Job DSL step that has the following script:

           
          {code:java}
          multibranchPipelineJob('job-dsl-plugin_multibranch') {
            branchSources {
              git {
                remote('https://github.com/jenkinsci/job-dsl-plugin.git')
              }
            }
          }
          {code}
           

           

          After running the Job DSL step and afterwards running branch indexing a "master" job is created with the following log message from branch indexing:
          {code:java}
          Checking branch master
                ‘Jenkinsfile’ found
          Met criteria
          Scheduled build for branch: master
          {code}
          When running branch indexing again, following log message are shown as expected:

           

           
          {code:java}
          Checking branch master
                ‘Jenkinsfile’ found
          Met criteria
          No changes detected: master (still at b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
          {code}
          However, if I run the Job DSL again and run branch indexing, the first log message (with takeover and so on) are shown. This is not what I would have expected, I would have expected something like a "no changes detected" message.

           
          When running the multibranchPipelineJob step it seems like it overrides the sources of a Multibranch Pipeline Job instead of just updating it. After running the step, even though nothing has changed the output of the branch indexing will become:
          {code:java}
          Checking branch master
                ‘Jenkinsfile’ found
          Met criteria
          Takeover for job-dsl-plugin_multibranch » master by source #1 from source that no longer exists
          Branch reopened: master (b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
          Scheduled build for branch: master
          {code}
           

          To test this, I have installed a brand new Jenkins with the Pipeline plugins and Job DSL plugin (all as the stable version) and made a freestyle job, with a Job DSL step that has the following script:
          {code:java}
          multibranchPipelineJob('job-dsl-plugin_multibranch') {
            branchSources {
              git {
                remote('https://github.com/jenkinsci/job-dsl-plugin.git')
              }
            }
          }
          {code}
           

          After running the Job DSL step and afterwards running branch indexing a "master" job is created with the following log message from branch indexing:
          {code:java}
          Checking branch master
                ‘Jenkinsfile’ found
          Met criteria
          Scheduled build for branch: master
          {code}
           

          When running branch indexing again, following log message are shown as expected:
          {code:java}
          Checking branch master
                ‘Jenkinsfile’ found
          Met criteria
          No changes detected: master (still at b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
          {code}
           

          However, if I run the Job DSL again and run branch indexing, the first log message (with takeover and so on) are shown. This is not what I would have expected, I would have expected something like a "no changes detected" message.

           
          Hide
          daspilker Daniel Spilker added a comment - - edited

          The generated config XML for the branch source contains an <id> element. Job DSL generates a random ID each time the seed job runs. I think the Multibranch plugin uses this ID to manage the repository state. So it's not OK to generate a random ID with each run of the seed job.

          You can use the Automatically Generated DSL to set the ID:

          multibranchPipelineJob('job-dsl-plugin_multibranch') {
            branchSources {
              branchSource {
                source {
                  git {
                    id('must-be-unique')
                    remote('https://github.com/jenkinsci/job-dsl-plugin.git')
                    credentialsId(null)
                    remoteName('origin')
                    rawRefSpecs('+refs/heads/*:refs/remotes/origin/*')
                    includes('*')
                    excludes('')
                    ignoreOnPushNotifications(false)
                  }
                }
              }
            }
          }
          
          Show
          daspilker Daniel Spilker added a comment - - edited The generated config XML for the branch source contains an <id> element. Job DSL generates a random ID each time the seed job runs. I think the Multibranch plugin uses this ID to manage the repository state. So it's not OK to generate a random ID with each run of the seed job. You can use the Automatically Generated DSL to set the ID: multibranchPipelineJob( 'job-dsl-plugin_multibranch' ) { branchSources { branchSource { source { git { id( 'must-be-unique' ) remote( 'https: //github.com/jenkinsci/job-dsl-plugin.git' ) credentialsId( null ) remoteName( 'origin' ) rawRefSpecs( '+refs/heads/*:refs/remotes/origin/*' ) includes( '*' ) excludes('') ignoreOnPushNotifications( false ) } } } } }
          Hide
          rocktavious kyle rockman added a comment -

          HUZZAH!  I've been fighting with this bug for a month or two now at work. Daniel Spilker Thank you that fixed the issue!

          Show
          rocktavious kyle rockman added a comment - HUZZAH!  I've been fighting with this bug for a month or two now at work. Daniel Spilker Thank you that fixed the issue!
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Daniel Spilker
          Path:
          docs/Home.md
          job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContext.groovy
          job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitBranchSourceContext.groovy
          job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitHubBranchSourceContext.groovy
          job-dsl-core/src/test/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContextsSpec.groovy
          http://jenkins-ci.org/commit/job-dsl-plugin/1bd2d95eaae1356de6b3f92613766527280f41ca
          Log:
          enhanced support for the Pipeline Multibranch Plugin

          [FIXES JENKINS-43693]

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Spilker Path: docs/Home.md job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContext.groovy job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitBranchSourceContext.groovy job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitHubBranchSourceContext.groovy job-dsl-core/src/test/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContextsSpec.groovy http://jenkins-ci.org/commit/job-dsl-plugin/1bd2d95eaae1356de6b3f92613766527280f41ca Log: enhanced support for the Pipeline Multibranch Plugin [FIXES JENKINS-43693]
          scm_issue_link SCM/JIRA link daemon made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Daniel Spilker
          Path:
          docs/Home.md
          job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContext.groovy
          job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitBranchSourceContext.groovy
          job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitHubBranchSourceContext.groovy
          job-dsl-core/src/test/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContextsSpec.groovy
          http://jenkins-ci.org/commit/job-dsl-plugin/f8b51912215c545d702b6e8c8fbe6fc129fb3130
          Log:
          Merge branch 'JENKINS-43693'

          Compare: https://github.com/jenkinsci/job-dsl-plugin/compare/c722b8bb538c...f8b51912215c

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Spilker Path: docs/Home.md job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContext.groovy job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitBranchSourceContext.groovy job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitHubBranchSourceContext.groovy job-dsl-core/src/test/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContextsSpec.groovy http://jenkins-ci.org/commit/job-dsl-plugin/f8b51912215c545d702b6e8c8fbe6fc129fb3130 Log: Merge branch ' JENKINS-43693 ' Compare: https://github.com/jenkinsci/job-dsl-plugin/compare/c722b8bb538c...f8b51912215c
          daspilker Daniel Spilker made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          chancez Chance Zibolski added a comment -

          I believe I am still hitting this issue. After updating to jobDSL 1.66, after running my jobDSL job, if I try to build one of the updated multibranch projects it fails with `ERROR: Could not determine exact tip revision of master`. Is it possible this is still unfixed?

          Show
          chancez Chance Zibolski added a comment - I believe I am still hitting this issue. After updating to jobDSL 1.66, after running my jobDSL job, if I try to build one of the updated multibranch projects it fails with `ERROR: Could not determine exact tip revision of master`. Is it possible this is still unfixed?
          Hide
          widgetpl Mike Dziedziela added a comment - - edited

          I face the same issue.

          Jenkins - v2.60.1

          Job DSL - v1.64

          Pipeline Multibranch - v2.16

          I have just commited to test branch but scanning has triggered build for all branches which had Jenkinsfile in it.

          Checking branch develop
                ‘Jenkinsfile’ found
              Met criteria
          Takeover for example_project » develop by source #1 from source that no longer exists
          Branch reopened: develop (11617f79c6408e1d5fd54ea60a64d31b787a9471)
          Scheduled build for branch: develop
          
          Checking branch test
                ‘Jenkinsfile’ found
              Met criteria
          Takeover for example_project » test by source #1 from source that no longer exists
          Branch reopened: test (f84bcda107da005f9363759a68af2008a135a84e)
          Scheduled build for branch: test
          
          Checking branch master
                ‘Jenkinsfile’ found
              Met criteria
          Takeover for example_project » master by source #1 from source that no longer exists
          Branch reopened: master (2a2521fe4ea2ab71edb51b3279b5e49c4c8e77d5)
          Scheduled build for branch: master

          let me know if you need more information.

          EDIT:

          Jenkins is integrated with GitLab v. 10.0.3 CE

          Show
          widgetpl Mike Dziedziela added a comment - - edited I face the same issue. Jenkins - v2.60.1 Job DSL - v1.64 Pipeline Multibranch - v2.16 I have just commited to test branch but scanning has triggered build for all branches which had Jenkinsfile in it. Checking branch develop ‘Jenkinsfile’ found Met criteria Takeover for example_project » develop by source #1 from source that no longer exists Branch reopened: develop (11617f79c6408e1d5fd54ea60a64d31b787a9471) Scheduled build for branch: develop Checking branch test ‘Jenkinsfile’ found Met criteria Takeover for example_project » test by source #1 from source that no longer exists Branch reopened: test (f84bcda107da005f9363759a68af2008a135a84e) Scheduled build for branch: test Checking branch master ‘Jenkinsfile’ found Met criteria Takeover for example_project » master by source #1 from source that no longer exists Branch reopened: master (2a2521fe4ea2ab71edb51b3279b5e49c4c8e77d5) Scheduled build for branch: master let me know if you need more information. EDIT: Jenkins is integrated with GitLab v. 10.0.3 CE
          Hide
          kizzie Kat added a comment -

          So the changes made to fix this one seem to still use a random UUID - I'm not sure whether or not generating it based off the name or the repo being cloned (or why not both) would fix this - If I'm reading the problem correctly you need a static UUID being generated for the job for it to be able to cope  - https://github.com/jenkinsci/job-dsl-plugin/compare/c722b8bb538c...f8b51912215c#diff-f468c06e75a4e86856c95107995aaa0fR6

          (and yes I'm also still getting this issue, and have been wondering what is causing it for a while - going to try setting the id manually for a bit instead)

          Show
          kizzie Kat added a comment - So the changes made to fix this one seem to still use a random UUID - I'm not sure whether or not generating it based off the name or the repo being cloned (or why not both) would fix this - If I'm reading the problem correctly you need a static UUID being generated for the job for it to be able to cope  - https://github.com/jenkinsci/job-dsl-plugin/compare/c722b8bb538c...f8b51912215c#diff-f468c06e75a4e86856c95107995aaa0fR6 (and yes I'm also still getting this issue, and have been wondering what is causing it for a while - going to try setting the id manually for a bit instead)
          Hide
          duemir Denys Digtiar added a comment -

          I have created a fresh environment from 2.121 and installing:

          • Pipeline: Multibranch 2.18
          • Github Branch Source Plugin 2.3.4
          • Job DSL 1.69

          I was able to reproduce the issue using the steps in the description. I can still see

          Takeover for job-dsl-plugin_multibranch » master by source #1 from source that no longer exists
          Branch reopened: master (5045094d82daa475fe79e7f5e52eba58c6df9020)
          Scheduled build for branch: master
          

          after I re-run the Seed Job.

          Show
          duemir Denys Digtiar added a comment - I have created a fresh environment from 2.121 and installing: Pipeline: Multibranch 2.18 Github Branch Source Plugin 2.3.4 Job DSL 1.69 I was able to reproduce the issue using the steps in the description. I can still see Takeover for job-dsl-plugin_multibranch » master by source #1 from source that no longer exists Branch reopened: master (5045094d82daa475fe79e7f5e52eba58c6df9020) Scheduled build for branch: master after I re-run the Seed Job.
          duemir Denys Digtiar made changes -
          Resolution Fixed [ 1 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          Hide
          duemir Denys Digtiar added a comment - - edited

          Sorry, Daniel Spilker . It looks like I misunderstood the resolution. Is the resolution to manually add unique Ids to the source definitions? Like this:

          multibranchPipelineJob('job-dsl-plugin_multibranch') {
            branchSources {
              def repo = 'https://github.com/jenkinsci/job-dsl-plugin.git'
              git {
                id = UUID.nameUUIDFromBytes(repo.getBytes())
                remote(repo)
              }
            }
          }
          Show
          duemir Denys Digtiar added a comment - - edited Sorry, Daniel Spilker . It looks like I misunderstood the resolution. Is the resolution to manually add unique Ids to the source definitions? Like this: multibranchPipelineJob( 'job-dsl-plugin_multibranch' ) { branchSources { def repo = 'https: //github.com/jenkinsci/job-dsl-plugin.git' git { id = UUID.nameUUIDFromBytes(repo.getBytes()) remote(repo) } } }
          Hide
          widgetpl Mike Dziedziela added a comment -

          Denys Digtiar yes, it worked for me and I use repository name for id.

          Show
          widgetpl Mike Dziedziela added a comment - Denys Digtiar yes, it worked for me and I use repository name for id.
          duemir Denys Digtiar made changes -
          Status Reopened [ 4 ] Closed [ 6 ]
          Resolution Fixed [ 1 ]

            People

            • Assignee:
              daspilker Daniel Spilker
              Reporter:
              anigif Anders Kielsholm
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: