Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47874

Support multiple repositories in multi-branch pipeline

    XMLWordPrintable

    Details

    • Sprint:
      Blue Ocean 1.5 - beta 3
    • Similar Issues:

      Description

      Within a single job, we need to source branches from multiple repositories that we do not have control over. Currently, if the same branch name occurs in different repositories, only the first is built and subsequent identical branches are dropped.

       

      I suggest that the remote name (if configured) be used to disambiguate the branches, so that they can all be built.

      eg.
      Remote name: linux
      Branch: master

      Remote Name: linux-next
      Branch: master11

      These would be resolved as linux/master and linux-next/master.

       

        Attachments

          Activity

          Hide
          markewaite Mark Waite added a comment -

          Can you describe what you're envisioning in greater detail? The plugin already allows a separately named origin for each repository it clones, and allows the branch name to include that origin for disambiguation.

          Are you in a freestyle job context, or a PIpeline context when reading from multiple repositories?

          Show
          markewaite Mark Waite added a comment - Can you describe what you're envisioning in greater detail? The plugin already allows a separately named origin for each repository it clones, and allows the branch name to include that origin for disambiguation. Are you in a freestyle job context, or a PIpeline context when reading from multiple repositories?
          Hide
          evildeece Alastair D'Silva added a comment - - edited

          Interesting, that is the behavior I was expecting, but not what I am seeing.

          I'm using a Multibranch pipeline job, I've attached an excerpt from the scan log (branchindexing.log).

          I have configured multiple git sources, each one has a unique "Configure Remote Name" value. Is there some other configuration option required to make the origin appear in the branch name?

           Note the collision at the bottom of the log:

          Checking branches...
            Checking branch master
              Met criteria
          Ignoring Linux Kernel » master from source #5 as source #3 owns the branch name

           

          Show
          evildeece Alastair D'Silva added a comment - - edited Interesting, that is the behavior I was expecting, but not what I am seeing. I'm using a Multibranch pipeline job, I've attached an excerpt from the scan log (branchindexing.log). I have configured multiple git sources, each one has a unique "Configure Remote Name" value. Is there some other configuration option required to make the origin appear in the branch name?  Note the collision at the bottom of the log: Checking branches...   Checking branch master     Met criteria Ignoring Linux Kernel » master from source #5 as source #3 owns the branch name  
          Hide
          markewaite Mark Waite added a comment -

          I don't understand how you've configured your multi-branch pipeline job, or more correctly, I don't understand what you're trying to achieve with your multi-branch pipeline job configuration.

          Based on your branch scanning log, I'd guess that you've configured multiple repositories in the multi-branch pipeline definition. I don't know what that should do, or how it should do it.

          As far as I understand a multi-branch pipeline, it is designed to create a single job for each branch in the repository which satisfies the branch name matching conditions. Why not just define a multi-branch pipeline job for each of your repositories, rather than combining them all into a single multi-branch pipeline job?

          I've confirmed in a test job that checkout of different repositories and/or different branches is well behaved from a pipeline job in a multi-branch pipeline. That test job runs as expected, reporting that each subdirectory where a checkout is performed contains the correct repository and branch.

          Can you confirm that you're placing multiple repositories in the multi-branch pipeline definition? If that's what you're doing, please explain more about the desired outcome of that multi-branch pipeline.

          Show
          markewaite Mark Waite added a comment - I don't understand how you've configured your multi-branch pipeline job, or more correctly, I don't understand what you're trying to achieve with your multi-branch pipeline job configuration. Based on your branch scanning log, I'd guess that you've configured multiple repositories in the multi-branch pipeline definition. I don't know what that should do, or how it should do it. As far as I understand a multi-branch pipeline, it is designed to create a single job for each branch in the repository which satisfies the branch name matching conditions. Why not just define a multi-branch pipeline job for each of your repositories, rather than combining them all into a single multi-branch pipeline job? I've confirmed in a test job that checkout of different repositories and/or different branches is well behaved from a pipeline job in a multi-branch pipeline. That test job runs as expected, reporting that each subdirectory where a checkout is performed contains the correct repository and branch. Can you confirm that you're placing multiple repositories in the multi-branch pipeline definition? If that's what you're doing, please explain more about the desired outcome of that multi-branch pipeline.
          Hide
          evildeece Alastair D'Silva added a comment - - edited

          That is correct, what I am after is a single multibranch job that sources it's branches from separate repositories. Each job created by the multibranch job only ever refers to a single branch, which originates from one of the many repos specified by the multibranch pipeline.

          We need these to be managed as a single job to avoid a maintenance nightmare - we have ~20 multibranch jobs, each picking branches from 3-8 repos. Also, from a UX perspective, it's a lot easier for our maintainers to look at a single Blue Ocean results page with all the branches reported, than have to cycle through one for each repo.

          It sounds like the Git plugin is behaving as expected, and the issue lies with how the Multibranch Pipeline plugin handles the branches. Would you agree that this should be logged against that plugin instead?

           

          [later]Looks like the Branch API is responsible:

              public String getName() {
                  // TODO this could include a uniquifying prefix defined in BranchSource
                  return head.getName();
              }

          Show
          evildeece Alastair D'Silva added a comment - - edited That is correct, what I am after is a single multibranch job that sources it's branches from separate repositories. Each job created by the multibranch job only ever refers to a single branch, which originates from one of the many repos specified by the multibranch pipeline. We need these to be managed as a single job to avoid a maintenance nightmare - we have ~20 multibranch jobs, each picking branches from 3-8 repos. Also, from a UX perspective, it's a lot easier for our maintainers to look at a single Blue Ocean results page with all the branches reported, than have to cycle through one for each repo. It sounds like the Git plugin is behaving as expected, and the issue lies with how the Multibranch Pipeline plugin handles the branches. Would you agree that this should be logged against that plugin instead?   [later] Looks like the Branch API is responsible:     public String getName() {         // TODO this could include a uniquifying prefix defined in BranchSource         return head.getName();     }
          Hide
          markewaite Mark Waite added a comment -

          I suspect that changes will be needed in multiple plugins to support your use case.

          I don't think changes are needed in the git plugin, but I would expect changes would be needed in the branch api, and possibly in the branch source plugins and the organization folder plugin.

          Show
          markewaite Mark Waite added a comment - I suspect that changes will be needed in multiple plugins to support your use case. I don't think changes are needed in the git plugin, but I would expect changes would be needed in the branch api, and possibly in the branch source plugins and the organization folder plugin.
          Hide
          povserok Rok Povse added a comment -

          We would really need this issue to be fixed. Could you please provide us with a solution?

          Show
          povserok Rok Povse added a comment - We would really need this issue to be fixed. Could you please provide us with a solution?

            People

            • Assignee:
              abayer Andrew Bayer
              Reporter:
              evildeece Alastair D'Silva
            • Votes:
              13 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated: