Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43749

Support multiple Jenkinsfiles from the same repository

    Details

    • Similar Issues:

      Description

      This would support scenarios where different "configurations" of a pipeline cannot share the same Jenkinsfile.

      If I had multiple Jenkinsfiles... repository github.com/apple/swift

      /Package.jenkinsfile 
      /Incremental.jenkinsfile
      /Incremental-RA.jenkinsfile
      /Assert.jenkinsfile
      /src/…
      

      I would like to create multibranch Pipelines for each so I have the resulting structure:

      /Apple
      /Apple/Swift - Package
      /Apple/Swift - Incremental
      /Apple/Swift - Incremental-RA
      /Apple/Swfit - Assert
      

      Note that in this example I have an organization folder for github.com/apple and it is creating multiple multibranch pipelines for each Jenkinsfile discovered in each repository.

      I have written up examples and use cases in this doc

        Attachments

          Issue Links

            Activity

            Hide
            fajran Fajran Rusadi added a comment -

            What I ended up doing to support our monorepo approach was the following

            • Create separate (multibranch pipeline) Jenkins job for every individual components in the monorepo
            • All Jenkins jobs will use the same repository URL but each has its own path to Jenkinsfile. This way each component can have its own distinct build pipeline
            • I disabled the build triggering function in Jenkins and instead create an additional service to handle build trigger
            • The build trigger service also manages the Jenkins jobs. Whenever it detects a new component, it will create the corresponding Jenkins job automatically, with the correct configuration
            Show
            fajran Fajran Rusadi added a comment - What I ended up doing to support our monorepo approach was the following Create separate (multibranch pipeline) Jenkins job for every individual components in the monorepo All Jenkins jobs will use the same repository URL but each has its own path to Jenkinsfile. This way each component can have its own distinct build pipeline I disabled the build triggering function in Jenkins and instead create an additional service to handle build trigger The build trigger service also manages the Jenkins jobs. Whenever it detects a new component, it will create the corresponding Jenkins job automatically, with the correct configuration
            Hide
            j3p0uk Jon-Paul Sullivan added a comment -

            When creating multiple jobs off the same GitHub repository, I found we required this plugin to get sane PR checking: https://plugins.jenkins.io/github-scm-trait-notification-context

             

            Default behaviour can play havoc with CI pipelines.

             

            Show
            j3p0uk Jon-Paul Sullivan added a comment - When creating multiple jobs off the same GitHub repository, I found we required this plugin to get sane PR checking: https://plugins.jenkins.io/github-scm-trait-notification-context   Default behaviour can play havoc with CI pipelines.  
            Hide
            tzach Tzach Yarimi added a comment -

            Fajran Rusadi is your service creating jobs inside the multibranch folders? Can you please share some code for doing that?

            Show
            tzach Tzach Yarimi added a comment - Fajran Rusadi is your service creating jobs inside the multibranch folders? Can you please share some code for doing that?
            Hide
            fajran Fajran Rusadi added a comment -

            Tzach Yarimi to clarify, we had different multibranch pipeline job for different component. So what service does is create a new multibranch pipeline job when it detects a new component added to the repository. The new job will be configured to use the Jenkinsfile specific of that new component. 

            If the job already exists, it will check if the branch is already registered otherwise it will update the job config to add a new branch and start the scanning process to make the branch job available. Once the branch job is available, the service will trigger the build of that.

            Unfortunately I could not share the code.

            Show
            fajran Fajran Rusadi added a comment - Tzach Yarimi to clarify, we had different multibranch pipeline job for different component. So what service does is create a new multibranch pipeline job when it detects a new component added to the repository. The new job will be configured to use the Jenkinsfile specific of that new component.  If the job already exists, it will check if the branch is already registered otherwise it will update the job config to add a new branch and start the scanning process to make the branch job available. Once the branch job is available, the service will trigger the build of that. Unfortunately I could not share the code.
            Hide
            ceztko Francesco Pretto added a comment - - edited

            I'm relatively new to Jenkins but for me the reason this feature can't just be straightforwardly implemented in Jenkins is that it lacks a concept of a first-citizen SCM resource, meaning that SCM repositories can be configured in jobs but they are not treated as shareable resources themselves. If repositories could be configured as shareable resources (with no build steps configurable in them) and job could be configured to source on these resources (instead of configuring a separate git/svn connection for each job) it would be easier to implement a single queue of jobs to trigger for SCM changes on a single repository and it would be easier to filter which pipeline to trigger in case Job1 is configured with Jenkinsfile1 and Job2 is configured with JenkinsFile2. The single queue and the filter can probably be implemented with scripting but to have UI support and more declarative approach in the upstream pipelines (as opposed to imperatively select which job to trigger in the downstream SCM projects) something like what I am suggesting could be needed.

            Show
            ceztko Francesco Pretto added a comment - - edited I'm relatively new to Jenkins but for me the reason this feature can't just be straightforwardly implemented in Jenkins is that it lacks a concept of a first-citizen SCM resource, meaning that SCM repositories can be configured in jobs but they are not treated as shareable resources themselves. If repositories could be configured as shareable resources (with no build steps configurable in them) and job could be configured to source on these resources (instead of configuring a separate git/svn connection for each job) it would be easier to implement a single queue of jobs to trigger for SCM changes on a single repository and it would be easier to filter which pipeline to trigger in case Job1 is configured with Jenkinsfile1 and Job2 is configured with JenkinsFile2. The single queue and the filter can probably be implemented with  scripting  but to have UI support and more declarative approach in the upstream pipelines (as opposed to imperatively select which job to trigger in the downstream SCM projects) something like what I am suggesting could be needed.

              People

              • Assignee:
                Unassigned
                Reporter:
                jamesdumay James Dumay
              • Votes:
                89 Vote for this issue
                Watchers:
                107 Start watching this issue

                Dates

                • Created:
                  Updated: