Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34931

Build merge commit instead of PR head

    Details

    • Similar Issues:

      Description

      Add a configuration option (active by default maybe) to build an artificial merge commit generated by merging the PR head commit and the target branch commit.

        Attachments

          Issue Links

            Activity

            Hide
            abayer Andrew Bayer added a comment -

            There's interest in being able to have webhook-triggered non-PR builds build the specific commit from the event rather than HEAD as well.

            Show
            abayer Andrew Bayer added a comment - There's interest in being able to have webhook-triggered non-PR builds build the specific commit from the event rather than HEAD as well.
            Hide
            dodoent Nenad Miksa added a comment -

            We also need this feature. Currently we have a workaround which creates a local merge commit if env.BRANCH_NAME.startsWith("PR-"). It would be better if this is done automatically.

            Show
            dodoent Nenad Miksa added a comment - We also need this feature. Currently we have a workaround which creates a local merge commit if env.BRANCH_NAME.startsWith("PR-"). It would be better if this is done automatically.
            Hide
            kevin_smets Kevin Smets added a comment -

            Nenad Miksa how can you get the target branch to merge with? Haven't found a solution for that so I would be happy to use your workaround for the time being .

            Show
            kevin_smets Kevin Smets added a comment - Nenad Miksa how can you get the target branch to merge with? Haven't found a solution for that so I would be happy to use your workaround for the time being .
            Hide
            dodoent Nenad Miksa added a comment -

            Kevin Smets, its kind of hardcoded with forward compatibility with current bug in bitbucket branch source plugin :

            // hardcoded global variable
            def targetBranch = "master"
            
            // then later in node before doing a checkout:
            if(env.CHANGE_TARGET != null) {
                targetBranch = env.CHANGE_TARGET
            }
            

            Here is the entire workaround, which uses merge before build when building pull request and also using different fastforward strategies based on global nonFFBranches set which is hardcoded in Jenkinsfile (it would be nice to be able to configure that in plugin settings):

            node {
                        if(env.BRANCH_NAME.startsWith("PR-")) {
                            if(env.CHANGE_TARGET != null) {
                                targetBranch = env.CHANGE_TARGET
                            }
                            echo "Source branches: ${scm.branches[0].name}"
            
                            def ffMode = 'FF_ONLY'
                            if(nonFFBranches.contains(scm.branches[0].name)) {
                                ffMode = 'NO_FF'
                            }
                            // do a full clone with merge before build when building pull request
                            checkout([
                                $class: 'GitSCM',
                                branches: scm.branches,
                                doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
                                extensions: scm.extensions + [[$class: 'CloneOption', noTags: false, reference: '', shallow: false, depth: 0, timeout: 60], [$class: 'PruneStaleBranch'], [$class: 'CheckoutOption', timeout: 60], [$class: 'PreBuildMerge', options: [mergeRemote: 'origin', mergeTarget: targetBranch, fastForwardMode: ffMode]]],
                                submoduleCfg: [],
                                userRemoteConfigs: scm.userRemoteConfigs,
                                browser: [$class: 'BitbucketWeb', repoUrl: 'https://bitbucket.org/microblink/core']
                              ])
                        } else {
                            // do a shallow clone when building feature branch HEAD
                            checkout([
                                $class: 'GitSCM',
                                branches: scm.branches,
                                doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
                                extensions: scm.extensions + [[$class: 'CloneOption', noTags: false, reference: '', shallow: true, depth: 1, timeout: 60], [$class: 'PruneStaleBranch'], [$class: 'CheckoutOption', timeout: 60]],
                                submoduleCfg: [],
                                userRemoteConfigs: scm.userRemoteConfigs,
                                browser: [$class: 'BitbucketWeb', repoUrl: 'https://bitbucket.org/microblink/core']
                              ])
                        }
            }
            

            This workaround requires approval of certain methods in In-Process Script Approval Jenkins settings (namely, access to scm variable getters).

            Show
            dodoent Nenad Miksa added a comment - Kevin Smets , its kind of hardcoded with forward compatibility with current bug in bitbucket branch source plugin : // hardcoded global variable def targetBranch = "master" // then later in node before doing a checkout: if (env.CHANGE_TARGET != null ) { targetBranch = env.CHANGE_TARGET } Here is the entire workaround, which uses merge before build when building pull request and also using different fastforward strategies based on global nonFFBranches set which is hardcoded in Jenkinsfile (it would be nice to be able to configure that in plugin settings): node { if (env.BRANCH_NAME.startsWith( "PR-" )) { if (env.CHANGE_TARGET != null ) { targetBranch = env.CHANGE_TARGET } echo "Source branches: ${scm.branches[0].name}" def ffMode = 'FF_ONLY' if (nonFFBranches.contains(scm.branches[0].name)) { ffMode = 'NO_FF' } // do a full clone with merge before build when building pull request checkout([ $class: 'GitSCM' , branches: scm.branches, doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations, extensions: scm.extensions + [[$class: 'CloneOption' , noTags: false , reference: '', shallow: false , depth: 0, timeout: 60], [$class: ' PruneStaleBranch '], [$class: ' CheckoutOption ', timeout: 60], [$class: ' PreBuildMerge ', options: [mergeRemote: ' origin', mergeTarget: targetBranch, fastForwardMode: ffMode]]], submoduleCfg: [], userRemoteConfigs: scm.userRemoteConfigs, browser: [$class: 'BitbucketWeb' , repoUrl: 'https: //bitbucket.org/microblink/core' ] ]) } else { // do a shallow clone when building feature branch HEAD checkout([ $class: 'GitSCM' , branches: scm.branches, doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations, extensions: scm.extensions + [[$class: 'CloneOption' , noTags: false , reference: '', shallow: true , depth: 1, timeout: 60], [$class: ' PruneStaleBranch '], [$class: ' CheckoutOption', timeout: 60]], submoduleCfg: [], userRemoteConfigs: scm.userRemoteConfigs, browser: [$class: 'BitbucketWeb' , repoUrl: 'https: //bitbucket.org/microblink/core' ] ]) } } This workaround requires approval of certain methods in In-Process Script Approval Jenkins settings (namely, access to scm variable getters).
            Hide
            dodoent Nenad Miksa added a comment -

            Kevin Smets, we have updated our workaround to work correctly, i.e. it can now resolve the PR target. We use curl and jsawk with Bitbucket API to get info about pull request in question. Here is the code:

            def getPullRequestID() {
                return env.BRANCH_NAME.substring(3)
            }
            
            def obtainBitbucketToken() {
                try {
                    echo "Obtaining BitBucket Access Token"
                    sh "curl -s -X POST https://bitbucket.org/site/oauth2/access_token -u \"${getBitbucketOAuthKey()}:${getBitbucketOAuthSecret()}\" -d grant_type=client_credentials | jsawk 'return this.access_token' | tr -d \"\\n\" > accessToken.txt"
            
                    def accessToken = readFile 'accessToken.txt'
            
                    echo "BitBucket Token request response: ${accessToken}"
            
                    return accessToken
                } catch (error) {
                    echo "Failed to obtain BitBucket access token!"
                    return null
                }
            }
            
            def getPullRequestTargetAndSourceCommit(String accessToken, String repository, String pullRequestID) {
                try {
                    sh "curl -s -X GET https://api.bitbucket.org/2.0/repositories/microblink/${repository}/pullrequests/${pullRequestID} -H \"Authorization: Bearer ${accessToken}\" > pullRequestInfo.json"
                    sh "cat pullRequestInfo.json | jsawk 'return this.source.commit.hash' | tr -d '\\n' > commitID.txt"
                    sh "cat pullRequestInfo.json | jsawk 'return this.destination.branch.name' | tr -d '\\n' > destinationBranch.txt"
                    return [commitHash: readFile('commitID.txt'), destinationBranch: readFile('destinationBranch.txt')]
                } catch (error) {
                    echo "Failed to obtain pull request target and source commit. Reason: ${error}"
                    return null
                }
            }
            

            This can be used like this:

            def accessToken = obtainBitbucketToken()
            def pullRequestID = getPullRequestID()
            def prInfo = getPullRequestTargetAndSourceCommit(accessToken, repository, pullRequestID)
            
            def targetBranch = 'master'
            if (prInfo != null) {
                targetBranch = prInfo.destinationBranch
            }
            

            This code needs to be executed on unix node which has curl and jsawk tools installed. We tried parsing JSON with Groovy's builtin JsonSlurper, but it kept crashing with serialization errors even when used entirely from within function marked with @NonCPS.

            The getPullRequestTargetAndSourceCommit method also returns commit hash which can be used to notify build status to BitBucket, as described in this comment: https://issues.jenkins-ci.org/browse/JENKINS-33507?focusedCommentId=261926&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-261926

            By using curl to directly communicate with BitBucket API we have additional flexibility, like support for approving/unapproving pull requests and much more.

            Show
            dodoent Nenad Miksa added a comment - Kevin Smets , we have updated our workaround to work correctly, i.e. it can now resolve the PR target. We use curl and jsawk with Bitbucket API to get info about pull request in question. Here is the code: def getPullRequestID() { return env.BRANCH_NAME.substring(3) } def obtainBitbucketToken() { try { echo "Obtaining BitBucket Access Token" sh "curl -s -X POST https: //bitbucket.org/site/oauth2/access_token -u \" ${getBitbucketOAuthKey()}:${getBitbucketOAuthSecret()}\ " -d grant_type=client_credentials | jsawk ' return this .access_token' | tr -d \" \\n\ " > accessToken.txt" def accessToken = readFile 'accessToken.txt' echo "BitBucket Token request response: ${accessToken}" return accessToken } catch (error) { echo "Failed to obtain BitBucket access token!" return null } } def getPullRequestTargetAndSourceCommit( String accessToken, String repository, String pullRequestID) { try { sh "curl -s -X GET https: //api.bitbucket.org/2.0/repositories/microblink/${repository}/pullrequests/${pullRequestID} -H \" Authorization: Bearer ${accessToken}\ " > pullRequestInfo.json" sh "cat pullRequestInfo.json | jsawk ' return this .source.commit.hash' | tr -d '\\n' > commitID.txt" sh "cat pullRequestInfo.json | jsawk ' return this .destination.branch.name' | tr -d '\\n' > destinationBranch.txt" return [commitHash: readFile( 'commitID.txt' ), destinationBranch: readFile( 'destinationBranch.txt' )] } catch (error) { echo "Failed to obtain pull request target and source commit. Reason: ${error}" return null } } This can be used like this: def accessToken = obtainBitbucketToken() def pullRequestID = getPullRequestID() def prInfo = getPullRequestTargetAndSourceCommit(accessToken, repository, pullRequestID) def targetBranch = 'master' if (prInfo != null ) { targetBranch = prInfo.destinationBranch } This code needs to be executed on unix node which has curl and jsawk tools installed. We tried parsing JSON with Groovy's builtin JsonSlurper, but it kept crashing with serialization errors even when used entirely from within function marked with @NonCPS. The getPullRequestTargetAndSourceCommit method also returns commit hash which can be used to notify build status to BitBucket, as described in this comment: https://issues.jenkins-ci.org/browse/JENKINS-33507?focusedCommentId=261926&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-261926 By using curl to directly communicate with BitBucket API we have additional flexibility, like support for approving/unapproving pull requests and much more.
            Hide
            nickbrown Nicholas Brown added a comment -

            Is this a duplicate of JENKINS-36283 ?

            Show
            nickbrown Nicholas Brown added a comment - Is this a duplicate of JENKINS-36283 ?
            Hide
            amuniz Antonio Muñiz added a comment -

            This is a subset of JENKINS-36283. Let's keep it around for now.

            Show
            amuniz Antonio Muñiz added a comment - This is a subset of JENKINS-36283 . Let's keep it around for now.
            Hide
            kevin_smets Kevin Smets added a comment -

            Nenad Miksa thank you very much for posting that! Looking into implementing this finally .

            Show
            kevin_smets Kevin Smets added a comment - Nenad Miksa thank you very much for posting that! Looking into implementing this finally .

              People

              • Assignee:
                amuniz Antonio Muñiz
                Reporter:
                amuniz Antonio Muñiz
              • Votes:
                6 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: