Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45489

checkout(scm) step can return wrong variables when used following another Git checkout

    Details

    • Similar Issues:

      Description

      This looks like the same type of code-path issue described in https://issues.jenkins-ci.org/browse/JENKINS-41996 

       

      @Library('someGitLibrary')
      
      node {
        final scmVars = checkout(scm)
        // scmVars may have Git data from either the someGitLibrary or the scmVars
       }

       

      Looks like its caused by https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/GitSCM.java#L1282-L1317 

       

        Attachments

          Issue Links

            Activity

            Hide
            markewaite Mark Waite added a comment - - edited

            Thanks for the bug report. I think that I've started to duplicate the bug conditions in the JENKINS-45489 job in my jenkins-bugs repository. At a minimum, I'm seeing surprising results where the return from the checkout() task includes a GIT_COMMITER_NAME which seems to come from the library, while the GIT_COMMIT seems to come from the commit in the intended repository. I haven't yet duplicated the precise result you're describing, but I'm exploring on that branch to understand the problem better.

            The GIT_COMMIT values match what I expected (taken from the checkout, not from the library), while the values of the author and committer related variables seem to come from the library. Specifically, I see the following values that seem correct:

            GIT_BRANCH=JENKINS-45489-original/JENKINS-45489
            GIT_COMMIT=477b365e6c984818825595bef528f13ee17bff02
            GIT_PREVIOUS_COMMIT=f16765f4165e900fea417aae57e30586898aa15e
            GIT_PREVIOUS_SUCCESSFUL_COMMIT=f16765f4165e900fea417aae57e30586898aa15e
            GIT_URL=https://github.com/MarkEWaite/jenkins-bugs
            

            and the following values seem incorrect:

            GIT_AUTHOR_EMAIL=jenkins@mark-pc2.markwaite.net
            GIT_AUTHOR_NAME=Jenkins Docker User
            GIT_COMMITTER_EMAIL=jenkins@mark-pc2.markwaite.net
            GIT_COMMITTER_NAME=Jenkins Docker User
            

            Do those results match with what you're seeing?

            Show
            markewaite Mark Waite added a comment - - edited Thanks for the bug report. I think that I've started to duplicate the bug conditions in the JENKINS-45489 job in my jenkins-bugs repository. At a minimum, I'm seeing surprising results where the return from the checkout() task includes a GIT_COMMITER_NAME which seems to come from the library, while the GIT_COMMIT seems to come from the commit in the intended repository. I haven't yet duplicated the precise result you're describing, but I'm exploring on that branch to understand the problem better. The GIT_COMMIT values match what I expected (taken from the checkout, not from the library), while the values of the author and committer related variables seem to come from the library. Specifically, I see the following values that seem correct: GIT_BRANCH=JENKINS-45489-original/JENKINS-45489 GIT_COMMIT=477b365e6c984818825595bef528f13ee17bff02 GIT_PREVIOUS_COMMIT=f16765f4165e900fea417aae57e30586898aa15e GIT_PREVIOUS_SUCCESSFUL_COMMIT=f16765f4165e900fea417aae57e30586898aa15e GIT_URL=https://github.com/MarkEWaite/jenkins-bugs and the following values seem incorrect: GIT_AUTHOR_EMAIL=jenkins@mark-pc2.markwaite.net GIT_AUTHOR_NAME=Jenkins Docker User GIT_COMMITTER_EMAIL=jenkins@mark-pc2.markwaite.net GIT_COMMITTER_NAME=Jenkins Docker User Do those results match with what you're seeing?
            Hide
            mkobit Mike Kobit added a comment -

            I have not seen that exactly, but I can try again tomorrow. I can provide the exact plugin and Jenkins version because maybe order matters?

            Show
            mkobit Mike Kobit added a comment - I have not seen that exactly, but I can try again tomorrow. I can provide the exact plugin and Jenkins version because maybe order matters?
            Hide
            mkobit Mike Kobit added a comment -

            Sorry, I never got around to getting an exact reproduction case but I see very similar results to yours where the GIT_ variables returned are non-deterministic.

            Show
            mkobit Mike Kobit added a comment - Sorry, I never got around to getting an exact reproduction case but I see very similar results to yours where the GIT_ variables returned are non-deterministic.
            Hide
            mstoykov mihail stoykov added a comment -

            I can confirm that I'm seeing this with git-plugin 3.5.0 and jenkins 2.60.2 and it seems to be pretty consistent on a particular pipeline job, where I have now replayed the job 10 times with the same output.

            I don't see any indication that the variables are mixed between both repositories, all of them seem to be from the one I import the "library" from.

            In my setup I have a jenkinsfile from a repository which includes a "library" file from the same repository, through the usage of fileloader.withGit. The actual code that I'm building/testing and am checking out with checkout step is a second repository.

            Show
            mstoykov mihail stoykov added a comment - I can confirm that I'm seeing this with git-plugin 3.5.0 and jenkins 2.60.2 and it seems to be pretty consistent on a particular pipeline job, where I have now replayed the job 10 times with the same output. I don't see any indication that the variables are mixed between both repositories, all of them seem to be from the one I import the "library" from. In my setup I have a jenkinsfile from a repository which includes a "library" file from the same repository, through the usage of fileloader.withGit. The actual code that I'm building/testing and am checking out with checkout step is a second repository.
            Hide
            kakapo4 Mark Wright added a comment - - edited

            The issue is repeatable without involving shared-libraries. Simply by having an SCM pipeline script which checks out a different revision to that used to load the pipeline sciprt. The checkout step returns the GIT_COMMIT of the original checkout and not the new checkout.

            I've also seen this manifested in my multibranch pipeline jobs, but it is best demonstrated by a simple parameterised pipeline.

            Create a Pipeline job configured with "refs/heads/master" as the branch specifier. Give the job a single "TAG" parameter, providing a tag to be checked out by the pipeline. Use this script:

            pipeline
            {
                agent any
                environment
                {
                    git_tool = tool(name: 'Default', type: 'git')
                }
                options
                {
                    skipDefaultCheckout()
                }
                stages
                {
                    stage('Checkout')
                    {
                        steps
                        {
                            script
                            {
                                def buildTag = env.TAG
                                echo "Building against tag '${buildTag}'"
            
                                def newScm = [$class: 'GitSCM', userRemoteConfigs: scm.userRemoteConfigs, branches: [[name: "refs/tags/${buildTag}"]]]
            
                                def checkoutDetails = checkout scm: newScm, poll: false, changelog: false
            
                                echo "checkout scm returned SHA = ${checkoutDetails.GIT_COMMIT}"
                                bat """git status"""
                            }
                        }
                    }
                }
            }
            

            When run with a tag at the tip of the specified branch (master) we see the returned SHA is as expected:

            Building against tag 'tag3'
            [Pipeline] checkout
             > C:\Program Files\Git\cmd\git.exe rev-parse --is-inside-work-tree # timeout=120
            Fetching changes from the remote Git repository
             > C:\Program Files\Git\cmd\git.exe config remote.origin.url git@github.aus.thenational.com:P640806/jenkins_testing.git # timeout=120
            Fetching upstream changes from git@github.aus.thenational.com:P640806/jenkins_testing.git
             > C:\Program Files\Git\cmd\git.exe --version # timeout=120
            using GIT_SSH to set credentials 
             > C:\Program Files\Git\cmd\git.exe fetch --tags --progress git@github.aus.thenational.com:P640806/jenkins_testing.git +refs/heads/*:refs/remotes/origin/*
             > C:\Program Files\Git\cmd\git.exe rev-parse "refs/tags/tag3^{commit}" # timeout=120
             > C:\Program Files\Git\cmd\git.exe rev-parse "refs/remotes/origin/refs/tags/tag3^{commit}" # timeout=120
            Checking out Revision e03409cea735e7353e421551c6bd5963f436c38a (refs/tags/tag3)
             > C:\Program Files\Git\cmd\git.exe config core.sparsecheckout # timeout=120
             > C:\Program Files\Git\cmd\git.exe checkout -f e03409cea735e7353e421551c6bd5963f436c38a
            Commit message: "Removed env.GIT_COMMIT usage"
            [Pipeline] echo
            checkout scm returned SHA = e03409cea735e7353e421551c6bd5963f436c38a
            [Pipeline] bat
            [test_scm_issue] Running batch script
            
            d:\jenkins\ws\test_scm_issue>git status 
            HEAD detached at e03409c
            nothing to commit, working tree clean
            

            If an additional commit is added to master (so the tag is no longer at the tip), we see the correct revision (e03409c) is checked out, however the revision state returned by the checkout step is the new tip of master (5e70b9e903582cd1055e60b819c6f2f09f01aee7):

            Building against tag 'tag3'
            [Pipeline] checkout
             > C:\Program Files\Git\cmd\git.exe rev-parse --is-inside-work-tree # timeout=120
            Fetching changes from the remote Git repository
             > C:\Program Files\Git\cmd\git.exe config remote.origin.url git@github.aus.thenational.com:P640806/jenkins_testing.git # timeout=120
            Fetching upstream changes from git@github.aus.thenational.com:P640806/jenkins_testing.git
             > C:\Program Files\Git\cmd\git.exe --version # timeout=120
            using GIT_SSH to set credentials 
             > C:\Program Files\Git\cmd\git.exe fetch --tags --progress git@github.aus.thenational.com:P640806/jenkins_testing.git +refs/heads/*:refs/remotes/origin/*
             > C:\Program Files\Git\cmd\git.exe rev-parse "refs/tags/tag3^{commit}" # timeout=120
             > C:\Program Files\Git\cmd\git.exe rev-parse "refs/remotes/origin/refs/tags/tag3^{commit}" # timeout=120
            Checking out Revision e03409cea735e7353e421551c6bd5963f436c38a (refs/tags/tag3)
             > C:\Program Files\Git\cmd\git.exe config core.sparsecheckout # timeout=120
             > C:\Program Files\Git\cmd\git.exe checkout -f e03409cea735e7353e421551c6bd5963f436c38a
            Commit message: "Removed env.GIT_COMMIT usage"
            [Pipeline] echo
            checkout scm returned SHA = 5e70b9e903582cd1055e60b819c6f2f09f01aee7
            [Pipeline] bat
            [test_scm_issue] Running batch script
            
            d:\jenkins\ws\test_scm_issue>git status 
            HEAD detached at e03409c
            nothing to commit, working tree clean
            
            Show
            kakapo4 Mark Wright added a comment - - edited The issue is repeatable without involving shared-libraries. Simply by having an SCM pipeline script which checks out a different revision to that used to load the pipeline sciprt. The checkout step returns the GIT_COMMIT of the original checkout and not the new checkout. I've also seen this manifested in my multibranch pipeline jobs, but it is best demonstrated by a simple parameterised pipeline. Create a Pipeline job configured with " refs/heads/master " as the branch specifier. Give the job a single "TAG" parameter, providing a tag to be checked out by the pipeline. Use this script: pipeline { agent any environment { git_tool = tool(name: 'Default' , type: 'git' ) } options { skipDefaultCheckout() } stages { stage( 'Checkout' ) { steps { script { def buildTag = env.TAG echo "Building against tag '${buildTag}' " def newScm = [$class: 'GitSCM' , userRemoteConfigs: scm.userRemoteConfigs, branches: [[name: "refs/tags/${buildTag}" ]]] def checkoutDetails = checkout scm: newScm, poll: false , changelog: false echo "checkout scm returned SHA = ${checkoutDetails.GIT_COMMIT}" bat """git status" "" } } } } } When run with a tag at the tip of the specified branch (master) we see the returned SHA is as expected: Building against tag 'tag3' [Pipeline] checkout > C:\Program Files\Git\cmd\git.exe rev-parse --is-inside-work-tree # timeout=120 Fetching changes from the remote Git repository > C:\Program Files\Git\cmd\git.exe config remote.origin.url git@github.aus.thenational.com:P640806/jenkins_testing.git # timeout=120 Fetching upstream changes from git@github.aus.thenational.com:P640806/jenkins_testing.git > C:\Program Files\Git\cmd\git.exe --version # timeout=120 using GIT_SSH to set credentials > C:\Program Files\Git\cmd\git.exe fetch --tags --progress git@github.aus.thenational.com:P640806/jenkins_testing.git +refs/heads/*:refs/remotes/origin/* > C:\Program Files\Git\cmd\git.exe rev-parse "refs/tags/tag3^{commit}" # timeout=120 > C:\Program Files\Git\cmd\git.exe rev-parse "refs/remotes/origin/refs/tags/tag3^{commit}" # timeout=120 Checking out Revision e03409cea735e7353e421551c6bd5963f436c38a (refs/tags/tag3) > C:\Program Files\Git\cmd\git.exe config core.sparsecheckout # timeout=120 > C:\Program Files\Git\cmd\git.exe checkout -f e03409cea735e7353e421551c6bd5963f436c38a Commit message: "Removed env.GIT_COMMIT usage" [Pipeline] echo checkout scm returned SHA = e03409cea735e7353e421551c6bd5963f436c38a [Pipeline] bat [test_scm_issue] Running batch script d:\jenkins\ws\test_scm_issue>git status HEAD detached at e03409c nothing to commit, working tree clean If an additional commit is added to master (so the tag is no longer at the tip), we see the correct revision (e03409c) is checked out, however the revision state returned by the checkout step is the new tip of master (5e70b9e903582cd1055e60b819c6f2f09f01aee7): Building against tag 'tag3' [Pipeline] checkout > C:\Program Files\Git\cmd\git.exe rev-parse --is-inside-work-tree # timeout=120 Fetching changes from the remote Git repository > C:\Program Files\Git\cmd\git.exe config remote.origin.url git@github.aus.thenational.com:P640806/jenkins_testing.git # timeout=120 Fetching upstream changes from git@github.aus.thenational.com:P640806/jenkins_testing.git > C:\Program Files\Git\cmd\git.exe --version # timeout=120 using GIT_SSH to set credentials > C:\Program Files\Git\cmd\git.exe fetch --tags --progress git@github.aus.thenational.com:P640806/jenkins_testing.git +refs/heads/*:refs/remotes/origin/* > C:\Program Files\Git\cmd\git.exe rev-parse "refs/tags/tag3^{commit}" # timeout=120 > C:\Program Files\Git\cmd\git.exe rev-parse "refs/remotes/origin/refs/tags/tag3^{commit}" # timeout=120 Checking out Revision e03409cea735e7353e421551c6bd5963f436c38a (refs/tags/tag3) > C:\Program Files\Git\cmd\git.exe config core.sparsecheckout # timeout=120 > C:\Program Files\Git\cmd\git.exe checkout -f e03409cea735e7353e421551c6bd5963f436c38a Commit message: "Removed env.GIT_COMMIT usage" [Pipeline] echo checkout scm returned SHA = 5e70b9e903582cd1055e60b819c6f2f09f01aee7 [Pipeline] bat [test_scm_issue] Running batch script d:\jenkins\ws\test_scm_issue>git status HEAD detached at e03409c nothing to commit, working tree clean
            Hide
            rjbwtr Richard Bowater added a comment - - edited

            I've just run up against this too.

            Fetching changes from the remote Git repository
            Fetching without tags
            Checking out Revision 5cc3005a298d50fa0c2bc919293fcf3b3ed936de (print-env)
            Commit message: "Print more env"
            
            GIT_COMMIT=eb65cd4f46c828362429cbbd2d904f114bb4b801
            GIT_BRANCH=master
            

            Doesn't seem to be an obvious way to work around it? My particular use case is:

            • a global shared library configured to checkout from git
            • a multibranch pipeline configured to load Jenkinsfiles from the shared libraries above

            We're using pipelines to build and test shared libraries before deployment, but branch builds are being unpredictable due to the environment variables being incorrect.

            Show
            rjbwtr Richard Bowater added a comment - - edited I've just run up against this too. Fetching changes from the remote Git repository Fetching without tags Checking out Revision 5cc3005a298d50fa0c2bc919293fcf3b3ed936de (print-env) Commit message: "Print more env" GIT_COMMIT=eb65cd4f46c828362429cbbd2d904f114bb4b801 GIT_BRANCH=master Doesn't seem to be an obvious way to work around it? My particular use case is: a global shared library configured to checkout from git a multibranch pipeline configured to load Jenkinsfiles from the shared libraries above We're using pipelines to build and test shared libraries before deployment, but branch builds are being unpredictable due to the environment variables being incorrect.
            Hide
            markewaite Mark Waite added a comment - - edited

            Richard Bowater the most direct work around is to compute the SHA1 of the working directory with a call to a shell script, batch script, or powershell script. Refer to the example in a stackoverflow article.

            The technique is also in the pipeline script utilities that I use.

            Show
            markewaite Mark Waite added a comment - - edited Richard Bowater the most direct work around is to compute the SHA1 of the working directory with a call to a shell script, batch script, or powershell script. Refer to the example in a stackoverflow article . The technique is also in the pipeline script utilities that I use.
            Hide
            kakapo4 Mark Wright added a comment -

            Richard Bowater, on (Windows) build-server, we use the following in our scripts as Mark Waite suggests:

            def sha = bat(returnStdout: true, script: '@git rev-parse HEAD').trim() 
            Show
            kakapo4 Mark Wright added a comment - Richard Bowater , on (Windows) build-server, we use the following in our scripts as Mark Waite suggests: def sha = bat(returnStdout: true , script: '@git rev-parse HEAD' ).trim()

              People

              • Assignee:
                Unassigned
                Reporter:
                mkobit Mike Kobit
              • Votes:
                7 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: