Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28335

Step to run Git commands w/ credentials & tool (was: GitPublisher support)

    Details

    • Similar Issues:

      Description

      It would be nice to be able to use the GitPublisher inside a workflow as it is possible in other jobs.

      This requires the plugin to be upgrade to Jenkins core 1.580.1+, to implement the jenkins.tasks.SimpleBuildStep in GitPublisher

        Attachments

          Issue Links

            Activity

            alecharp Adrien Lecharpentier created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Labels workflow
            Hide
            jglick Jesse Glick added a comment -

            I already considered this possibility but do not consider it straightforward. GitPublisher relies on obtaining the single GitSCM configured for the project; there is no equivalent concept for Workflow.

            Anyway the extensive hardcoded logic in this publisher probably does not make sense in an otherwise flexible script. More useful would be an independent Step which lets you run arbitrary git commands, integrates with Credentials for authentication, and lets you pick a GitTool (certainly CLI, perhaps also JGit).

            Show
            jglick Jesse Glick added a comment - I already considered this possibility but do not consider it straightforward. GitPublisher relies on obtaining the single GitSCM configured for the project; there is no equivalent concept for Workflow. Anyway the extensive hardcoded logic in this publisher probably does not make sense in an otherwise flexible script. More useful would be an independent Step which lets you run arbitrary git commands, integrates with Credentials for authentication, and lets you pick a GitTool (certainly CLI, perhaps also JGit).
            jglick Jesse Glick made changes -
            Summary Be able to use GitPublisher in a workflow description Be able to use GitPublisher in a workflow
            Issue Type Improvement [ 4 ] New Feature [ 2 ]
            Priority Trivial [ 5 ] Major [ 3 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            COMPATIBILITY.md
            http://jenkins-ci.org/commit/workflow-plugin/26beaf00bff3fc235cb73d53c77bf64b632fdc75
            Log:
            JENKINS-28335 Noting.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: COMPATIBILITY.md http://jenkins-ci.org/commit/workflow-plugin/26beaf00bff3fc235cb73d53c77bf64b632fdc75 Log: JENKINS-28335 Noting.
            jglick Jesse Glick made changes -
            Summary Be able to use GitPublisher in a workflow Step to run Git commands w/ credentials & tool (was: GitPublisher support)
            jglick Jesse Glick made changes -
            Assignee Nicolas De Loof [ ndeloof ]
            jglick Jesse Glick made changes -
            Link This issue depends on JENKINS-26055 [ JENKINS-26055 ]
            Hide
            djviking Sverre Moe added a comment - - edited

            Meanwhile it is possible to use Groovy Post-build within Pipeline (formerly Workflow).

            Using this script one could publish git tags:

            def build = manager.build
            def workspace = build.getWorkspace()
            def listener = manager.listener
            def environment = build.getEnvironment(listener)
            
            final def project = build.getProject()
            final def gitScm = project.getScm()
            final GitClient gitClient = gitScm.createClient(listener, environment, build, workspace);
            
            final def gitTagName = "TAG_NAME"
            final def comment = "COMMENT"
            final def remoteURI = new URIish("origin")
            
            gitClient.tag(gitTagName, comment)
            gitClient.push().tags(true).to(remoteURI).execute()
            
            Show
            djviking Sverre Moe added a comment - - edited Meanwhile it is possible to use Groovy Post-build within Pipeline (formerly Workflow). Using this script one could publish git tags: def build = manager.build def workspace = build.getWorkspace() def listener = manager.listener def environment = build.getEnvironment(listener) final def project = build.getProject() final def gitScm = project.getScm() final GitClient gitClient = gitScm.createClient(listener, environment, build, workspace); final def gitTagName = "TAG_NAME" final def comment = "COMMENT" final def remoteURI = new URIish( "origin" ) gitClient.tag(gitTagName, comment) gitClient.push().tags( true ).to(remoteURI).execute()
            Hide
            woland Alexander Siniouguine added a comment - - edited

            Sverre Moe How do you run postbuild from within jenkinsfile itself, the "manager" object is not available when it runs (the actual jenkinsfile is here: https://github.com/VirtoCommerce/vc-module-core/blob/eeee111fb33e12f93c9bf64988a193627dfb533a/Jenkinsfile)

            Ideally I'd like to commit the file changed during the build (version file).

            Show
            woland Alexander Siniouguine added a comment - - edited Sverre Moe How do you run postbuild from within jenkinsfile itself, the "manager" object is not available when it runs (the actual jenkinsfile is here: https://github.com/VirtoCommerce/vc-module-core/blob/eeee111fb33e12f93c9bf64988a193627dfb533a/Jenkinsfile ) Ideally I'd like to commit the file changed during the build (version file).
            woland Alexander Siniouguine made changes -
            Rank Ranked higher
            Hide
            arty13 Art V added a comment -

            I also had issues with this.. What I did is use the sshagent plugin and ran the git tag commands inside this. It may not be as flexible/robust as above, but an easier straight forward workaround until we have a better solution to get the GitSCM client. I have not done much testing but hope this helps you in the meantime.

            def call(String tag, String comment, String credentialsId, String repoName = 'origin') {
                sshagent([credentialsId]) {
                    sh("git tag -a -f -m '${comment}' ${tag}")
                    sh("git -c core.askpass=true push ${repoName} ${tag}")
                }
            }
            
            Show
            arty13 Art V added a comment - I also had issues with this.. What I did is use the sshagent plugin and ran the git tag commands inside this. It may not be as flexible/robust as above, but an easier straight forward workaround until we have a better solution to get the GitSCM client. I have not done much testing but hope this helps you in the meantime. def call( String tag, String comment, String credentialsId, String repoName = 'origin' ) { sshagent([credentialsId]) { sh( "git tag -a -f -m '${comment}' ${tag}" ) sh( "git -c core.askpass= true push ${repoName} ${tag}" ) } }
            Hide
            djviking Sverre Moe added a comment -

            How do you run postbuild from within jenkinsfile itself, the "manager" object is not available when it runs

            The manager object should be available if you have installed the Groovy Postbuild Plugin.

            Show
            djviking Sverre Moe added a comment - How do you run postbuild from within jenkinsfile itself, the "manager" object is not available when it runs The manager object should be available if you have installed the Groovy Postbuild Plugin.
            markewaite Mark Waite made changes -
            Link This issue is related to JENKINS-36496 [ JENKINS-36496 ]
            Hide
            mocsharp mocsharp added a comment - - edited

            Sverre Boschman I tried the code you've provided but am getting the following error:

            Build Failed: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getWorkspace
            
            org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getWorkspace
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:113)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146)
            	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:15)
            	at WorkflowScript.stageTagBuild(WorkflowScript:234)
            	at WorkflowScript.run(WorkflowScript:52)
            	at ___cps.transform___(Native Method)
            	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:55)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:74)
            	at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
            	at java.lang.reflect.Method.invoke(Unknown Source)
            	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
            	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
            	at com.cloudbees.groovy.cps.Next.step(Next.java:58)
            	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:32)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:29)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:29)
            	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:297)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$000(CpsThreadGroup.java:78)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:206)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:204)
            	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)
            	at java.util.concurrent.FutureTask.run(Unknown Source)
            	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
            	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
            	at java.util.concurrent.FutureTask.run(Unknown Source)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            	at java.lang.Thread.run(Unknown Source)
            

            Any idea? Thanks!

            Edit: nvm. I used pwd() instead.

            Edit: getting the following as well:

            Build Failed: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getProject
            
            Show
            mocsharp mocsharp added a comment - - edited Sverre Boschman I tried the code you've provided but am getting the following error: Build Failed: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getWorkspace org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getWorkspace at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:15) at WorkflowScript.stageTagBuild(WorkflowScript:234) at WorkflowScript.run(WorkflowScript:52) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:55) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:74) at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:58) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:32) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:29) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:29) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:297) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$000(CpsThreadGroup.java:78) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:206) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:204) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47) at java.util.concurrent.FutureTask.run(Unknown Source) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Any idea? Thanks! Edit: nvm. I used pwd() instead. Edit: getting the following as well: Build Failed: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getProject
            Hide
            rg Russell Gallop added a comment -

            The workaround published in https://github.com/jenkinsci/pipeline-examples/blob/master/pipeline-examples/push-git-repo/pushGitRepo.Groovy doesn't work with special characters in the password (e.g. @). You can get around that by URL encoding the password:

            withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'MyID', usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD']]) {
                String encoded_password = java.net.URLEncoder.encode(env.GIT_PASSWORD, "UTF-8")
                sh("git tag -a some_tag -m 'Jenkins'")
                sh("git push https://${env.GIT_USERNAME}:${encoded_password}@<REPO> --tags")
            }
            

            but this defeats the credential binding attempts to obscure the password in the console output. Could do with a proper fix.

            Show
            rg Russell Gallop added a comment - The workaround published in https://github.com/jenkinsci/pipeline-examples/blob/master/pipeline-examples/push-git-repo/pushGitRepo.Groovy doesn't work with special characters in the password (e.g. @). You can get around that by URL encoding the password: withCredentials([[$class: 'UsernamePasswordMultiBinding' , credentialsId: 'MyID' , usernameVariable: 'GIT_USERNAME' , passwordVariable: 'GIT_PASSWORD' ]]) { String encoded_password = java.net.URLEncoder.encode(env.GIT_PASSWORD, "UTF-8" ) sh( "git tag -a some_tag -m 'Jenkins' " ) sh( "git push https: //${env.GIT_USERNAME}:${encoded_password}@<REPO> --tags" ) } but this defeats the credential binding attempts to obscure the password in the console output. Could do with a proper fix.
            Hide
            leandro_freitas_softdevelop Leandro Kersting de Freitas added a comment -

            +1

            Show
            leandro_freitas_softdevelop Leandro Kersting de Freitas added a comment - +1
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 163171 ] JNJira + In-Review [ 181120 ]
            Hide
            bargemayur05 Mayur Barge added a comment -

            Hi Jesse Glick
            Greetings !
            Are there any plans to get GitPublisher working soon ?
            OR to fix the special characters issue mentioned by Russell Gallop

            Thanks Russell Gallop for the workaround. But as you mentioned it shows password in the console output. Any other suggestions ?

            Show
            bargemayur05 Mayur Barge added a comment - Hi Jesse Glick Greetings ! Are there any plans to get GitPublisher working soon ? OR to fix the special characters issue mentioned by Russell Gallop Thanks Russell Gallop for the workaround. But as you mentioned it shows password in the console output. Any other suggestions ?
            Hide
            slavik334 Viachaslau Kabak added a comment -

            Hello,
            need the pipeline functioning ASAP
            could you please provide such feature?
            thanks

            Show
            slavik334 Viachaslau Kabak added a comment - Hello, need the pipeline functioning ASAP could you please provide such feature? thanks
            abayer Andrew Bayer made changes -
            Labels workflow pipeline workflow
            abayer Andrew Bayer made changes -
            Labels pipeline workflow pipeline
            Hide
            dcsobral Daniel Sobral added a comment -

            Workaround:

                withCredentials([[$class  : 'FileBinding', credentialsId: 'your-credential-id',
                                  variable: 'CREDENTIALS']]) {
                    sh 'git config --local credential.username RepoUserName'
                    sh "git config --local credential.helper 'store --file=${env.CREDENTIALS}'"
            
                    sh "git do-your-stuff"
            
                    sh 'git config --local --remove-section credential'
                }
            

            See git man page for the git store credential for information on the file format.

            Show
            dcsobral Daniel Sobral added a comment - Workaround: withCredentials([[$class : 'FileBinding' , credentialsId: 'your-credential-id' , variable: 'CREDENTIALS' ]]) { sh 'git config --local credential.username RepoUserName' sh "git config --local credential.helper 'store --file=${env.CREDENTIALS}' " sh "git do -your-stuff" sh 'git config --local --remove-section credential' } See git man page for the git store credential for information on the file format.
            Hide
            dcsobral Daniel Sobral added a comment -

            Mind you, it doesn't work for submodules. Submodules have a fundamental flaw: they are repos on their own, so the credentials for the module that include them does not apply, but you can't set local credentials for them before they are fetched. Catch-22.

            For submodules, either use relative paths, SSH keys, or run a sed on the URL of .gitmodules.

            Show
            dcsobral Daniel Sobral added a comment - Mind you, it doesn't work for submodules. Submodules have a fundamental flaw: they are repos on their own, so the credentials for the module that include them does not apply, but you can't set local credentials for them before they are fetched. Catch-22. For submodules, either use relative paths, SSH keys, or run a sed on the URL of .gitmodules.
            Hide
            busata Dries De Smet added a comment - - edited

            Daniel Sobral Do you by any chance know why the following script: (which I found as an alternative to the one above), gives the following error?

            fatal: repository 'https://****:****@github.com/ourorg/therepo.git/' not found
            
            withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '48bfc941-310e-4a0b-bf87-15bc4c4e06cd',
                                        usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']]) {
                    sh "git checkout develop"
                    sh "git checkout master"
                    sh "git merge develop"
                    sh "git tag " + versionLabel
                    sh "git push 'https://${env.USERNAME}:${env.PASSWORD}@github.com/ourorg/therepo.git' --tags"
                }
            

            Locally, the command works fine with the ' , I triple checked spelling & tried different quotes, but in the pipeline it keeps saying that it can't find the repo.

            Show
            busata Dries De Smet added a comment - - edited Daniel Sobral Do you by any chance know why the following script: (which I found as an alternative to the one above), gives the following error? fatal: repository 'https: //****:****@github.com/ourorg/therepo.git/' not found withCredentials([[$class: 'UsernamePasswordMultiBinding' , credentialsId: '48bfc941-310e-4a0b-bf87-15bc4c4e06cd' , usernameVariable: 'USERNAME' , passwordVariable: 'PASSWORD' ]]) { sh "git checkout develop" sh "git checkout master" sh "git merge develop" sh "git tag " + versionLabel sh "git push 'https: //${env.USERNAME}:${env.PASSWORD}@github.com/ourorg/therepo.git' --tags" } Locally, the command works fine with the ' , I triple checked spelling & tried different quotes, but in the pipeline it keeps saying that it can't find the repo.
            Hide
            maneken Ruslan Bondarau added a comment - - edited
            withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '9e5r57b0-5649-3d65-rt41-e900d8c76774', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME']])
            {
                sh "git tag -a some_tag${env.BUILD_NUMBER} -m 'Jenkins'"
                sh ("git -c core.askpass=true push https://${env.GIT_USERNAME}:${env.GIT_PASSWORD}@github.com/some-tag/some-tag.git some_tag${env.BUILD_NUMBER}")
            }
            
            Show
            maneken Ruslan Bondarau added a comment - - edited withCredentials([[$class: 'UsernamePasswordMultiBinding' , credentialsId: '9e5r57b0-5649-3d65-rt41-e900d8c76774' , passwordVariable: 'GIT_PASSWORD' , usernameVariable: 'GIT_USERNAME' ]]) { sh "git tag -a some_tag${env.BUILD_NUMBER} -m 'Jenkins' " sh ( "git -c core.askpass= true push https: //${env.GIT_USERNAME}:${env.GIT_PASSWORD}@github.com/some-tag/some-tag.git some_tag${env.BUILD_NUMBER}" ) }
            Hide
            panga Leonardo Zanivan added a comment -

            There's a workaround using ssh-agent-plugin:

            sshagent(['git-credentials-id']) {
                  sh "git push origin master"
            }
            
            Show
            panga Leonardo Zanivan added a comment - There's a workaround using ssh-agent-plugin: sshagent([ 'git-credentials-id' ]) { sh "git push origin master" }
            Hide
            amaksoft Andrey Makeev added a comment - - edited

            I want to share my Jenkins Pipeline setup and my solution to publish changes/tags to git repo via SSH (While this task is under development). Please check it out for more info, any improvement ideas are welcome
            In short you just add file git_push_ssh.groovy to your project and call method pushSSH() from Jenkinsfile like this:

             env.BRANCH_NAME = "mycoolbranch"// BRANCH_NAME is predefined in multibranch pipeline job
             env.J_GIT_CONFIG = "true"
             env.J_USERNAME = "Jenkins CI"
             env.J_EMAIL = "jenkins-ci@example.com"
             env.J_CREDS_IDS = '02aa92ec-593e-4a90-ac85-3f43a06cfae3' // Use credentials id from Jenkins
             def gitLib = load "git_push_ssh.groovy"
             ...
             gitLib.pushSSH(commitMsg: "Jenkins build #${env.BUILD_NUMBER}", tagName: "build-${env.BUILD_NUMBER}", files: "changelog.txt someotherfile.txt");
            
            Show
            amaksoft Andrey Makeev added a comment - - edited I want to share my Jenkins Pipeline setup and my solution to publish changes/tags to git repo via SSH (While this task is under development). Please check it out for more info, any improvement ideas are welcome In short you just add file git_push_ssh.groovy to your project and call method pushSSH() from Jenkinsfile like this: env.BRANCH_NAME = "mycoolbranch" // BRANCH_NAME is predefined in multibranch pipeline job env.J_GIT_CONFIG = " true " env.J_USERNAME = "Jenkins CI" env.J_EMAIL = "jenkins-ci@example.com" env.J_CREDS_IDS = '02aa92ec-593e-4a90-ac85-3f43a06cfae3' // Use credentials id from Jenkins def gitLib = load "git_push_ssh.groovy" ... gitLib.pushSSH(commitMsg: "Jenkins build #${env.BUILD_NUMBER}" , tagName: "build-${env.BUILD_NUMBER}" , files: "changelog.txt someotherfile.txt" );
            Hide
            cowwoc cowwoc added a comment -

            Beware: ssh-agent will not currently work under windows: JENKINS-28279 ("git plugin", such as the "checkout" task will fail to authenticate)

            As far as I can tell, the only option under Windows is to pass the username/password to an HTTPS url: http://stackoverflow.com/a/33630506/14731

            Show
            cowwoc cowwoc added a comment - Beware: ssh-agent will not currently work under windows: JENKINS-28279 ("git plugin", such as the "checkout" task will fail to authenticate) As far as I can tell, the only option under Windows is to pass the username/password to an HTTPS url: http://stackoverflow.com/a/33630506/14731
            Hide
            jimjaeger jim jaeger added a comment -

            Any Updates for the usage of the GitPublisher in JenkinsFiles?

            Show
            jimjaeger jim jaeger added a comment - Any Updates for the usage of the GitPublisher in JenkinsFiles?
            Hide
            hogarthj James Hogarth added a comment -

            I like the files idea but we needed a user/pass version.

            My contribution and modification to this is:

            1. Our ansible deployment configures:
              - name: configure git username
                become_user: jenkins
                git_config:
                  scope: global
                  name: user.name
                  value: "{{ lookup('env','USER') }}"
              
              - name: configure git email
                become_user: jenkins
                git_config:
                  scope: global
                  name: user.email
                  value: "{{ lookup('env','USER') }}@{{ ansible_hostname }}"
              
              - name: enable git cache credential storage
                become_user: jenkins
                git_config:
                  scope: global
                  name: credential.helper
                  value: cache
              
            2. The jenkins pipeline contains:
              stage("tag the commit with datetime") {
                withCredentials([usernamePassword(credentialsId: 'my_cred_id' usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD')]) {
              
                // use date for tag
                def tag = new Date().format("yyyyMMddHHmm")
              
                // configure the git credentials, these are cached in RAM for several minutes to use
                // this is required until https://issues.jenkins-ci.org/browse/JENKINS-28335 is resolved upstream
                sh "echo 'protocol=https\nhost=<git-host-goes-here>\nusername=${GIT_USERNAME}\npassword=${GIT_PASSWORD}\n\n' | git credential approve "
              
                sh "git tag -a ${tag} -m '${USER} tagging'"
                sh "git push --tags"
                }
              }
              

            This example uses new Date() which needs whitelisting right now but could easily be BUILD_ID if you were sure that you'd never need to rebuild, and lose an incrementing BUILD_ID, or whatever other metric/value was preferred.

            Show
            hogarthj James Hogarth added a comment - I like the files idea but we needed a user/pass version. My contribution and modification to this is: Our ansible deployment configures: - name: configure git username become_user: jenkins git_config: scope: global name: user.name value: "{{ lookup( 'env' , 'USER' ) }}" - name: configure git email become_user: jenkins git_config: scope: global name: user.email value: "{{ lookup( 'env' , 'USER' ) }}@{{ ansible_hostname }}" - name: enable git cache credential storage become_user: jenkins git_config: scope: global name: credential.helper value: cache The jenkins pipeline contains: stage( "tag the commit with datetime" ) { withCredentials([usernamePassword(credentialsId: 'my_cred_id' usernameVariable: 'GIT_USERNAME' , passwordVariable: 'GIT_PASSWORD' )]) { // use date for tag def tag = new Date().format( "yyyyMMddHHmm" ) // configure the git credentials, these are cached in RAM for several minutes to use // this is required until https://issues.jenkins-ci.org/browse/JENKINS-28335 is resolved upstream sh "echo 'protocol=https\nhost=<git-host-goes-here>\nusername=${GIT_USERNAME}\npassword=${GIT_PASSWORD}\n\n' | git credential approve " sh "git tag -a ${tag} -m '${USER} tagging' " sh "git push --tags" } } This example uses new Date() which needs whitelisting right now but could easily be BUILD_ID if you were sure that you'd never need to rebuild, and lose an incrementing BUILD_ID, or whatever other metric/value was preferred.
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-42279 [ JENKINS-42279 ]
            Hide
            mocsharp mocsharp added a comment -

            Thank you James Hogarth

             

            For Windows, do

            bat "cmd /c echo protocol=https & echo.host=<git-host-goes-here> & echo.username=${GIT_USERNAME} & echo.password=${GIT_PASSWORD} | git credential approve "

            Note the dot after echo.

            Show
            mocsharp mocsharp added a comment - Thank you James Hogarth   For Windows, do bat "cmd /c echo protocol=https & echo.host=<git-host-goes-here> & echo.username=${GIT_USERNAME} & echo.password=${GIT_PASSWORD} | git credential approve " Note the dot after echo.
            Hide
            jglick Jesse Glick added a comment -

            I would not expect the stated RFE to be implemented as such. Rather, there can be a withGit block-scoped step which takes an installation name and/or credentials ID, defines suitable environment variables, and passes off control to nested steps like sh.

            Show
            jglick Jesse Glick added a comment - I would not expect the stated RFE to be implemented as such. Rather, there can be a withGit block-scoped step which takes an installation name and/or credentials ID, defines suitable environment variables, and passes off control to nested steps like sh .
            Hide
            medianick Nick Jones added a comment - - edited

            Indeed, Jesse Glick, what you described is essentially what I ended up doing (in declarative pipeline):

            environment {
              GITUSER = credentials('some_credential_id')
            }
            steps {
              sh 'git tag -a some.tag -m "some message"'
              sh 'git push https://${GITUSER_USR}:${GITUSER_PSW}@some.git.url some.tag'
            }

            I've omitted unimportant aspects but this basic approach is working for us.

            Show
            medianick Nick Jones added a comment - - edited Indeed, Jesse Glick , what you described is essentially what I ended up doing (in declarative pipeline): environment { GITUSER = credentials( 'some_credential_id' ) } steps { sh 'git tag -a some.tag -m "some message" ' sh 'git push https: //${GITUSER_USR}:${GITUSER_PSW}@some.git.url some.tag' } I've omitted unimportant aspects but this basic approach is working for us.
            Hide
            jglick Jesse Glick added a comment -

            Yes; the idea would be to allow you to write something along the lines of

            withGit(credentialsId: 'git-login') {
              sh 'git tag whatever && git push origin whatever'
            }
            Show
            jglick Jesse Glick added a comment - Yes; the idea would be to allow you to write something along the lines of withGit(credentialsId: 'git-login' ) { sh 'git tag whatever && git push origin whatever' }
            Hide
            darthgandalf Alexey Sokolov added a comment -

            FTR, the workaround I used until there is a better way:

            withCredentials([sshUserPrivateKey(credentialsId: '...', keyFileVariable: 'GITHUB_KEY')]) {
                sh 'echo ssh -i $GITHUB_KEY -l git -o StrictHostKeyChecking=no \\"\\$@\\" > run_ssh.sh'
                sh 'chmod +x run_ssh.sh'
                withEnv(['GIT_SSH=run_ssh.sh']) {
                    sh 'git push origin whatever'
                }
            }
            
            Show
            darthgandalf Alexey Sokolov added a comment - FTR, the workaround I used until there is a better way: withCredentials([sshUserPrivateKey(credentialsId: '...' , keyFileVariable: 'GITHUB_KEY' )]) { sh 'echo ssh -i $GITHUB_KEY -l git -o StrictHostKeyChecking=no \\ "\\$@\\" > run_ssh.sh' sh 'chmod +x run_ssh.sh' withEnv([ 'GIT_SSH=run_ssh.sh' ]) { sh 'git push origin whatever' } }
            medianick Nick Jones made changes -
            Link This issue is related to JENKINS-47733 [ JENKINS-47733 ]
            Hide
            apgray Andrew Gray added a comment -

            Is this going to be fixed anytime soon?

            Show
            apgray Andrew Gray added a comment - Is this going to be fixed anytime soon?
            Hide
            skbp Paul Skiba added a comment -

            I added this issue as a project idea for GSoC 2018 but I hope that this issue will be fixed before summer. I don't think that a student will take this project but hope dies last 

            Show
            skbp Paul Skiba added a comment - I added this issue as a project idea for GSoC 2018  but I hope that this issue will be fixed before summer. I don't think that a student will take this project but hope dies last 
            Hide
            apgray Andrew Gray added a comment -

            I find it amazing that there are still these massive glaring gaps in functionality (esp for Git) in critical functions all over declarative pipeline.

             

            They're supposed to be the future but not there yet.

            Show
            apgray Andrew Gray added a comment - I find it amazing that there are still these massive glaring gaps in functionality (esp for Git) in critical functions all over declarative pipeline.   They're supposed to be the future but not there yet.
            Hide
            kwiering Kyle Wiering added a comment -

            The CLI scripted work-around is functional. That is the problem. Andrew Gray.

            Show
            kwiering Kyle Wiering added a comment - The CLI scripted work-around is functional. That is the problem. Andrew Gray .
            Hide
            trane9991 Taras Postument added a comment - - edited

            A bit cleaner workaround here:

                withCredentials([sshUserPrivateKey(credentialsId: 'your-github-key-id', keyFileVariable: 'GITHUB_KEY')]) {
                    withEnv(["GIT_SSH_COMMAND=ssh -i $GITHUB_KEY -o StrictHostKeyChecking=no"]) {
                        git clone/push git@...
                    }
                }
            
            Show
            trane9991 Taras Postument added a comment - - edited A bit cleaner workaround here: withCredentials([sshUserPrivateKey(credentialsId: 'your-github-key-id' , keyFileVariable: 'GITHUB_KEY' )]) { withEnv([ "GIT_SSH_COMMAND=ssh -i $GITHUB_KEY -o StrictHostKeyChecking=no" ]) { git clone/push git@... } }
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            Its preferable to avoid writing secrets to the workspace. Although, ssh agent could theoretically be accessed from another worker by figuring out the SSH_AUTH_SOCK.

            withEnv(['GIT_SSH_COMMAND=ssh -o StrictHostKeyChecking=no']) {
              sshagent(credentials: ['your-github-key']) {
              ...
              }
            }
            
            Show
            jhoblitt Joshua Hoblitt added a comment - Its preferable to avoid writing secrets to the workspace. Although, ssh agent could theoretically be accessed from another worker by figuring out the SSH_AUTH_SOCK . withEnv([ 'GIT_SSH_COMMAND=ssh -o StrictHostKeyChecking=no' ]) { sshagent(credentials: [ 'your-github-key' ]) { ... } }
            Hide
            jglick Jesse Glick added a comment -

            The sshagent step is more convenient, but it boils down to more or less the same thing. withCredentials does not write files to the workspace per se, but to an @tmp sibling directory.

            Show
            jglick Jesse Glick added a comment - The sshagent step is more convenient, but it boils down to more or less the same thing. withCredentials does not write files to the workspace per se, but to an @tmp sibling directory.
            Hide
            taylor01 David Taylor added a comment -

            any updates would be appreciated. my team is working on pipelines and we need this plugin to be updated in order to push back to Bitbucket from windows without using ssh keys.

            Show
            taylor01 David Taylor added a comment - any updates would be appreciated. my team is working on pipelines and we need this plugin to be updated in order to push back to Bitbucket from windows without using ssh keys.
            Hide
            bicschneider Claus Schneider added a comment -

            You can consider using https://wiki.jenkins.io/display/JENKINS/Pretested+Integration+Plugin if it matches your needs. If not feel free to propose your missing needs..

            It is using the credentials from the Git SCM plugin and uses it in the "publisher". It supports http/windows ..

            Show
            bicschneider Claus Schneider added a comment - You can consider using https://wiki.jenkins.io/display/JENKINS/Pretested+Integration+Plugin  if it matches your needs. If not feel free to propose your missing needs.. It is using the credentials from the Git SCM plugin and uses it in the "publisher". It supports http/windows ..
            Hide
            taylor01 David Taylor added a comment -

            Claus Schneider thank you for the plugin recommendation. we will look at this and see what works!

            Show
            taylor01 David Taylor added a comment - Claus Schneider thank you for the plugin recommendation. we will look at this and see what works!
            Hide
            taylor01 David Taylor added a comment -

            Claus Schneider we've looked at the pre-tested plugin and we're not sure if it will fit with what we're trying to do. we really just need a way to push changes back to Bitbucket without using SSH keys from the build machine. The Git Plugin / Git Publisher plugin should allow for this but we're not sure how to implement it yet.

            Show
            taylor01 David Taylor added a comment - Claus Schneider we've looked at the pre-tested plugin and we're not sure if it will fit with what we're trying to do. we really just need a way to push changes back to Bitbucket without using SSH keys from the build machine. The Git Plugin / Git Publisher plugin should allow for this but we're not sure how to implement it yet.
            Hide
            jglick Jesse Glick added a comment -

            As already noted in this issue, you can already use withCredentials or sshagent to accomplish this use case. This enhancement would simply make it more convenient.

            Show
            jglick Jesse Glick added a comment - As already noted in this issue, you can already use withCredentials or sshagent to accomplish this use case. This enhancement would simply make it more convenient.
            Hide
            zeninr Ruslan Zenin added a comment - - edited

            Hi Jesse Glick . would it be possible to provide a sample pipeline script how we can accomplish "git push withCredentials or sshagent"? There is a use case for my team to build projects and change some of the artifacts and then push back to the SCM.
            Thank you in advance!
            Note: I found this example: https://jenkins.io/doc/pipeline/examples/#push-git-repo

            withCredentials([usernamePassword(credentialsId: 'git-pass-credentials-ID', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) { sh("git tag -a some_tag -m 'Jenkins'") sh('git push https://${GIT_USERNAME}:${GIT_PASSWORD}@<REPO> --tags') }

            Show
            zeninr Ruslan Zenin added a comment - - edited Hi Jesse Glick  . would it be possible to provide a sample pipeline script how we can accomplish "git push withCredentials or sshagent" ? There is a use case for my team to build projects and change some of the artifacts and then push back to the SCM. Thank you in advance! Note: I found this example: https://jenkins.io/doc/pipeline/examples/#push-git-repo withCredentials( [usernamePassword(credentialsId: 'git-pass-credentials-ID', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')] ) { sh("git tag -a some_tag -m 'Jenkins'") sh('git push https://$ {GIT_USERNAME}:${GIT_PASSWORD}@<REPO> --tags') }
            Hide
            kodstark Kamil Demecki added a comment -

            Ruslan Zenin Above there are examples working for me with ssh key. I didn't check how to use credentials with password if you are asking about this case specific.

            sshagent(['credentiald-id-using-ssh-key']) 
             {
                sh('git command or program calling git inside') 
             }
            
            Show
            kodstark Kamil Demecki added a comment - Ruslan Zenin Above there are examples working for me with ssh key. I didn't check how to use credentials with password if you are asking about this case specific. sshagent([ 'credentiald-id-using-ssh-key' ]) {     sh( 'git command or program calling git inside' ) }
            Hide
            zeninr Ruslan Zenin added a comment - - edited

            Kamil Demecki Thank you! That worked for me!

            Show
            zeninr Ruslan Zenin added a comment - - edited Kamil Demecki Thank you! That worked for me!
            Hide
            drdamour chirs damour added a comment - - edited

            we had been using the sshagent workaround, but recently changed to using the "Checkout over SSH" git scm custom behaviour.  This made the sshagent wrapper unecessary in the scripts (i'm guessing cause it basically wraps your whole pipeline as such..kind of). Not sure how portable this is to windows or non git environments but was easier for us.

            Also to control the commit/author email was used the Custom user name/e-mail address custom behaviour

            Show
            drdamour chirs damour added a comment - - edited we had been using the sshagent workaround, but recently changed to using the "Checkout over SSH" git scm custom behaviour.  This made the sshagent wrapper unecessary in the scripts (i'm guessing cause it basically wraps your whole pipeline as such..kind of). Not sure how portable this is to windows or non git environments but was easier for us. Also to control the commit/author email was used the  Custom user name/e-mail address  custom behaviour
            Hide
            matthiesenj Jesper Matthiesen added a comment - - edited

            I'm trying to get a post-build git push --tags to work on windows using user/pass with jenkins credentials outside of pipeline/workflow. The git plugin fetches from a remote private repo this way without problems, but I can't get authentication to work when invoking git through windows batch, or groovy as suggested in https://issues.jenkins-ci.org/browse/JENKINS-28335?focusedCommentId=258095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-258095 above. Shouldn't git operations through the git client in groovy work, if things like fetch, performed by the git plugin, work?

            When fetching, 'using GIT_ASKPASS to set credentials' is printed in the console, but when I try to tag, it isn't, so maybe that's a hint?

             Failing push:

            using credential dd64d000-6f87-4271-90dd-58babca8bb5e
            > git.exe --version # timeout=10
            > git.exe push origin --tags
            ERROR: Failed to evaluate groovy script.
            hudson.plugins.git.GitException: Command "git.exe push origin --tags" returned status code 128:
            stdout: 
            stderr: remote: Invalid username or password
            fatal: Authentication failed for 'https://name@bitbucket.org/.../...git/' 
            Show
            matthiesenj Jesper Matthiesen added a comment - - edited I'm trying to get a post-build git push --tags to work on windows using user/pass with jenkins credentials outside of pipeline/workflow. The git plugin fetches from a remote private repo this way without problems, but I can't get authentication to work when invoking git through windows batch, or groovy as suggested in https://issues.jenkins-ci.org/browse/JENKINS-28335?focusedCommentId=258095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-258095 above. Shouldn't git operations through the git client in groovy work, if things like fetch, performed by the git plugin, work? When fetching, 'using GIT_ASKPASS to set credentials' is printed in the console, but when I try to tag, it isn't, so maybe that's a hint?  Failing push: using credential dd64d000-6f87-4271-90dd-58babca8bb5e > git.exe --version # timeout=10 > git.exe push origin --tags ERROR: Failed to evaluate groovy script. hudson.plugins.git.GitException: Command "git.exe push origin --tags" returned status code 128: stdout: stderr: remote: Invalid username or password fatal: Authentication failed for 'https: //name@bitbucket.org/.../...git/'
            Hide
            medianick Nick Jones added a comment -

            Jesper Matthiesen, perhaps my answer at https://stackoverflow.com/a/37753202/466874 might help with the authentication specifically? With Git configured as I noted in that answer (basically, disabling the credential.helper), we're using the git CLI successfully on Windows agents to push tags, branches, etc.. After configuring Git this way, the steps we do are 1) bind the Git credentials (to make them available as username/password environment variables), 2) use the https://USERNAME:PASSWORD@URL syntax when pushing anything back to GitHub (e.g., https://foo:bar@github.com/MyRepository).

            Show
            medianick Nick Jones added a comment - Jesper Matthiesen , perhaps my answer at https://stackoverflow.com/a/37753202/466874 might help with the authentication specifically? With Git configured as I noted in that answer (basically, disabling the credential.helper), we're using the git CLI successfully on Windows agents to push tags, branches, etc.. After configuring Git this way, the steps we do are 1) bind the Git credentials (to make them available as username/password environment variables), 2) use the https://USERNAME:PASSWORD@URL syntax when pushing anything back to GitHub (e.g., https://foo:bar@github.com/MyRepository ).
            Hide
            matthiesenj Jesper Matthiesen added a comment -

            Nick Jones I've now made it work by converting the job to pipeline and using the withCredentials directive, thereby injecting the credentials into the url myself. I never found a way to access the credentials to do the same with my previous freestyle project.

            But what I don't understand is why we must do this credential fetching and url massaging ourselves, i.e. why for instance Sverre Moe's example using the GitClient class directly doesn't (seem to) work.

            Show
            matthiesenj Jesper Matthiesen added a comment - Nick Jones I've now made it work by converting the job to pipeline and using the  withCredentials directive, thereby injecting the credentials into the url myself. I never found a way to access the credentials to do the same with my previous freestyle project. But what I don't understand is why we must do this credential fetching and url massaging ourselves, i.e. why for instance Sverre Moe 's example using the GitClient class directly doesn't (seem to) work.
            Hide
            michaelbeaumont Michael Beaumont added a comment - - edited

            I think I've come up with a good workaround. We can set the following at the beginning of our build (declarative pipeline in a Github organization):

            sh 'git config --local credential.helper "!p() { echo username=\\$GIT_USERNAME; echo password=\\$GIT_PASSWORD; }; p"'
            

            See git credential helpers

             

            Then, when we want to use the credentials we can use a block like the following: 

            sh 'git tag -m "" ${VERSION_NUMBER}'
            withCredentials([
              usernamePassword(credentialsId: 'github', usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD')
            ]) {
              sh 'git push origin ${VERSION_NUMBER}'
            }
            

            This way we don't have to even repeat the URL for the origin remote, which is already set. 

            Show
            michaelbeaumont Michael Beaumont added a comment - - edited I think I've come up with a good workaround. We can set the following at the beginning of our build (declarative pipeline in a Github organization): sh 'git config --local credential.helper "!p() { echo username=\\$GIT_USERNAME; echo password=\\$GIT_PASSWORD; }; p" ' See git credential helpers   Then, when we want to use the credentials we can use a block like the following:  sh 'git tag -m "" ${VERSION_NUMBER}' withCredentials([ usernamePassword(credentialsId: 'github' , usernameVariable: 'GIT_USERNAME' , passwordVariable: 'GIT_PASSWORD' ) ]) { sh 'git push origin ${VERSION_NUMBER}' } This way we don't have to even repeat the URL for the origin remote, which is already set. 

              People

              • Assignee:
                Unassigned
                Reporter:
                alecharp Adrien Lecharpentier
              • Votes:
                133 Vote for this issue
                Watchers:
                138 Start watching this issue

                Dates

                • Created:
                  Updated: