Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22547

Git timeout setting does not work for checkout

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: git-plugin
    • Environment:
      Windows (8) (the slow disk access probably makes it more likely to manifest)
    • Similar Issues:

      Description

      Some git processes error out with ERROR: Timeout after 10 minutes even though longer timeout is configured in the job.

      It affects the `git checkout` operation itself. The fetch operation is OK.

      The log says:

      using GIT_SSH to set credentials jenkins key for git
       > git fetch --tags --progress git@git.company.com:project +refs/heads/*:refs/remotes/origin/* --prune
      Checking out Revision 159bc2b21669bc7b5217341fc8de9cd6b48439b2 (origin/dev/jan.hudec/pu)
       > git config core.sparsecheckout
       > git checkout -f 159bc2b21669bc7b5217341fc8de9cd6b48439b2
      ERROR: Timeout after 10 minutes
      FATAL: Could not checkout null with start point 159bc2b21669bc7b5217341fc8de9cd6b48439b2
      

      When I manually removed the lock and repeated the checkout operation, it indeed took 11 minutes 15 seconds on the node where it failed.

      The global timeout does work, so it's not a blocker anymore. It is, however, rather non-obvious configuration as the -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=30 (or whatever sufficiently large value) option needs to be added to both JVM options of the master and JVM options of all slaves. The master options can only be configured in the servlet container and while the slave options can be configured in node settings (hidden out under "Advanced" button), slaves running as windows service don't take this into account without reinstalling the service.

        Attachments

          Issue Links

            Activity

            Hide
            guss77 Oded Arbel added a comment - - edited

            I have the same problem - timeout is always "10" and there is no way to change it.

            I've added the advanced checkout behavior and that did not change a thing. Here's the pipeline syntax I'm using:

             

            checkout scm: [$class: 'GitSCM', 
                branches: [[name: '*/master']], 
                doGenerateSubmoduleConfigurations: false, 
                extensions: [[$class: 'CheckoutOption', timeout: 240]], 
                gitTool: 'Default', 
                submoduleCfg: [], 
                userRemoteConfigs: [[
                    credentialsId: '981cbff3-3a30-4a61-be40-99f441ea0559', 
                    url: 'git@server:project.git'
                ]]]
            

            And it looks the same - the logs shows `timeout=10` and it ends with:

            ERROR: Timeout after 10 minutes
            ERROR: Error cloning remote repo 'origin'
            
            Show
            guss77 Oded Arbel added a comment - - edited I have the same problem - timeout is always "10" and there is no way to change it. I've added the advanced checkout behavior and that did not change a thing. Here's the pipeline syntax I'm using:   checkout scm: [$class: 'GitSCM' , branches: [[name: '*/master' ]], doGenerateSubmoduleConfigurations: false , extensions: [[$class: 'CheckoutOption' , timeout: 240]], gitTool: 'Default' , submoduleCfg: [], userRemoteConfigs: [[ credentialsId: '981cbff3-3a30-4a61-be40-99f441ea0559' , url: 'git@server:project.git' ]]] And it looks the same - the logs shows `timeout=10` and it ends with: ERROR: Timeout after 10 minutes ERROR: Error cloning remote repo 'origin'
            Hide
            guss77 Oded Arbel added a comment -

            Update, my mistake - should have used advanced clone behavior

            Show
            guss77 Oded Arbel added a comment - Update, my mistake - should have used advanced clone behavior
            Hide
            lglussen Luke Lussenden added a comment -

            Oded Arbel can you provide an example or link of what you mean by advanced clone behaviour? I'm still getting the 10 minute timeout using the following:

            git branch: '$BRANCH',
            
                credentialsId: 'XXXX', 
                url: 'ssh://git@example.com/myRepo.git', 
                extensions: [[$class: 'CheckoutOption', timeout: 100]]
            

            I tried the `scm checkout` syntax you reference in your first comment and I see the 10 minute timeout just the same as you were describing.

            Additionally, how did you confirm the "advanced behaviour" was working?  Will the log messages begin to display the configured timeout value, or do I just have to test it and hope my network is running slow enough to validate the test?

             

            Show
            lglussen Luke Lussenden added a comment - Oded Arbel can you provide an example or link of what you mean by advanced clone behaviour? I'm still getting the 10 minute timeout using the following: git branch: '$BRANCH' , credentialsId: 'XXXX' , url: 'ssh: //git@example.com/myRepo.git' , extensions: [[$class: 'CheckoutOption' , timeout: 100]] I tried the `scm checkout` syntax you reference in your first comment and I see the 10 minute timeout just the same as you were describing. Additionally, how did you confirm the "advanced behaviour" was working?  Will the log messages begin to display the configured timeout value, or do I just have to test it and hope my network is running slow enough to validate the test?  
            Hide
            guss77 Oded Arbel added a comment -

            Yes - adding an "advanced clone behavior" and then setting the timeout there solved my problem.

            Show
            guss77 Oded Arbel added a comment - Yes - adding an "advanced clone behavior" and then setting the timeout there solved my problem.
            Hide
            markewaite Mark Waite added a comment -

            Vadivel Natarajan the bug was fixed over 2 years ago. and is as described by Tim Knight in the comment above your comment that you're reopening the bug.

            It is not enough to say "the same issue occurring". You'll need to provide much more context than "same issue occurring". What have you tried? What is the context where timeouts are not behaving as you expect? What is the log content when the timeout does not behave as you expect? What job type are you using?

            Please gather those details and submit a new bug report, rather than reopening this report.

            There are many, many users that are successfully using extended timeouts to clone and checkout large repositories. I've presented talks at Jenkins World 2016, Jenkins World 2017,at a 2016 online meetup and at a 2017 online meetup that describe techniques to better manage large repositories. All of those talks depend on adjusting timeout values as needed, and they work.

            In addition to those resources, CloudBees support has provided detailed instructions for configuring a reference repository to speed clone operations.

            Show
            markewaite Mark Waite added a comment - Vadivel Natarajan the bug was fixed over 2 years ago. and is as described by Tim Knight in the comment above your comment that you're reopening the bug. It is not enough to say "the same issue occurring". You'll need to provide much more context than "same issue occurring". What have you tried? What is the context where timeouts are not behaving as you expect? What is the log content when the timeout does not behave as you expect? What job type are you using? Please gather those details and submit a new bug report, rather than reopening this report. There are many, many users that are successfully using extended timeouts to clone and checkout large repositories. I've presented talks at Jenkins World 2016 , Jenkins World 2017 ,at a 2016 online meetup and at a 2017 online meetup that describe techniques to better manage large repositories. All of those talks depend on adjusting timeout values as needed, and they work. In addition to those resources, CloudBees support has provided detailed instructions for configuring a reference repository to speed clone operations.

              People

              • Assignee:
                markewaite Mark Waite
                Reporter:
                bulb Jan Hudec
              • Votes:
                1 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: