Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26660

Add another '-f' to git clean to remove deleted submodules or other repos checked out in workspace in pipeline

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Labels:
      None
    • Environment:
      Jenkins and multi SCM plugin configured with git repository using submodules
      Alternatively: A pipeline that checks out a second repository in a sub-dir
    • Similar Issues:
    • Released As:
      git-client-plugin 3.0.0, git-plugin 4.1.0

      Description

      Submodules clean up

      Jenkins SCM plugin can handle git repositories with submodules except for clean up.

      When "Clean before checkout" is selected, it runs 'git reset --hard && git clean -dxf' on the repository and submodules if "Recursively update submodules" is selected. This works for most stuff but it does not work if git submodule is removed from the tree which leaves the submodule directory in workspace even after 'git clean -dxf'.

      If another '-f' is added, also the removed submodule will be deleted which is the behavior I'd expect when git submodules are in use.

      So Jenkins git plugin should call 'git clean -dxf -f' in that case.

      Pipelines workspace reuse

      An alternative use case, which I believe is very common, is that a pipeline checks out a repository in a sub-directory. Since the workspaces are reused between builds, there is a big risk that code or generated files in that sub-directory from previous builds are still there in the next run. This can seriously impact reliability of build or even fail them. One case is that old cucumber test reports are found.

      Completely wiping the workspace has a performance cost that is unreasonable.

        Attachments

          Issue Links

            Activity

            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Samantha Schwarz
            Path:
            src/main/java/hudson/plugins/git/GitAPI.java
            src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
            src/main/java/org/jenkinsci/plugins/gitclient/GitClient.java
            src/main/java/org/jenkinsci/plugins/gitclient/JGitAPIImpl.java
            src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java
            src/test/java/org/jenkinsci/plugins/gitclient/GitAPITestCase.java
            http://jenkins-ci.org/commit/git-client-plugin/58a322534f70180f5cba17e8f09c0e40df7fd962
            Log:
            Clean takes a flag that will add a --force flag to clean.

            See bug report https://issues.jenkins-ci.org/browse/JENKINS-26660 for more details

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Samantha Schwarz Path: src/main/java/hudson/plugins/git/GitAPI.java src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java src/main/java/org/jenkinsci/plugins/gitclient/GitClient.java src/main/java/org/jenkinsci/plugins/gitclient/JGitAPIImpl.java src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java src/test/java/org/jenkinsci/plugins/gitclient/GitAPITestCase.java http://jenkins-ci.org/commit/git-client-plugin/58a322534f70180f5cba17e8f09c0e40df7fd962 Log: Clean takes a flag that will add a --force flag to clean. See bug report https://issues.jenkins-ci.org/browse/JENKINS-26660 for more details
            Hide
            renescheibe René Scheibe added a comment - - edited

            Current State

            Show
            renescheibe René Scheibe added a comment - - edited Current State Step 1 is done. With git-client-plugin 3.0.0 https://github.com/jenkinsci/git-client-plugin/pull/222 is released now. Step 2 is nearly done. Using the new feature from git-client-plugin in git-plugin ( https://github.com/jenkinsci/git-plugin/pull/792 ). TODO: release
            Hide
            markewaite Mark Waite added a comment -

            Built hpi of a pre-release of the plugin can be pulled from the master branch of the ci.jenkins.io job.

            Plan is to release git plugin 4.1 before Dec 31, 2019 with this fix included.

            Show
            markewaite Mark Waite added a comment - Built hpi of a pre-release of the plugin can be pulled from the master branch of the ci.jenkins.io job . Plan is to release git plugin 4.1 before Dec 31, 2019 with this fix included.
            Hide
            markewaite Mark Waite added a comment -

            Fixed in git plugin 4.1.0, released Jan 18, 2020.

            Show
            markewaite Mark Waite added a comment - Fixed in git plugin 4.1.0, released Jan 18, 2020.
            Hide
            renescheibe René Scheibe added a comment -

            NOTE

            This change breaks backward compatibility regarding Job DSL for the methods cleanBeforeCheckoutTrait() and cleanAfterCheckoutTrait().

            The changes in the UI are backward compatible.

            old syntax

            multibranchPipelineJob('example-old') {
              branchSources {
                branchSource {
                  source {
                    git {
                      id('example-old')
                      remote('https://github.com/jenkinsci/job-dsl-plugin.git')
                      traits {
                        cleanBeforeCheckoutTrait() // <--- fails with git-plugin v4.1.0
                        cleanAfterCheckoutTrait()  // <--- fails with git-plugin v4.1.0
                      }
                    }
                  }
                }
              }
            }

            new syntax

            multibranchPipelineJob('example-new') {
              branchSources { 
                branchSource { 
                  source { 
                    git { 
                      id('example-new')
                      remote('https://github.com/jenkinsci/job-dsl-plugin.git')
                      traits { 
                        cleanBeforeCheckoutTrait {
                          extension {
                            deleteUntrackedNestedRepositories(false)
                          }
                        }
                        cleanAfterCheckoutTrait {
                          extension {
                            deleteUntrackedNestedRepositories(false)
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
            Show
            renescheibe René Scheibe added a comment - NOTE This change breaks backward compatibility regarding Job DSL for the methods cleanBeforeCheckoutTrait() and cleanAfterCheckoutTrait() . The changes in the UI are backward compatible. old syntax multibranchPipelineJob( 'example-old' ) { branchSources { branchSource { source { git { id( 'example-old' ) remote( 'https: //github.com/jenkinsci/job-dsl-plugin.git' ) traits { cleanBeforeCheckoutTrait() // <--- fails with git-plugin v4.1.0 cleanAfterCheckoutTrait() // <--- fails with git-plugin v4.1.0 } } } } } } new syntax multibranchPipelineJob( 'example- new ' ) { branchSources { branchSource { source { git { id( 'example- new ' ) remote( 'https: //github.com/jenkinsci/job-dsl-plugin.git' ) traits { cleanBeforeCheckoutTrait { extension { deleteUntrackedNestedRepositories( false ) } } cleanAfterCheckoutTrait { extension { deleteUntrackedNestedRepositories( false ) } } } } } } } }

              People

              • Assignee:
                Unassigned
                Reporter:
                mcfrisk Mikko Rapeli
              • Votes:
                23 Vote for this issue
                Watchers:
                31 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: