Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42204

Scan of git pipeline fails with NPE due to recent GitSCMSource change

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Fixed
    • Component/s: git-plugin
    • Labels:
      None
    • Environment:
      Docker instance as described later in bug report, including unreleased version of git plugin
    • Similar Issues:

      Description

      Initial scan of branches in a multi-branch-project fails with a null pointer exception. It seems to be due to the addition of GitSCMSource: support custom remote and refspec.

      Steps to duplicate the problem:

      1. Clone, build, and run the docker instance
          $ git lfs clone https://github.com/MarkEWaite/docker-lfs JENKINS-42204
          $ cd JENKINS-42204
          $ git lfs fetch origin 4bd3aba9910323f8bc279ad5bd7ac3ae8d5ffd5f
          $ git checkout -b lts-with-plugins 4bd3aba9910323f8bc279ad5bd7ac3ae8d5ffd5f
          $ docker build -t jenkins:JENKINS-42204 .
          $ docker run -i --rm --publish 8080:8080 jenkins:JENKINS-42204
        
      2. Connect a web browser to that docker instance (http://localhost:8080)
      3. Open the "Bugs - Pipeline Checks"
      4. Open the "jenkins-bugs" multi-branch pipeline job
      5. Click the "Scan Multibranch Pipeline" link and then the "Run Now" link to start branch indexing
      6. Click the "Log" link to view the log - stack trace will appear in that log

      If I revert the commit for GitSCMSource: support custom remote and refspec, the problem no longer appears. Refer to PR466 for more details.

        Attachments

          Activity

          Hide
          jbq jbq added a comment -

          See https://github.com/jenkinsci/git-plugin/pull/475

          NOTE: instructions were slightly off, I had to issue a `git lfs checkout` before `docker build`. Never used `git lfs` before but I could figure out by myself as the plugin files only contained git commit references.

          Show
          jbq jbq added a comment - See https://github.com/jenkinsci/git-plugin/pull/475 NOTE: instructions were slightly off, I had to issue a `git lfs checkout` before `docker build`. Never used `git lfs` before but I could figure out by myself as the plugin files only contained git commit references.
          Hide
          markewaite Mark Waite added a comment -

          I was able to run your change in a later version of that docker instance tonight. It passed the check described in this bug report, bug fails with a null pointer exception when scanning the "jenkins-bugs-bitbucket" repository.

          The stack trace is:

          Started by user anonymous
          [Tue Feb 21 21:36:19 MST 2017] Starting branch indexing...
           > git rev-parse --is-inside-work-tree # timeout=11
          Setting origin to https://bitbucket.org/markewaite/jenkins-bugs.git
           > git config remote.origin.url https://bitbucket.org/markewaite/jenkins-bugs.git # timeout=11
          Fetching & pruning origin...
          Fetching upstream changes from origin
           > git --version # timeout=11
           > git fetch --tags --progress origin +refs/heads/*:refs/remotes/origin/* --prune
          Getting remote branches...
          Seen branch in repository origin/JENKINS-39905
          Seen 1 remote branch
          Checking branch JENKINS-39905
          ERROR: [Tue Feb 21 21:36:20 MST 2017] Could not fetch branches from source 542506ca-b4e4-4721-a50f-e301e193846e
          java.lang.NullPointerException
          	at jenkins.plugins.git.AbstractGitSCMSource.getPattern(AbstractGitSCMSource.java:458)
          	at jenkins.plugins.git.AbstractGitSCMSource.isExcluded(AbstractGitSCMSource.java:447)
          	at jenkins.plugins.git.AbstractGitSCMSource$2.run(AbstractGitSCMSource.java:256)
          	at jenkins.plugins.git.AbstractGitSCMSource$2.run(AbstractGitSCMSource.java:242)
          	at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:212)
          	at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:242)
          	at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300)
          	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:210)
          	at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634)
          	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219)
          	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:141)
          	at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:973)
          	at hudson.model.ResourceController.execute(ResourceController.java:98)
          	at hudson.model.Executor.run(Executor.java:404)
          [Tue Feb 21 21:36:20 MST 2017] Finished branch indexing. Indexing took 0.58 sec
          FATAL: Failed to recompute children of Bugs - Pipeline Checks » jenkins-bugs bitbucket
          java.lang.NullPointerException
          	at jenkins.plugins.git.AbstractGitSCMSource.getPattern(AbstractGitSCMSource.java:458)
          	at jenkins.plugins.git.AbstractGitSCMSource.isExcluded(AbstractGitSCMSource.java:447)
          	at jenkins.plugins.git.AbstractGitSCMSource$2.run(AbstractGitSCMSource.java:256)
          	at jenkins.plugins.git.AbstractGitSCMSource$2.run(AbstractGitSCMSource.java:242)
          	at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:212)
          	at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:242)
          	at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300)
          	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:210)
          	at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634)
          	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219)
          	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:141)
          	at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:973)
          	at hudson.model.ResourceController.execute(ResourceController.java:98)
          	at hudson.model.Executor.run(Executor.java:404)
          Finished: FAILURE
          
          Show
          markewaite Mark Waite added a comment - I was able to run your change in a later version of that docker instance tonight. It passed the check described in this bug report, bug fails with a null pointer exception when scanning the "jenkins-bugs-bitbucket" repository. The stack trace is: Started by user anonymous [Tue Feb 21 21:36:19 MST 2017] Starting branch indexing... > git rev-parse --is-inside-work-tree # timeout=11 Setting origin to https://bitbucket.org/markewaite/jenkins-bugs.git > git config remote.origin.url https://bitbucket.org/markewaite/jenkins-bugs.git # timeout=11 Fetching & pruning origin... Fetching upstream changes from origin > git --version # timeout=11 > git fetch --tags --progress origin +refs/heads/*:refs/remotes/origin/* --prune Getting remote branches... Seen branch in repository origin/JENKINS-39905 Seen 1 remote branch Checking branch JENKINS-39905 ERROR: [Tue Feb 21 21:36:20 MST 2017] Could not fetch branches from source 542506ca-b4e4-4721-a50f-e301e193846e java.lang.NullPointerException at jenkins.plugins.git.AbstractGitSCMSource.getPattern(AbstractGitSCMSource.java:458) at jenkins.plugins.git.AbstractGitSCMSource.isExcluded(AbstractGitSCMSource.java:447) at jenkins.plugins.git.AbstractGitSCMSource$2.run(AbstractGitSCMSource.java:256) at jenkins.plugins.git.AbstractGitSCMSource$2.run(AbstractGitSCMSource.java:242) at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:212) at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:242) at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:210) at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634) at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219) at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:141) at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:973) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) [Tue Feb 21 21:36:20 MST 2017] Finished branch indexing. Indexing took 0.58 sec FATAL: Failed to recompute children of Bugs - Pipeline Checks » jenkins-bugs bitbucket java.lang.NullPointerException at jenkins.plugins.git.AbstractGitSCMSource.getPattern(AbstractGitSCMSource.java:458) at jenkins.plugins.git.AbstractGitSCMSource.isExcluded(AbstractGitSCMSource.java:447) at jenkins.plugins.git.AbstractGitSCMSource$2.run(AbstractGitSCMSource.java:256) at jenkins.plugins.git.AbstractGitSCMSource$2.run(AbstractGitSCMSource.java:242) at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:212) at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:242) at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:210) at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634) at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219) at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:141) at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:973) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) Finished: FAILURE
          Hide
          jbq jbq added a comment - - edited

          I initially removed the DEFAULT_INCLUDES and DEFAULT_EXCLUDES constants because Intellij reported them to be unused. Guess there is some kind of magic somewhere. I restored them as they were before, it should fix the problem. Do you have a pointer to a Jenkins core documentation mentioning how default values work?

          Also I think we have some room for improvement on the unit tests, because they are currently unable to detect such cases. The whole issue is about the way we write unit tests, and in particular how we instantiate GitSCMSource. In all tests that I know, we use the constructor GitSCMSource(String id, String remote, String credentialsId, String includes, String excludes, boolean ignoreOnPushNotifications) but that is cheating. I don't think Jenkins works like that in the real world otherwise this issue would be void. Shouldn't we use something like the Job DSL or some high-level API to write unit tests? To make sure that we don't use the constructors directly. Of course they are unit tests, but we shouldn't be too far from how the code is run in the real world. Any idea?

          Show
          jbq jbq added a comment - - edited I initially removed the DEFAULT_INCLUDES and DEFAULT_EXCLUDES constants because Intellij reported them to be unused. Guess there is some kind of magic somewhere. I restored them as they were before, it should fix the problem. Do you have a pointer to a Jenkins core documentation mentioning how default values work? Also I think we have some room for improvement on the unit tests, because they are currently unable to detect such cases. The whole issue is about the way we write unit tests, and in particular how we instantiate GitSCMSource . In all tests that I know, we use the constructor GitSCMSource(String id, String remote, String credentialsId, String includes, String excludes, boolean ignoreOnPushNotifications) but that is cheating. I don't think Jenkins works like that in the real world otherwise this issue would be void. Shouldn't we use something like the Job DSL or some high-level API to write unit tests? To make sure that we don't use the constructors directly. Of course they are unit tests, but we shouldn't be too far from how the code is run in the real world. Any idea?
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Mark Waite
          Path:
          src/main/java/jenkins/plugins/git/GitSCMSource.java
          http://jenkins-ci.org/commit/git-plugin/608a378e37bfca1692d23751955de0a9e998f9b2
          Log:
          Merge pull request #475 from jbq/scm_source_compat

          JENKINS-42204 GitSCMSource: Fix backwards compatibility for advanced options.

          Compare: https://github.com/jenkinsci/git-plugin/compare/d780f27fb1d2...608a378e37bf

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Mark Waite Path: src/main/java/jenkins/plugins/git/GitSCMSource.java http://jenkins-ci.org/commit/git-plugin/608a378e37bfca1692d23751955de0a9e998f9b2 Log: Merge pull request #475 from jbq/scm_source_compat JENKINS-42204 GitSCMSource: Fix backwards compatibility for advanced options. Compare: https://github.com/jenkinsci/git-plugin/compare/d780f27fb1d2...608a378e37bf
          Hide
          markewaite Mark Waite added a comment -

          Confirmed fixed on the master branch. May still need further compatibility testing, since there is still not an automated test which shows the problem as reported here.

          Show
          markewaite Mark Waite added a comment - Confirmed fixed on the master branch. May still need further compatibility testing, since there is still not an automated test which shows the problem as reported here.
          Hide
          markewaite Mark Waite added a comment -

          Git plugin 3.1.0 released on 4 Mar 2017 includes this fix.  This bug never existed in any released version of the plugin, only in pre-releases.

          Show
          markewaite Mark Waite added a comment - Git plugin 3.1.0 released on 4 Mar 2017 includes this fix.  This bug never existed in any released version of the plugin, only in pre-releases.

            People

            • Assignee:
              jbq jbq
              Reporter:
              markewaite Mark Waite
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: