Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42278

Branch indexing failing on MultiBranch Pipeline when at least one branch has no Jenkinsfile

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Blocker
    • Resolution: Fixed
    • Component/s: mercurial-plugin
    • Labels:
    • Environment:
      Jenkins 2.32.2, Branch API plugin 2.0.7, Mercurial plugin 1.58 and 1.59
    • Similar Issues:

      Description

      We're using mercurial as our SCM exclusively, in combination with Jenkins, thanks to this plugin.

      When Branch Indexing is being run (either triggered or manually), on a multibranch pipeline job, it normally iterates over all the fetched heads to find the Jenkinsfile for each. This is a feature that seems to be added in v 1.58 of the mercurial plugin. However, if one of the branches in the repository does not have a Jenkinsfile, the indexing is terminated with status failed.

      Exception

      [ED1EAACEE3A3A53C6B19AA947A23C57976B19FFE-SOMEJOB] $ /usr/bin/hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" locate -r b8b87f2c24b721d475750dfbba88c63bf2bda060 -I path:Jenkinsfile
      ERROR: Failed to run /usr/bin/hg --config auth.jenkins.prefix=* --config auth.jenkins.username=CLEARTXTUSER --config auth.jenkins.password=CLEARTXTPASS --config "auth.jenkins.schemes=http https" locate -r b8b87f2c24b721d475750dfbba88c63bf2bda060 -I path:Jenkinsfile
      ERROR: [Thu Feb 23 07:57:10 UTC 2017] Could not fetch branches from source 82b88e38-6997-4253-8e0d-7211e698983b
      hudson.AbortException
      at hudson.plugins.mercurial.HgExe.popen(HgExe.java:425)
      at hudson.plugins.mercurial.MercurialSCMSource$1.stat(MercurialSCMSource.java:150)
      at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:52)
      at hudson.plugins.mercurial.MercurialSCMSource.retrieve(MercurialSCMSource.java:176)
      at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300)
      at jenkins.scm.api.SCMSource.fetch(SCMSource.java:210)
      at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:628)
      at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219)
      at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:154)
      at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:967)
      at hudson.model.ResourceController.execute(ResourceController.java:98)
      at hudson.model.Executor.run(Executor.java:404)
      [Thu Feb 23 07:57:10 UTC 2017] Finished branch indexing. Indexing took 5.2 sec
      FATAL: null
      Finished: FAILURE

      How to reproduce

      This problem can be reproduced consistently using the mercurial plugin v1.58 or 1.59, by:

      1. Configure a multibranch pipeline job with Mercurial repository, with:
        1. One or more branches in repo
        2. At least one branch without Jenkinsfile
      2. Trigger the 'Scan Multibranch Pipeline' (either manually or on commit)

      Likely cause

      The interaction between Mercurial plugin and Branch API is fairly complex, but my estimation is that the below section of code is throwing an exit code of 1 when no Jenkinsfile is present. This subsequently causes the branch indexing to fail (with exception thrown).

      https://github.com/jenkinsci/mercurial-plugin/blob/master/src/main/java/hudson/plugins/mercurial/MercurialSCMSource.java#L151

      From what I understand of it, the Branch API is designed to throw the exception and stop trying. This would mean that the Mercurial plugin can't pass exit code 1 when a Jenkinsfile is missing.
      Note that both the methods committed over the last months, 'hg files' and 'hg locate' exit with code 1 if no pattern match is found.

      Workaround

      Rollback to Mercurial plugin v1.57 resolved the issue and allows the branch indexing to complete. Of course it removes some of the new and desirable behavior introduced for properly tracking the pipeline enabled branches and marking closed branches.

        Attachments

          Issue Links

            Activity

            Hide
            markl_lagendijk Mark Lagendijk added a comment -

            This issue is solved in this commit: https://github.com/jenkinsci/mercurial-plugin/pull/96/commits/c9674b9e0de463f719bdf5eb21501fcc4d50fbdf

            It is a commit of a pull request on the Mercurial plugin. When that is merged, this issue would be fixed.

            Show
            markl_lagendijk Mark Lagendijk added a comment - This issue is solved in this commit: https://github.com/jenkinsci/mercurial-plugin/pull/96/commits/c9674b9e0de463f719bdf5eb21501fcc4d50fbdf It is a commit of a pull request on the Mercurial plugin. When that is merged, this issue would be fixed.
            Hide
            wynx WynX Alucard added a comment -

            Thank you Mark Lagendijk, I'm hoping your commit can be reviewed by someone shortly, because for me this issue is a blocker (we're running 1.57 still).

            Show
            wynx WynX Alucard added a comment - Thank you Mark Lagendijk , I'm hoping your commit can be reviewed by someone shortly, because for me this issue is a blocker (we're running 1.57 still).
            Hide
            msebire Mathieu SEBIRE added a comment -

            Hello,

            I've created a new pull request that is compatible with the lastest version of the master branch.

            https://github.com/jenkinsci/mercurial-plugin/pull/106 

            Can you integrate it ?

            Thanks

            Show
            msebire Mathieu SEBIRE added a comment - Hello, I've created a new pull request that is compatible with the lastest version of the master branch. https://github.com/jenkinsci/mercurial-plugin/pull/106   Can you integrate it ? Thanks
            Hide
            batmat Baptiste Mathus added a comment -

            Apparently so, https://github.com/jenkinsci/mercurial-plugin/pull/96 and https://github.com/jenkinsci/mercurial-plugin/pull/106 are both open and said to fix the same issue? Mathieu SEBIRE and Mark Lagendijk can you possibly review each other and help decide if those PRs are complementary, or if one should be just closed?

            (Note: I'm not a maintainer of this plugin, and have never used Mercurial, just trying to move this issue forward).

            Show
            batmat Baptiste Mathus added a comment - Apparently so, https://github.com/jenkinsci/mercurial-plugin/pull/96  and https://github.com/jenkinsci/mercurial-plugin/pull/106  are both open and said to fix the same issue? Mathieu SEBIRE and Mark Lagendijk can you possibly review each other and help decide if those PRs are complementary, or if one should be just closed? (Note: I'm not a maintainer of this plugin, and have never used Mercurial, just trying to move this issue forward).
            Hide
            msebire Mathieu SEBIRE added a comment -

            Both pull request fix the issue the same way.

            The only difference is that https://github.com/jenkinsci/mercurial-plugin/pull/106 contains only the fix and is compatible with the release 2.0.0.

            The pull request https://github.com/jenkinsci/mercurial-plugin/pull/96 contains also other features and is in conflict when we try to merge it with the master branch.

            Show
            msebire Mathieu SEBIRE added a comment - Both pull request fix the issue the same way. The only difference is that https://github.com/jenkinsci/mercurial-plugin/pull/106  contains only the fix and is compatible with the release 2.0.0. The pull request  https://github.com/jenkinsci/mercurial-plugin/pull/96  contains also other features and is in conflict when we try to merge it with the master branch.
            Hide
            wynx WynX Alucard added a comment -

            Thank you Mathieu for the updated pull request for the current master.

            Jesse Glick, not sure if you are the active maintainer of this plugin, would you be able to integrate this?

            Show
            wynx WynX Alucard added a comment - Thank you Mathieu for the updated pull request for the current master. Jesse Glick , not sure if you are the active maintainer of this plugin, would you be able to integrate this?
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/hudson/plugins/mercurial/MercurialSCMSource.java
            src/test/java/hudson/plugins/mercurial/PipelineTest.java
            http://jenkins-ci.org/commit/mercurial-plugin/4b49070979d49d5c263bcd92ae7b57f4eb83eaa2
            Log:
            [FIXED JENKINS-42278] SCMProbeImpl.stat was returning REGULAR_FILE but failing rather than return NONEXISTENT.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/plugins/mercurial/MercurialSCMSource.java src/test/java/hudson/plugins/mercurial/PipelineTest.java http://jenkins-ci.org/commit/mercurial-plugin/4b49070979d49d5c263bcd92ae7b57f4eb83eaa2 Log: [FIXED JENKINS-42278] SCMProbeImpl.stat was returning REGULAR_FILE but failing rather than return NONEXISTENT.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/hudson/plugins/mercurial/MercurialSCMSource.java
            src/test/java/hudson/plugins/mercurial/MercurialSCMSource2Test.java
            http://jenkins-ci.org/commit/mercurial-plugin/0ca40a7ca810d2e89771feb37b00e4750cdac781
            Log:
            JENKINS-42278 Reproduced in test.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/plugins/mercurial/MercurialSCMSource.java src/test/java/hudson/plugins/mercurial/MercurialSCMSource2Test.java http://jenkins-ci.org/commit/mercurial-plugin/0ca40a7ca810d2e89771feb37b00e4750cdac781 Log: JENKINS-42278 Reproduced in test.

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                wynx WynX Alucard
              • Votes:
                8 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: