Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27115

GIt polling queries wrong head

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      When setting up a git job to query a branch for new commits, if there are similarly named branches the git-plugin will query the wrong branch.

      I have a job setup to query https://github.com/tork-a/euslisp-release.git The branch specifier is release/indigo/euslisp but when I look at the polling log I see the following:

      Started on Feb 24, 2015 2:06:00 PM
      Using strategy: Default
      [poll] Last Built Revision: Revision 445d613f27d3cf28291919f29ccfbe4482082ae5 (refs/remotes/origin/release/indigo/euslisp)
      > git --version # timeout=10
      > git -c core.askpass=true ls-remote -h https://github.com/tork-a/euslisp-release.git euslisp # timeout=10
      [poll] Latest remote head revision is: c80239beb5b1e524ba8e7ee18f6ea8719a994d44
      Done. Took 0.8 sec
      Changes found

      Running the ls-remote command manually resolves as follows:

      $ git -c core.askpass=true ls-remote -h https://github.com/tork-a/euslisp-release.git euslisp
      c80239beb5b1e524ba8e7ee18f6ea8719a994d44 refs/heads/debian/hydro/euslisp
      fb4412b90baaf2e49754dec1771ca96781c20796 refs/heads/debian/hydro/precise/euslisp
      4db89fb3b3c8cd89bccbfcf2de4aba2284e73cf8 refs/heads/debian/hydro/quantal/euslisp
      65e602d48960d099c12865ac8dd2d674dca7e0c1 refs/heads/debian/hydro/raring/euslisp
      1c9b897308856fce5ba543bdb1c4e3d07c7fd50e refs/heads/debian/indigo/euslisp
      521400334fc44c9f7399fabb807c0d71c2be59fa refs/heads/debian/indigo/saucy/euslisp
      9f3eabefe587fbb075ac59334f60ab3f609c6a70 refs/heads/debian/indigo/trusty/euslisp
      c9ef091b81d6e9e5da7583d4fc390a80dc373987 refs/heads/debian/jade/euslisp
      506855f661af2dbc6295e1ae2d84aabf5994cedc refs/heads/debian/jade/trusty/euslisp
      6b9e56f4bc148ee1e6e44c1bf3d74a98ad2f7bd8 refs/heads/debian/jade/utopic/euslisp
      ddfdd6ee79de4dd9b330dd3328fbf1b7e7383b29 refs/heads/debian/jade/vivid/euslisp
      467ec8a124a48a0589b7edf5ab4e89a89cb23f00 refs/heads/patches/debian/hydro/euslisp
      d6407dba72dbfef3ae60b47b87d9be263622c830 refs/heads/patches/debian/hydro/precise/euslisp
      5db895b78ab206a4f9942156e0c091d2053273e2 refs/heads/patches/debian/hydro/quantal/euslisp
      3dba15359404e619a81dddae2d8362c51ba1c1fb refs/heads/patches/debian/hydro/raring/euslisp
      d125c5d45eb4c792fe436191af420d668ece18ad refs/heads/patches/debian/indigo/euslisp
      8d6645b41893ce19b5c0b4ff908a18a291ad60fd refs/heads/patches/debian/indigo/saucy/euslisp
      eee1a1abd8208614aa3edf599bb8a66f5253ec07 refs/heads/patches/debian/indigo/trusty/euslisp
      087f11bee271f103a3dc8c9894b46d46b8675fe7 refs/heads/patches/debian/jade/euslisp
      6c38bf4e061d96a8e3fedc8e7bd58274fa242b19 refs/heads/patches/debian/jade/trusty/euslisp
      a0426bffa153ab4256fb5444984a54d3ec514cc3 refs/heads/patches/debian/jade/utopic/euslisp
      6c7fd0d28d98d5b38edff1ac3a1772cd65271b34 refs/heads/patches/debian/jade/vivid/euslisp
      cb0c906cb76e1fbdceb5b7dae54f19d57b7e38b6 refs/heads/patches/release/hydro/euslisp
      406ac292659b5fba42809e626c3776604028162c refs/heads/patches/release/indigo/euslisp
      01a58c693cf44398399a746a0458ac997214d1f7 refs/heads/patches/release/jade/euslisp
      ccfb5d2358374c1a2130a6dc105492595900d9c6 refs/heads/patches/rpm/hydro/euslisp
      e0bd6c3cb9f3893d362145def530426367e2ef32 refs/heads/patches/rpm/hydro/heisenbug/euslisp
      fa79128633b1391e20d7dd2898251cb1053f3316 refs/heads/patches/rpm/indigo/21/euslisp
      8b3f34e19b1dc2dcaed704dee6e960af7720d8de refs/heads/patches/rpm/indigo/euslisp
      ece33cf90a704dc5592332e239fa8c7d36a0c0e5 refs/heads/patches/rpm/indigo/heisenbug/euslisp
      8651f63aa1c498b154448658ba4abe8479c1e825 refs/heads/patches/rpm/jade/21/euslisp
      6bbb1d7b592f856060feb8facc7a3e2830a88683 refs/heads/patches/rpm/jade/22/euslisp
      002f72bb26b9eedae4791a8d5097dab15b35af30 refs/heads/patches/rpm/jade/euslisp
      7e1f06488fc30f898cd7710efcbead2528309512 refs/heads/release/hydro/euslisp
      445d613f27d3cf28291919f29ccfbe4482082ae5 refs/heads/release/indigo/euslisp
      cba7950078c9f5d6f8324ffe03804023d19c253a refs/heads/release/jade/euslisp
      50fa3905158dee647468457b7dab0af0817766a1 refs/heads/rpm/hydro/euslisp
      90a74237698b8bb62c55178a283e0f6cd3c257cb refs/heads/rpm/hydro/heisenbug/euslisp
      25caabd70bcd3c12b8ba7de5913a1ceee754f4e5 refs/heads/rpm/indigo/21/euslisp
      ddf2a287abdaa60214a87c24520e126e7a859044 refs/heads/rpm/indigo/euslisp
      eb17f4c21c479d7405a195568a77a6ba56e067af refs/heads/rpm/indigo/heisenbug/euslisp
      c612745405d1fdd72944853925e75a8fa6c2c5d4 refs/heads/rpm/jade/21/euslisp
      d93805c9b08bfd8ce1d1d1a90cac6707a30feef2 refs/heads/rpm/jade/22/euslisp
      82d7cb092ed3b1c0a0378cae4a1288d3c3b43059 refs/heads/rpm/jade/euslisp

      As you can see it's matching the wrong branch debian/hydro/euslisp not release/indigo/euslisp when checking for the hashsum.

      The first thing that I note is that the branch specifier is being stripped of everything with slashes, from release/indigo/euslisp to just euslisp which is causing overmatching.

      If I update the command to use the full branch name it still over matches.

      quote
      $ git -c core.askpass=true ls-remote -h https://github.com/tork-a/euslisp-release.git release/indigo/euslisp
      406ac292659b5fba42809e626c3776604028162c refs/heads/patches/release/indigo/euslisp
      445d613f27d3cf28291919f29ccfbe4482082ae5 refs/heads/release/indigo/euslisp
      quote

      This is because ls-remote only does tail matching: https://github.com/git/git/blob/18d0fec24027ac226dc2c4df2b955eef2a16462a/builtin/ls-remote.c#L130

      So to accurately distinguish the correct branch another layer must be added on top to verify the full branch name.

      If you pass the full refspec as recommended by the help button, refs/remotes/origin/release/indigo/euslisp it still fails due what appears to be stripping all but the last element based on slashes.

      Help button snippet: The safest way is to use the refs/heads/<branchName> syntax. This way the expected branch is unambiguous.

      If it was passed through completely it would poll correctly:
      quote
      $ git ls-remote -h https://github.com/tork-a/euslisp-release.git refs/heads/release/indigo/euslisp
      445d613f27d3cf28291919f29ccfbe4482082ae5 refs/heads/release/indigo/euslisp
      quote

      The polling must pass through the full branch specifier to avoid ambiguous branch resolutions.

      This is an issue here: https://github.com/ros/rosdistro/issues/7346 where we have several repositories triggering every hour. Until this is resolved we have resolved to remove these repositories from our continuous integration.

      Before this can be used successfully this issue must also be resolved: https://issues.jenkins-ci.org/browse/JENKINS-27114 since it will fail to checkout if you use the full refspec.

        Attachments

          Activity

          Hide
          markewaite Mark Waite added a comment -

          You might try using a regular expression to match the exact branch name pattern you're trying to build.

          The rather surprising trimming behavior on "origin/master" and "origin/xxx/master" is a legacy behavior that I'm unwilling to change for fear of breaking far too many of the 50,000 installations of the git plugin. You can read more details about it in Alexander Link's analysis of the alternatives.

          Show
          markewaite Mark Waite added a comment - You might try using a regular expression to match the exact branch name pattern you're trying to build. The rather surprising trimming behavior on "origin/master" and "origin/xxx/master" is a legacy behavior that I'm unwilling to change for fear of breaking far too many of the 50,000 installations of the git plugin. You can read more details about it in Alexander Link's analysis of the alternatives.
          Hide
          tfoote Tully Foote added a comment -

          Is this the discussion from Alexander Link you are referring to? https://issues.jenkins-ci.org/browse/JENKINS-20767

          When searching for that I also found related tickets: https://issues.jenkins-ci.org/browse/JENKINS-25580 and https://issues.jenkins-ci.org/browse/JENKINS-14026

          I've been trying many regex invocations. Using the java regex http://www.tutorialspoint.com/java/java_regular_expressions.htm
          But it does not appear that the simple regex :branch/name works. The ls-remote gets passed the entire regex including the : and fails to match anything. And then also passes the full string through to the clone which also fails.

          I think I've found that I can make it work with the wildcard evaluation using */full/branch/name, based on the first link above.

          I completely understand keeping backwards compatibility so as not to break existing users relying on the behavior. However, I would like to suggest adding an option which defaults to off which can disable the legacy trimming behavior to allow users the option to avoid it if they want.

          It's also really frustrating that these things are not documented in the help fields. The trimming is not mentioned in the help fields, and recommending the fully qualified paths to "avoid ambiguity" without mentioning that they are actually going to be trimmed and ambiguous is very frustrating.

          From finding the other related issues, it seems that I'm not the only one encountering this problem. It would be really great to get an option to disable the trimming. With or without that updating the documentation to mention the trimming and potential work around solutions would be greatly appreciated.

          Show
          tfoote Tully Foote added a comment - Is this the discussion from Alexander Link you are referring to? https://issues.jenkins-ci.org/browse/JENKINS-20767 When searching for that I also found related tickets: https://issues.jenkins-ci.org/browse/JENKINS-25580 and https://issues.jenkins-ci.org/browse/JENKINS-14026 I've been trying many regex invocations. Using the java regex http://www.tutorialspoint.com/java/java_regular_expressions.htm But it does not appear that the simple regex : branch/name works. The ls-remote gets passed the entire regex including the : and fails to match anything. And then also passes the full string through to the clone which also fails. I think I've found that I can make it work with the wildcard evaluation using */full/branch/name, based on the first link above. I completely understand keeping backwards compatibility so as not to break existing users relying on the behavior. However, I would like to suggest adding an option which defaults to off which can disable the legacy trimming behavior to allow users the option to avoid it if they want. It's also really frustrating that these things are not documented in the help fields. The trimming is not mentioned in the help fields, and recommending the fully qualified paths to "avoid ambiguity" without mentioning that they are actually going to be trimmed and ambiguous is very frustrating. From finding the other related issues, it seems that I'm not the only one encountering this problem. It would be really great to get an option to disable the trimming. With or without that updating the documentation to mention the trimming and potential work around solutions would be greatly appreciated.
          Hide
          markewaite Mark Waite added a comment -

          I think you'd need to use a regular expression like

          :.*origin/.*er
          

          as a way to match the branch name containing origin followed by a slash followed by any text which ends in "er".

          I like your suggestion on the improvement of the help text. Are you willing to submit a pull request containing the help text that you think would best clarify the situation? The current help text was a result of Alexander Link's significant effort to understand the behavior. I'm sure he'd be glad to see further improvements to that text.

          Show
          markewaite Mark Waite added a comment - I think you'd need to use a regular expression like :.*origin/.*er as a way to match the branch name containing origin followed by a slash followed by any text which ends in "er". I like your suggestion on the improvement of the help text. Are you willing to submit a pull request containing the help text that you think would best clarify the situation? The current help text was a result of Alexander Link's significant effort to understand the behavior. I'm sure he'd be glad to see further improvements to that text.
          Hide
          tfoote Tully Foote added a comment -

          I submitted a pull-request here: https://github.com/jenkinsci/git-plugin/pull/305 For help text mentioning that foo/bar => bar

          I've found that the trimming does not happen if you use the full refs/heads/ I think my previous tests used refs/head/ by accident.

          The PR takes care of the primary issue here.

          Show
          tfoote Tully Foote added a comment - I submitted a pull-request here: https://github.com/jenkinsci/git-plugin/pull/305 For help text mentioning that foo/bar => bar I've found that the trimming does not happen if you use the full refs/heads/ I think my previous tests used refs/head/ by accident. The PR takes care of the primary issue here.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Tully Foote
          Path:
          src/main/resources/hudson/plugins/git/BranchSpec/help-name.html
          http://jenkins-ci.org/commit/git-plugin/a9e34a703da8af95c298c100e640e63335d5a868
          Log:
          Add comment about trimming before slashes

          This is a follow up to https://issues.jenkins-ci.org/browse/JENKINS-27115

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Tully Foote Path: src/main/resources/hudson/plugins/git/BranchSpec/help-name.html http://jenkins-ci.org/commit/git-plugin/a9e34a703da8af95c298c100e640e63335d5a868 Log: Add comment about trimming before slashes This is a follow up to https://issues.jenkins-ci.org/browse/JENKINS-27115
          Hide
          markewaite Mark Waite added a comment - - edited

          Help text improvement will be included in 2.3.6 and later.

          The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin. Problems detected in the pre-release should be e-mailed to Mark Waite and Nicolas De Loof.

          I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases.

          Show
          markewaite Mark Waite added a comment - - edited Help text improvement will be included in 2.3.6 and later. The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin . Problems detected in the pre-release should be e-mailed to Mark Waite and Nicolas De Loof . I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases.
          Hide
          markewaite Mark Waite added a comment -

          Fix included in git plugin 2.4.0 released 18 July 2015

          Show
          markewaite Mark Waite added a comment - Fix included in git plugin 2.4.0 released 18 July 2015

            People

            • Assignee:
              ndeloof Nicolas De Loof
              Reporter:
              tfoote Tully Foote
            • Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: