Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55330

multi-branch pipeline project: github tag not found when building a tag

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      The git tag was firstly pushed to GitHub and the commits are pushed later.

      "Fetch Tags" is already enabled, However, Jenkins failed to get updated tag metadata through the GitHub API.

      This also happens when we re-tag a tag and force pushed to GitHub to rewrite the commit.

      We tried to use cURL with the same credential to get the tag via the GitHub API and which works fine, we got the correct tag response.

      However, Jenkins still failed to the tag data. It seems like Jenkins cached the API response and didn't invalidate the cache.

      14:36:53 Connecting to https://api.github.com using myuser/****** (myuser-github username/password with personal token)java.io.FileNotFoundException:https://api.github.com/repos/xxxxxxxxxx/xxxxxxxx/git/refs/tags/v0.13.3 at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:625) at org.kohsuke.github.Requester.parse(Requester.java:607) at org.kohsuke.github.Requester._to(Requester.java:285)
      
      Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/git/refs/#get-a-reference"} at org.kohsuke.github.Requester.handleApiError(Requester.java:699) at org.kohsuke.github.Requester._to(Requester.java:306) at org.kohsuke.github.Requester.to(Requester.java:247) at org.kohsuke.github.GHRepository.getRef(GHRepository.java:891) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1457) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:583) at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:95) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) 

        Attachments

          Issue Links

            Activity

            Hide
            c9s Yo-An Lin added a comment -

            Tag update makes Jenkins failed to get correct tag information.

            Currently we can only create a new tag to bypass this bug.

            Show
            c9s Yo-An Lin added a comment - Tag update makes Jenkins failed to get correct tag information. Currently we can only create a new tag to bypass this bug.
            Hide
            c9s Yo-An Lin added a comment -

            This is related to the okhttp used by the GitHub branch source plugin:
            src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java LINE 393

                        OkHttpClient client = new OkHttpClient().setProxy(getProxy(host));
                            if (cacheDir != null) {
                                Cache cache = new Cache(cacheDir, cacheSize * 1024L * 1024L);
                                client.setCache(cache);
                            }
            

            And here is the related part of the okhttp client https://github.com/square/okhttp/wiki/Recipes#response-caching

            Show
            c9s Yo-An Lin added a comment - This is related to the okhttp used by the GitHub branch source plugin: src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java LINE 393 OkHttpClient client = new OkHttpClient().setProxy(getProxy(host)); if (cacheDir != null ) { Cache cache = new Cache(cacheDir, cacheSize * 1024L * 1024L); client.setCache(cache); } And here is the related part of the okhttp client https://github.com/square/okhttp/wiki/Recipes#response-caching
            Hide
            c9s Yo-An Lin added a comment - - edited

            When I use cURL to call the GitHub API to get an existing tag, I got this:

            HTTP/1.1 200 OK
            Server: GitHub.com
            Date: Wed, 26 Dec 2018 09:35:06 GMT
            Content-Type: application/json; charset=utf-8
            Content-Length: 386
            Status: 200 OK
            X-RateLimit-Limit: 5000
            X-RateLimit-Remaining: 4999
            X-RateLimit-Reset: 1545820506
            Cache-Control: private, max-age=60, s-maxage=60
            Vary: Accept, Authorization, Cookie, X-GitHub-OTP
            ETag: "xxxxxxxxxxxxxxxxxxxxxxxxx"
            Last-Modified: Wed, 26 Dec 2018 05:04:44 GMT
            X-Poll-Interval: 300
            X-OAuth-Scopes: repo
            X-Accepted-OAuth-Scopes: repo
            X-GitHub-Media-Type: github.v3; format=json
            Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP,
            X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes,
            X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
            Access-Control-Allow-Origin: *
            Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
            X-Frame-Options: deny
            X-Content-Type-Options: nosniff
            X-XSS-Protection: 1; mode=block
            Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
            Content-Security-Policy: default-src 'none'
            
            Show
            c9s Yo-An Lin added a comment - - edited When I use cURL to call the GitHub API to get an existing tag, I got this: HTTP/1.1 200 OK Server: GitHub.com Date: Wed, 26 Dec 2018 09:35:06 GMT Content-Type: application/json; charset=utf-8 Content-Length: 386 Status: 200 OK X-RateLimit-Limit: 5000 X-RateLimit-Remaining: 4999 X-RateLimit-Reset: 1545820506 Cache-Control: private , max-age=60, s-maxage=60 Vary: Accept, Authorization, Cookie, X-GitHub-OTP ETag: "xxxxxxxxxxxxxxxxxxxxxxxxx" Last-Modified: Wed, 26 Dec 2018 05:04:44 GMT X-Poll-Interval: 300 X-OAuth-Scopes: repo X-Accepted-OAuth-Scopes: repo X-GitHub-Media-Type: github.v3; format=json Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type Access-Control-Allow-Origin: * Strict-Transport-Security: max-age=31536000; includeSubdomains; preload X-Frame-Options: deny X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin Content-Security-Policy: default -src 'none'
            Hide
            c9s Yo-An Lin added a comment -
                            } else if (head instanceof GitHubTagSCMHead) {
                                GitHubTagSCMHead tagHead = (GitHubTagSCMHead) head;
                                GHRef tag = ghRepository.getRef("tags/" + tagHead.getName());    // <------------- using cached response
                                String sha = tag.getObject().getSha();
                                if ("tag".equalsIgnoreCase(tag.getObject().getType())) {
                                    // annotated tag object
                                    GHTagObject tagObject = ghRepository.getTagObject(sha);
                                    // we want the sha of the tagged commit not the tag object
                                    sha = tagObject.getObject().getSha();
                                }
                                return new GitTagSCMRevision(tagHead, sha);
            
            Show
            c9s Yo-An Lin added a comment - } else if (head instanceof GitHubTagSCMHead) { GitHubTagSCMHead tagHead = (GitHubTagSCMHead) head; GHRef tag = ghRepository.getRef( "tags/" + tagHead.getName()); // <------------- using cached response String sha = tag.getObject().getSha(); if ( "tag" .equalsIgnoreCase(tag.getObject().getType())) { // annotated tag object GHTagObject tagObject = ghRepository.getTagObject(sha); // we want the sha of the tagged commit not the tag object sha = tagObject.getObject().getSha(); } return new GitTagSCMRevision(tagHead, sha);
            Show
            c9s Yo-An Lin added a comment - Pull request https://github.com/jenkinsci/github-branch-source-plugin/pull/197
            Hide
            bitwiseman Liam Newman added a comment -

            Yo-An Lin

            Could you try the patched version of the github-branch-source-plugin linked in the description of https://issues.jenkins-ci.org/browse/JENKINS-57411 ? 

            I'm wondering if perhaps the using a more recent version of okhttp3 will address these issues with the cache. 

             

             

            Show
            bitwiseman Liam Newman added a comment - Yo-An Lin Could you try the patched version of the github-branch-source-plugin linked in the description of  https://issues.jenkins-ci.org/browse/JENKINS-57411  ?  I'm wondering if perhaps the using a more recent version of okhttp3 will address these issues with the cache.     

              People

              • Assignee:
                lanwen Kirill Merkushev
                Reporter:
                c9s Yo-An Lin
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: