Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-1224

Public repo should mirror OSSRH releases

    Details

    • Type: Improvement
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: artifactory
    • Labels:
      None
    • Similar Issues:

      Description

      For my company's internal repository manager, our group repositories include a mirror of https://oss.sonatype.org/content/repositories/releases/ (the OSSRH releases repository which gets synched periodically to Central). This is very handy because if we do a release to OSSRH—for example, mvn -f .../groovy-cps release:{prepare,perform}—then CI jobs pick it up immediately, without needing to wait for the Central synch.

      Unfortunately the Jenkins repository does not seem to do this, resulting in failures like https://ci.jenkins.io/job/Plugins/job/workflow-cps-plugin/job/PR-143/2/consoleFull

      Failed to execute goal on project workflow-cps: Could not resolve dependencies for project org.jenkins-ci.plugins.workflow:workflow-cps:hpi:2.35-SNAPSHOT: Could not find artifact com.cloudbees:groovy-cps:jar:1.15 in azure (https://repo.azure.jenkins.io/public/)
      

      In this case https://oss.sonatype.org/content/repositories/releases/com/cloudbees/groovy-cps/1.15/groovy-cps-1.15.jar is a 200 while https://repo.maven.apache.org/maven2/com/cloudbees/groovy-cps/1.15/groovy-cps-1.15.jar is a 404 for a while, as is our https://repo.jenkins-ci.org/public/com/cloudbees/groovy-cps/1.15/groovy-cps-1.15.jar and so we have to wait a little while before being able to do integration testing.

      I see that http://repo.jenkins-ci.org/oss.sonatype.org-releases/com/cloudbees/groovy-cps/ exists but has nothing since January...?

      Now the artifact has appeared on Central but our mirror is still a 404; the https://repo.jenkins-ci.org/public/com/cloudbees/groovy-cps/ directory listing shows

      ...
      1.14-SNAPSHOT/            02-Jun-2017 22:28    -
      1.15/->                       -    -
      1.15-SNAPSHOT/            09-Jun-2017 19:09    -
      ...
      

      and https://repo.jenkins-ci.org/public/com/cloudbees/groovy-cps/1.15/ shows the right artifacts but all looking like broken symlinks, which is even weirder:

      ...
      groovy-cps-1.15-tests.jar.sha1->            -    -
      groovy-cps-1.15.jar->                       -    -
      groovy-cps-1.15.jar.asc->                   -    -
      

        Attachments

          Issue Links

            Activity

            Hide
            danielbeck Daniel Beck added a comment -

            I lowered the "metadata cache" from 12 to 2 hours, whatever that means.

            The dir listing is still old, but the maven-metadata is new after I requested it, so I'd expect this to work now.

            Show
            danielbeck Daniel Beck added a comment - I lowered the "metadata cache" from 12 to 2 hours, whatever that means. The dir listing is still old, but the maven-metadata is new after I requested it, so I'd expect this to work now.
            Show
            jglick Jesse Glick added a comment - I think this is working. After a recent release of https://oss.sonatype.org/content/repositories/releases/org/kohsuke/groovy-sandbox/1.14/  I can see https://repo.jenkins-ci.org/public/org/kohsuke/groovy-sandbox/1.14/  even though http://repo1.maven.org/maven2/org/kohsuke/groovy-sandbox/1.14/  is a 404.
            Hide
            jglick Jesse Glick added a comment -

            After cutting a release requested by Sam Van Oort, this does not seem to be working:

            $ for repo in https://oss.sonatype.org/content/groups/public/ https://repo.jenkins-ci.org/public/; do for art in net/java/sezpoz/pom/1.13/pom-1.13.pom net/java/sezpoz/sezpoz/1.13/sezpoz-1.13.jar; do url=${repo}${art}; echo $url; curl -IL $url; done; done
            https://oss.sonatype.org/content/groups/public/net/java/sezpoz/pom/1.13/pom-1.13.pom
            HTTP/1.1 200 OK
            Accept-Ranges: bytes
            Content-Length: 4141
            Content-Type: application/xml
            Date: Fri, 22 Jun 2018 14:08:21 GMT
            ETag: "{SHA1{aa631d8bdafb4c3bd53072ffb04b7dbadb7117a8}}"
            Last-Modified: Fri, 22 Jun 2018 13:31:48 GMT
            Server: nginx
            X-Content-Type-Options: nosniff
            X-Frame-Options: SAMEORIGIN
            Connection: keep-alive
            
            https://oss.sonatype.org/content/groups/public/net/java/sezpoz/sezpoz/1.13/sezpoz-1.13.jar
            HTTP/1.1 200 OK
            Accept-Ranges: bytes
            Content-Length: 28254
            Content-Type: application/java-archive
            Date: Fri, 22 Jun 2018 14:08:21 GMT
            ETag: "{SHA1{822ac382900587d0a7d6aaec7b1859f1cf5de2fb}}"
            Last-Modified: Fri, 22 Jun 2018 13:32:10 GMT
            Server: nginx
            X-Content-Type-Options: nosniff
            X-Frame-Options: SAMEORIGIN
            Connection: keep-alive
            
            https://repo.jenkins-ci.org/public/net/java/sezpoz/pom/1.13/pom-1.13.pom
            HTTP/2 200 
            date: Fri, 22 Jun 2018 14:08:22 GMT
            content-type: application/x-maven-pom+xml
            content-length: 4141
            server: Artifactory/5.11.3
            x-artifactory-id: aolshared1b-jenkinsci
            last-modified: Fri, 22 Jun 2018 13:31:48 GMT
            etag: aa631d8bdafb4c3bd53072ffb04b7dbadb7117a8
            x-checksum-sha1: aa631d8bdafb4c3bd53072ffb04b7dbadb7117a8
            x-checksum-sha256: 81d06985e44b1993a310bccd062e83fad4af8b3654638c4a44c1fdeafd1f4651
            x-checksum-md5: 36da9a6e6812a55313617bbc3acbfb3c
            accept-ranges: bytes
            x-artifactory-filename: pom-1.13.pom
            content-disposition: attachment; filename="pom-1.13.pom"; filename*=UTF-8''pom-1.13.pom
            x-node: nginx-shared4b.gcoss-use1
            via: 1.1 google
            alt-svc: clear
            
            https://repo.jenkins-ci.org/public/net/java/sezpoz/sezpoz/1.13/sezpoz-1.13.jar
            HTTP/2 404 
            date: Fri, 22 Jun 2018 14:08:22 GMT
            content-type: application/json;charset=ISO-8859-1
            server: Artifactory/5.11.3
            x-artifactory-id: aolshared1b-jenkinsci
            x-node: nginx-shared3c.gcoss-use1
            via: 1.1 google
            alt-svc: clear
            
            
            Show
            jglick Jesse Glick added a comment - After cutting a release requested by Sam Van Oort , this does not seem to be working: $ for repo in https://oss.sonatype.org/content/groups/public/ https://repo.jenkins-ci.org/public/; do for art in net/java/sezpoz/pom/1.13/pom-1.13.pom net/java/sezpoz/sezpoz/1.13/sezpoz-1.13.jar; do url=${repo}${art}; echo $url; curl -IL $url; done; done https://oss.sonatype.org/content/groups/public/net/java/sezpoz/pom/1.13/pom-1.13.pom HTTP/1.1 200 OK Accept-Ranges: bytes Content-Length: 4141 Content-Type: application/xml Date: Fri, 22 Jun 2018 14:08:21 GMT ETag: "{SHA1{aa631d8bdafb4c3bd53072ffb04b7dbadb7117a8}}" Last-Modified: Fri, 22 Jun 2018 13:31:48 GMT Server: nginx X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN Connection: keep-alive https://oss.sonatype.org/content/groups/public/net/java/sezpoz/sezpoz/1.13/sezpoz-1.13.jar HTTP/1.1 200 OK Accept-Ranges: bytes Content-Length: 28254 Content-Type: application/java-archive Date: Fri, 22 Jun 2018 14:08:21 GMT ETag: "{SHA1{822ac382900587d0a7d6aaec7b1859f1cf5de2fb}}" Last-Modified: Fri, 22 Jun 2018 13:32:10 GMT Server: nginx X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN Connection: keep-alive https://repo.jenkins-ci.org/public/net/java/sezpoz/pom/1.13/pom-1.13.pom HTTP/2 200 date: Fri, 22 Jun 2018 14:08:22 GMT content-type: application/x-maven-pom+xml content-length: 4141 server: Artifactory/5.11.3 x-artifactory-id: aolshared1b-jenkinsci last-modified: Fri, 22 Jun 2018 13:31:48 GMT etag: aa631d8bdafb4c3bd53072ffb04b7dbadb7117a8 x-checksum-sha1: aa631d8bdafb4c3bd53072ffb04b7dbadb7117a8 x-checksum-sha256: 81d06985e44b1993a310bccd062e83fad4af8b3654638c4a44c1fdeafd1f4651 x-checksum-md5: 36da9a6e6812a55313617bbc3acbfb3c accept-ranges: bytes x-artifactory-filename: pom-1.13.pom content-disposition: attachment; filename="pom-1.13.pom"; filename*=UTF-8''pom-1.13.pom x-node: nginx-shared4b.gcoss-use1 via: 1.1 google alt-svc: clear https://repo.jenkins-ci.org/public/net/java/sezpoz/sezpoz/1.13/sezpoz-1.13.jar HTTP/2 404 date: Fri, 22 Jun 2018 14:08:22 GMT content-type: application/json;charset=ISO-8859-1 server: Artifactory/5.11.3 x-artifactory-id: aolshared1b-jenkinsci x-node: nginx-shared3c.gcoss-use1 via: 1.1 google alt-svc: clear
            Hide
            jglick Jesse Glick added a comment -

            And still true despite the fact that Central synch worked!

            https://repo.maven.apache.org/maven2/net/java/sezpoz/pom/1.13/pom-1.13.pom
            HTTP/2 200 
            etag: "36da9a6e6812a55313617bbc3acbfb3c"
            content-type: text/xml
            last-modified: Fri, 22 Jun 2018 13:31:48 GMT
            x-checksum-md5: 36da9a6e6812a55313617bbc3acbfb3c
            x-checksum-sha1: aa631d8bdafb4c3bd53072ffb04b7dbadb7117a8
            via: 1.1 varnish
            accept-ranges: bytes
            date: Fri, 22 Jun 2018 14:10:08 GMT
            via: 1.1 varnish
            age: 0
            x-served-by: cache-iad2139-IAD, cache-pdk17841-PDK
            x-cache: HIT, MISS
            x-cache-hits: 1, 0
            x-timer: S1529676609.529468,VS0,VE13
            content-length: 4141
            
            https://repo.maven.apache.org/maven2/net/java/sezpoz/sezpoz/1.13/sezpoz-1.13.jar
            HTTP/2 200 
            etag: "f28b0ae5e0677aeb5d91a6811160cbcc"
            content-type: application/java-archive
            last-modified: Fri, 22 Jun 2018 13:32:10 GMT
            x-checksum-md5: f28b0ae5e0677aeb5d91a6811160cbcc
            x-checksum-sha1: 822ac382900587d0a7d6aaec7b1859f1cf5de2fb
            via: 1.1 varnish
            accept-ranges: bytes
            date: Fri, 22 Jun 2018 14:10:08 GMT
            via: 1.1 varnish
            age: 0
            x-served-by: cache-iad2151-IAD, cache-pdk17826-PDK
            x-cache: HIT, MISS
            x-cache-hits: 1, 0
            x-timer: S1529676609.665807,VS0,VE16
            content-length: 28254
            
            
            Show
            jglick Jesse Glick added a comment - And still true despite the fact that Central synch worked! https://repo.maven.apache.org/maven2/net/java/sezpoz/pom/1.13/pom-1.13.pom HTTP/2 200 etag: "36da9a6e6812a55313617bbc3acbfb3c" content-type: text/xml last-modified: Fri, 22 Jun 2018 13:31:48 GMT x-checksum-md5: 36da9a6e6812a55313617bbc3acbfb3c x-checksum-sha1: aa631d8bdafb4c3bd53072ffb04b7dbadb7117a8 via: 1.1 varnish accept-ranges: bytes date: Fri, 22 Jun 2018 14:10:08 GMT via: 1.1 varnish age: 0 x-served-by: cache-iad2139-IAD, cache-pdk17841-PDK x-cache: HIT, MISS x-cache-hits: 1, 0 x-timer: S1529676609.529468,VS0,VE13 content-length: 4141 https://repo.maven.apache.org/maven2/net/java/sezpoz/sezpoz/1.13/sezpoz-1.13.jar HTTP/2 200 etag: "f28b0ae5e0677aeb5d91a6811160cbcc" content-type: application/java-archive last-modified: Fri, 22 Jun 2018 13:32:10 GMT x-checksum-md5: f28b0ae5e0677aeb5d91a6811160cbcc x-checksum-sha1: 822ac382900587d0a7d6aaec7b1859f1cf5de2fb via: 1.1 varnish accept-ranges: bytes date: Fri, 22 Jun 2018 14:10:08 GMT via: 1.1 varnish age: 0 x-served-by: cache-iad2151-IAD, cache-pdk17826-PDK x-cache: HIT, MISS x-cache-hits: 1, 0 x-timer: S1529676609.665807,VS0,VE16 content-length: 28254
            Hide
            jglick Jesse Glick added a comment -

            Seems to be broken. I think because the releases repo is now redirecting to Central:

            $ curl -IL https://oss.sonatype.org/content/repositories/releases/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom
            HTTP/2 302 
            server: awselb/2.0
            date: Thu, 25 Apr 2019 18:55:32 GMT
            content-type: text/html
            content-length: 126
            location: https://repo1.maven.org:443/content/repositories/releases/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom
            
            HTTP/2 301 
            location: https://repo1.maven.org/maven2/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom
            via: 1.1 varnish
            accept-ranges: bytes
            date: Thu, 25 Apr 2019 18:55:33 GMT
            via: 1.1 varnish
            age: 0
            x-served-by: cache-iad2147-IAD, cache-atl6250-ATL
            x-cache: HIT, MISS
            x-cache-hits: 1, 0
            x-timer: S1556218534.801525,VS0,VE14
            vary: Fastly-SSL,
            content-length: 0
            
            HTTP/2 404 
            last-modified: Wed, 10 Aug 2016 15:08:35 GMT
            etag: "1fb066da6a67c7c02962f59b4b8cd1ee"
            x-amz-error-code: NoSuchKey
            x-amz-error-message: The specified key does not exist.
            x-amz-error-detail-key: maven2/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom
            content-type: text/html
            via: 1.1 varnish
            accept-ranges: bytes
            date: Thu, 25 Apr 2019 18:55:33 GMT
            via: 1.1 varnish
            age: 79
            x-served-by: cache-iad2136-IAD, cache-atl6250-ATL
            x-cache: HIT, HIT
            x-cache-hits: 1, 1
            x-timer: S1556218534.836600,VS0,VE1
            content-length: 554
            
            $ curl -IL https://oss.sonatype.org/content/groups/public/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom
            HTTP/2 200 
            date: Thu, 25 Apr 2019 18:54:27 GMT
            content-type: application/xml
            content-length: 6497
            server: nginx
            x-frame-options: SAMEORIGIN
            x-content-type-options: nosniff
            accept-ranges: bytes
            etag: "{SHA1{80265804a627abeee9b056c96858bbed8cecb7bf}}"
            last-modified: Thu, 25 Apr 2019 18:42:21 GMT
            
            $ curl -IL https://repo.jenkins-ci.org/public/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom
            HTTP/2 404 
            date: Thu, 25 Apr 2019 18:53:59 GMT
            content-type: application/json;charset=ISO-8859-1
            server: Artifactory/6.8.4
            x-artifactory-id: aolshared1b-jenkinsci
            x-node: nginx-shared3b.gcoss-use1
            via: 1.1 google
            alt-svc: clear
            
            

            CC Devin Nusbaum. Yet another reason to run away screaming from OSSRH.

            Show
            jglick Jesse Glick added a comment - Seems to be broken. I think because the releases repo is now redirecting to Central: $ curl -IL https://oss.sonatype.org/content/repositories/releases/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom HTTP/2 302 server: awselb/2.0 date: Thu, 25 Apr 2019 18:55:32 GMT content-type: text/html content-length: 126 location: https://repo1.maven.org:443/content/repositories/releases/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom HTTP/2 301 location: https://repo1.maven.org/maven2/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom via: 1.1 varnish accept-ranges: bytes date: Thu, 25 Apr 2019 18:55:33 GMT via: 1.1 varnish age: 0 x-served-by: cache-iad2147-IAD, cache-atl6250-ATL x-cache: HIT, MISS x-cache-hits: 1, 0 x-timer: S1556218534.801525,VS0,VE14 vary: Fastly-SSL, content-length: 0 HTTP/2 404 last-modified: Wed, 10 Aug 2016 15:08:35 GMT etag: "1fb066da6a67c7c02962f59b4b8cd1ee" x-amz-error-code: NoSuchKey x-amz-error-message: The specified key does not exist. x-amz-error-detail-key: maven2/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom content-type: text/html via: 1.1 varnish accept-ranges: bytes date: Thu, 25 Apr 2019 18:55:33 GMT via: 1.1 varnish age: 79 x-served-by: cache-iad2136-IAD, cache-atl6250-ATL x-cache: HIT, HIT x-cache-hits: 1, 1 x-timer: S1556218534.836600,VS0,VE1 content-length: 554 $ curl -IL https://oss.sonatype.org/content/groups/public/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom HTTP/2 200 date: Thu, 25 Apr 2019 18:54:27 GMT content-type: application/xml content-length: 6497 server: nginx x-frame-options: SAMEORIGIN x-content-type-options: nosniff accept-ranges: bytes etag: "{SHA1{80265804a627abeee9b056c96858bbed8cecb7bf}}" last-modified: Thu, 25 Apr 2019 18:42:21 GMT $ curl -IL https://repo.jenkins-ci.org/public/com/cloudbees/groovy-cps/1.27/groovy-cps-1.27.pom HTTP/2 404 date: Thu, 25 Apr 2019 18:53:59 GMT content-type: application/json;charset=ISO-8859-1 server: Artifactory/6.8.4 x-artifactory-id: aolshared1b-jenkinsci x-node: nginx-shared3b.gcoss-use1 via: 1.1 google alt-svc: clear CC Devin Nusbaum . Yet another reason to run away screaming from OSSRH.

              People

              • Assignee:
                Unassigned
                Reporter:
                jglick Jesse Glick
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: