Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-496

Setup a local (to CI) NPM registry mirror

    Details

    • Similar Issues:

      Description

      Downloading of NPM packages from the public registry can be flaky, causing CI builds to randomly fail. Would be good to have a local (more reliable) registry mirror.

      Mirroring options:

      1. a private NPM registry instance,
      2. artifactory
      3. nexus

        Attachments

          Issue Links

            Activity

            tfennelly Tom FENNELLY created issue -
            Hide
            tfennelly Tom FENNELLY added a comment -

            Should have created this in INFRA (I think)

            Show
            tfennelly Tom FENNELLY added a comment - Should have created this in INFRA (I think)
            tfennelly Tom FENNELLY made changes -
            Field Original Value New Value
            Status Open [ 1 ] Closed [ 6 ]
            Resolution Won't Do [ 10001 ]
            tfennelly Tom FENNELLY made changes -
            Project Jenkins [ 10172 ] Infrastructure [ 10301 ]
            Key JENKINS-31742 INFRA-496
            Workflow JNJira [ 167183 ] classic default workflow [ 167183 ]
            Component/s ci [ 18920 ]
            Component/s core [ 15593 ]
            tfennelly Tom FENNELLY made changes -
            Resolution Won't Do [ 10001 ]
            Status Closed [ 6 ] Reopened [ 4 ]
            Hide
            rtyler R. Tyler Croy added a comment -

            I'm unclear on why this is required right now Tom FENNELLY. I've seen flakey builds before at previous workplaces but I'm not sure where in our project infrastructure this is impacting right now

            Show
            rtyler R. Tyler Croy added a comment - I'm unclear on why this is required right now Tom FENNELLY . I've seen flakey builds before at previous workplaces but I'm not sure where in our project infrastructure this is impacting right now
            Hide
            tfennelly Tom FENNELLY added a comment -

            R. Tyler Croy Workflow CI build has failed on a few occasions due to intermittent connection failures to the NPM registry.

            Show
            tfennelly Tom FENNELLY added a comment - R. Tyler Croy Workflow CI build has failed on a few occasions due to intermittent connection failures to the NPM registry.
            Hide
            rtyler R. Tyler Croy added a comment -

            Tom FENNELLY will you link some of those failed builds to this JIRA so I have some concrete problems to weigh the priority of this request with?

            Show
            rtyler R. Tyler Croy added a comment - Tom FENNELLY will you link some of those failed builds to this JIRA so I have some concrete problems to weigh the priority of this request with?
            Hide
            tfennelly Tom FENNELLY added a comment -

            Hasn't failed in the recent build history of workflow, so will downgrade the priority of this.

            Show
            tfennelly Tom FENNELLY added a comment - Hasn't failed in the recent build history of workflow , so will downgrade the priority of this.
            tfennelly Tom FENNELLY made changes -
            Priority Major [ 3 ] Minor [ 4 ]
            rtyler R. Tyler Croy made changes -
            Rank Ranked higher
            Hide
            rtyler R. Tyler Croy added a comment -

            This should be something we can do via repo.jenkins-ci.org relatively easily at some point in the future.

            Since that mirror would still be public, it has the potential to have the same availability issues (from the builders' perspective) that the public package mirrors do.

            Show
            rtyler R. Tyler Croy added a comment - This should be something we can do via repo.jenkins-ci.org relatively easily at some point in the future. Since that mirror would still be public, it has the potential to have the same availability issues (from the builders' perspective) that the public package mirrors do.
            rtyler R. Tyler Croy made changes -
            Assignee Tom FENNELLY [ tfennelly ]
            rtyler R. Tyler Croy made changes -
            Link This issue is related to WEBSITE-94 [ WEBSITE-94 ]
            rtyler R. Tyler Croy made changes -
            Link This issue is related to WEBSITE-88 [ WEBSITE-88 ]
            rtyler R. Tyler Croy made changes -
            Link This issue is related to INFRA-563 [ INFRA-563 ]
            rtyler R. Tyler Croy made changes -
            Assignee R. Tyler Croy [ rtyler ]
            Hide
            tfennelly Tom FENNELLY added a comment -

            If we do this, I wonder if it could be done such that the mirror is only used for CI server builds i.e. the CI job needs to be configured to use the registry, otherwise the public NPM registry is used (e.g. local dev builds)?

            Show
            tfennelly Tom FENNELLY added a comment - If we do this, I wonder if it could be done such that the mirror is only used for CI server builds i.e. the CI job needs to be configured to use the registry, otherwise the public NPM registry is used (e.g. local dev builds)?
            Hide
            tfennelly Tom FENNELLY added a comment -

            And yeah, like you say, it probably only helps make things more stable if the mirror is local to the CI infra.

            Show
            tfennelly Tom FENNELLY added a comment - And yeah, like you say, it probably only helps make things more stable if the mirror is local to the CI infra.
            Hide
            tfennelly Tom FENNELLY added a comment -

            We're getting bitten by this again atm ... CI jobs failing.

            Show
            tfennelly Tom FENNELLY added a comment - We're getting bitten by this again atm ... CI jobs failing.
            Hide
            aheritier Arnaud Héritier added a comment -

            Hi Tom,

            If your builds are on DEV@cloud we are impacted by an issue from NPM : http://status.cloudbees.com/incidents/kg1yzxkgtsnx

            Sorry for this.

            Show
            aheritier Arnaud Héritier added a comment - Hi Tom, If your builds are on DEV@cloud we are impacted by an issue from NPM : http://status.cloudbees.com/incidents/kg1yzxkgtsnx Sorry for this.
            Hide
            tfennelly Tom FENNELLY added a comment -

            Arnaud Héritier Thanks Arnaud. Yes, this is the issue I was referring to in the previous comment.

            Show
            tfennelly Tom FENNELLY added a comment - Arnaud Héritier Thanks Arnaud. Yes, this is the issue I was referring to in the previous comment.
            rtyler R. Tyler Croy made changes -
            Status Reopened [ 4 ] In Progress [ 3 ]
            Hide
            rtyler R. Tyler Croy added a comment -

            An npm registry should be set up as a caching repo here: https://repo.jenkins-ci.org/api/npm/npm

            Please test it out and let me know if there are issues

            Show
            rtyler R. Tyler Croy added a comment - An npm registry should be set up as a caching repo here: https://repo.jenkins-ci.org/api/npm/npm Please test it out and let me know if there are issues
            rtyler R. Tyler Croy made changes -
            Assignee R. Tyler Croy [ rtyler ] Tom FENNELLY [ tfennelly ]
            Hide
            rtyler R. Tyler Croy added a comment -

            As far as I can tell, this has been completed. I ran this command npm install azure --registry https://repo.jenkins-ci.org/api/npm/npm and that seemed to work fine.

            Show
            rtyler R. Tyler Croy added a comment - As far as I can tell, this has been completed. I ran this command npm install azure --registry https://repo.jenkins-ci.org/api/npm/npm and that seemed to work fine.
            rtyler R. Tyler Croy made changes -
            Status In Progress [ 3 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]

              People

              • Assignee:
                tfennelly Tom FENNELLY
                Reporter:
                tfennelly Tom FENNELLY
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: