Details

    • Similar Issues:

      Description

      Shrinkwrap is an unbearable pain in the ass (locally known as "shrinkcrap"):

      1. It just stops working (without error on install etc) if the version numbers of packages do not line up perfectly in package.json, npm-shrinkwrap.json and node_modules.
      2. Getting it back into a working state (where an npm install updates npm-shrinkwrap.json etc) can take ages i.e. can take up to an hour (some people have said it takes them multiple hours).
      3. Regenration of the npm-shrinkwrap.json from scratch is a horrible option too because that then results in commit diffs with line change counts in the 1,000s.
      4. From what I (TF) can remember, it's not possible to use tools like slink or linklocal when using shrinkwrap. Can't remember the issue(s).
        1. I (CM) believe I'd had that working fine provided I set a new/beta version in the module I'm installing, install it, then slink it. (see comment below)
      5. Works with maven build - we will need some changes here I think ... not sure if frontend will work for anything other than npm when it comes to installing packages.

      Scope

      1. Decent errors when package installs do not do as expected i.e. update relevant build files.
      2. Easy and fast process for getting things back into a working order if things are inconsistent e.g. regeneration of whatever fills the roll of npm-shrinkwrap.json is simple and does not result in line change counts in the 1,000s.
      3. Possible to work in dev mode and use tools like slink or linklocal.

      Possibilities

      1. Switch to using Yarn. Cliff Meyers has already done some experimenting with this.
        1. Yarn uses a lockfile by default, and has explicit commands that differentiate between adding a dependency (add) and installing a project's dependencies (install).
        2. Yarn supports link/unlink, so it may eliminate the need for slink
        3. Link to commit/branch in the JDL where I experimented using yarn: https://github.com/jenkinsci/jenkins-design-language/commit/2538a4f08be8ec5adc99d41b09c975ba49dbbc94
      2. Write some node.js scripts that help streamline npm/shrink operations, and perform additional validation (some of validation already exists and is called from the Jenkinsfile)
        1. I (CM) find the trickiest scenario to manage is when there is a conflict during merge. I usually follow this process:
          1. Accept "theirs" for package.json and npm-shrinkwrap.json
          2. Run "npm install" so that package.json, npm-shrinkwrap.json and node_modules are in sync
          3. Individually install the correct version of each package that I have changed: "npm install cliff-code@1.5.2 -S -E"
          4. We could write a script that does this? Perhaps one that does it across multiple/all modules at once? For example, we could use "./setlib.js blueocean-core-js@1.5.2" which would step into all projects, find those which depend on "blueocean-core-js" and run the correct install command in each
      3. Give up on JS package management entirely and just check things into node_modules.

        Attachments

          Issue Links

            Activity

            Hide
            michaelneale Michael Neale added a comment -

            cc Tom FENNELLY epic has been created

            Show
            michaelneale Michael Neale added a comment - cc Tom FENNELLY epic has been created
            Hide
            cliffmeyers Cliff Meyers added a comment - - edited

            Tom FENNELLY you're right, slinking is messier than I'd hoped. You have to do the following:

            1. Set a new version in the package.json of the module you're working on
            2. Hand-edit the package.json and npm-shrinkwrap.json of the module(s) that depend on it
            3. Run slink from those modules
            Show
            cliffmeyers Cliff Meyers added a comment - - edited Tom FENNELLY you're right, slinking is messier than I'd hoped. You have to do the following: Set a new version in the package.json of the module you're working on Hand-edit the package.json and npm-shrinkwrap.json of the module(s) that depend on it Run slink from those modules
            Hide
            cliffmeyers Cliff Meyers added a comment -

            This project might be looking at as well, from the yarn guys:
            https://lernajs.io/

            I haven't dug deep on it yet but it seems related to things we are doing.

            Show
            cliffmeyers Cliff Meyers added a comment - This project might be looking at as well, from the yarn guys: https://lernajs.io/ I haven't dug deep on it yet but it seems related to things we are doing.
            Hide
            tfennelly Tom FENNELLY added a comment -

            Has also been pointed out to us (by Yoann Dubreuil ) that we can't use an NPM registry (or there are serious issues using one) because of shrinkwrap.

            Show
            tfennelly Tom FENNELLY added a comment - Has also been pointed out to us (by Yoann Dubreuil ) that we can't use an NPM registry (or there are serious issues using one) because of shrinkwrap.
            Hide
            cliffmeyers Cliff Meyers added a comment -

            To add to the prior point, my recollection is that since shrinkwrap holds fully-qualified URL's to each artifact, the registry being used at the time of shrinkwrapping in what npm will use. We had experimented with doing a search/replace for the main registry path in shrinkwrap.json which I believe did work - perhaps Yoann Dubreuil can confirm. However it's not great since a CI build would behave differently from the user's local machine, unless each developer configured their npmrc file to use the same registry.

            My recollection is that Yarn wouldn't have this problem, as the Lockfiles do not specify URL's and I know it supports registries (see: https://github.com/yarnpkg/yarn/issues/606 )

            Show
            cliffmeyers Cliff Meyers added a comment - To add to the prior point, my recollection is that since shrinkwrap holds fully-qualified URL's to each artifact, the registry being used at the time of shrinkwrapping in what npm will use. We had experimented with doing a search/replace for the main registry path in shrinkwrap.json which I believe did work - perhaps Yoann Dubreuil can confirm. However it's not great since a CI build would behave differently from the user's local machine, unless each developer configured their npmrc file to use the same registry. My recollection is that Yarn wouldn't have this problem, as the Lockfiles do not specify URL's and I know it supports registries (see: https://github.com/yarnpkg/yarn/issues/606 )

              People

              • Assignee:
                tfennelly Tom FENNELLY
                Reporter:
                tfennelly Tom FENNELLY
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: