So you think you're just gonna `npm install`? Think again

We embraced the birth of package lockfiles with open arms, which introduced: deterministic installations across different environments, and enforced dependency expectations across team collaboration.

Life is good! Or so I thought…
what would have happened had I slipped a change into the project’s package.json file but had forgotten to commit the lockfile along side of it?

Both Yarn, and npm act the same during dependency installation . When they detect an inconsistency between the project’s package.json and the lockfile, they compensate for such change based on the package.json manifest by installing different versions than those that were recorded in the lockfile.

This kind of situation can be hazardous for build and production environments as they could pull in unintended package versions and render the entire benefit of a lockfile futile.

Luckily, there is a way to tell both Yarn and npm to adhere to a specified set of dependencies and their versions by referencing them from the lockfile. Any inconsistency will abort the installation. The command line should read as follows:

• If you’re using Yarn, run yarn install --frozen-lockfile
• If you’re using npm run npm ci

--

I also wrote a complete 10 npm security best practices you should adopt in a post that includes a high-resolution printable PDF like the snippet you see below.

Thanks for reading and to Juan Picado from the Verdaccio team who worked with me on it. Check it out

Comments

[Boris Jamot ✊ /]

To prevent from mismatch between package.json and package.lock (or between whatever-dependency-manager.json and .lock files) just don't edit them manually. Always use the package manager, whether it be for adding, upgrading or removing a dep.

And you could also add a check in your pre-commit hook to be sure that no one in the team will commit something wrong.

[Liran Tal]

Hey Boris! Nice to e-meet you :)

The issue that can arise with out-of-sync lockfiles isn't due to people editing them manually. To be honest, I don't think anyone does.

The problem is with changes people might do to package.json. They might not even edit that manually, but instead it would happen while developers will resolve a merge conflict, and unintentionally a change in the package manifest will slip in.

Pre-commit hooks - I'm all up for those! :-)

[Mihail Malo]

I sometimes edit them "by hand" with Version Lens

There's also the issue of pnpm vs yarn (and sometimes even npm, imagine that), because they create separate lockfiles.
So, whichever you used to edit package.json, you will still probably need to run npm install --package-lock-only and, if the default for the project is yarn, literally run a whole yarn. (Or is there a --lockfile-only equivalent for yarn?)

[Liran Tal]

I'm not using version lens. Does that also care to update the package lock file when you do that? (as in, not update it "by hand" too, but actually re-ran the locking through the relevant package manager.

Also, I think you mean --package-lock-only? In yarn, just a yarn install will resolve conflicts.

The problem is not how the files get out of sync, but rather the fact that you'd not want to propagate this 'out of sync' behavior to your CIs or other devs (which is even worse as it will just drive more confusion).

[Mihail Malo]

Does that also care

It does not. It totally should, but at the moment it does not. Literally just writes to the file for you.

Yeah, npm i --package-lock-only && pnpm i --lockfile-only

The yarn [install] will also actually do the install, which is slower and clobbers my nice node_modules made by pnpm.

[Liran Tal]

Not so ideal when the package.json alone changes.
These are changes that aren't as soft as other things that you can force on the team by putting them on commit hooks.

I think we agree that regardless, your CI/build systems should work with the pure lock file and not try resolve.

[Erick Wilder]

We introduced in our team a workflow that uses both npm audit and npm ci in our CI pipeline isolating the installation in a docker container. If someone introduces or changes dependencies it hardly goes unnoticed. Relying solely on our best intentions sometimes is not enough and it's necessary to automate some tasks. This is a good example where automation can save you from some troubles.

[Liran Tal]

How exactly are you combining audit and npm's install/ci?
I didn't understand how that's related to docker. Do you mean you are doing this inside the docker image you're building?

[Erick Wilder]

Yes, exactly. We build everything inside a docker image (using npm ci) and later on we run a step in our pipeline with npm audit (and other homegrown checks) to ensure that inconsistent state or malicious code never goes to production. There is also some integration with slack to notify everyone that we have to fix it.

[Liran Tal]

Sounds good.

If I could help you in getting started with Snyk for auditing, monitoring etc I'd be more than happy to connect over DM or something.

One point that stands out is, while your pipeline checks for vulnerabilities, if you didn't deploy/run CI for say 2 weeks, and during this time a vulnerability was disclosed, then you wouldn't catch it, where-as with Snyk we constantly monitor your package manifest snapshots, alert, and open PRs that automatically fix and relock the relevant lockfile.

[Brenda 🐱]

Question: if you have issues with the lock file, could you just delete it and rerun npm install to recreate it with the correct entries? Or is that bad practice?

[Liran Tal]

Glad you asked! It's an easy mistake to make.
That would be a bad practice. It would mean that the lockfile completely loses of it's original purpose.

If you do that in a project/lib that you alone maintain perhaps that's not the end of the world (still not ideal IMO though), but as part of a team it would completely make the lockfile redundant.

[Brenda 🐱]

So if you have the same dependencies in your different environments (prod, dev, test, etc.) and your merge causes a package lock file conflict, it would
be better to manually edit the package lock file?

I think I need to look more into the value and purpose of the lock file.

[Liran Tal]

No. In no circumstance it would be appropriate to manually edit a lock file. Generally speaking if you need to "fix" a lock file. It's better to use the package manager built-in tooling for it. For example, yarn supports automatic resolution of conflict merge issues that fix the lock file.

If that doesn't help. You can update the lock file by running the install/remove commands using the package manager so that it takes care of updating the lock file for you.

Regardless of this, during CI builds you want to maintain reproducible builds and be strict on what packages you install.

Maybe it wasn't very clear in the post but to provide this as an example:

1. package.json defines dependency myCoolModule version ^1.2.3
2. lockfile has myCoolModule version 0.0.1

At this point, it doesn't really matter how (1) and (2) are different (whether merge conflict issue, or someone forgot to sync the package.json with the lock file)

1. if your CI server runs npm install then it would detect that the package lock is out of date and both update it (which means it modifies the git copy it cloned, bad to begin with), and brings in ^1.2.3 which is what the package.json defined, but not the lock file. Therefore, making the lock file useless.

Instead, you'd want in your CI to run npm ci which reads the lock file as a source of truth, and will bring you myCoolModule version 0.0.1. If it breaks your tests - all the better, it means that the build stopped you from shipping something you didn't expect.

Hope it's a bit clearer :-)

[Brenda 🐱]

I see. Thanks for the reply.

[Saurabh Daware 🌻]

Hi, great article! I have a question:
So I've seen dependabot updating package-lock file in its commits, so how does it work? Does changing the integrity hash in package-lock.json change things ?

[Liran Tal]

if you run an npm install with npm ci then npm will only consult the lockfile, and so changing the lockfile directly will work.