When not to use package-lock.json

I maintain over 200 repositories on GitHub and one of the most common PRs that I receive is someone adding package-lock.json or yarn.lock. These PRs are closed without merging because dependency lock files are not designed to be used by packages that are themselves dependencies of other packages.

What's going wrong?

Official NPM documentation encourages to commit package-lock.json files to the source code version control:

It is highly recommended you commit the generated package lock to source control: this will allow anyone else on your team, your deployments, your CI/continuous integration, and anyone else who runs npm install in your package source to get the exact same dependency tree that you were developing on. Additionally, the diffs from these changes are human-readable and will inform you of any changes npm has made to your node_modules, so you can notice if any transitive dependencies were updated, hoisted, etc.

– https://docs.npmjs.com/files/package-locks#using-locked-packages

Committing package-lock.json to the source code version control means that the project maintainers and CI systems will use a specific version of dependencies that may or may not match those defined in package.json. Because package-lock.json cannot be added to NPM registry (by design; see NPM shrinkwrap), projects that depend on a project that uses package-lock.json will themselves use package.json to resolve project's dependencies, i.e. what works for project maintainers/ CI systems might not work when the project is used as a dependency.

The origin of this misuse is NPM documentation. It should instead explain that package-lock.json should only be committed to the source code version control when the project is not a dependency of other projects, i.e. package-lock.json should only by committed to source code version control for top-level projects (programs consumed by the end user, not other programs).

I have already asked NPM to update the documentation, but it was archived without an action.

Responding to criticism

Some comments suggested that the biggest advantage of package-lock.json is that it allows to replicate the development environment.

I would support a variation of package-lock.json if it could somehow only apply to devDependencies. I can see some (albeit small and with tradeoffs) benefit to wanting your development environment not break if there is a broken release among your dependencies. I would personally prefer my environment to break and become aware that a dependency in my toolkit requires attention (and depending on the nature of the issue either offer help, subscribe to an issue or replace the dependency). After all, you can easily patch your dependency tree if you need to lock down a specific version for development purposes.

However, there is no such option and using lock files at the moment will create the risks described in this article – namely that the dependencies that you use do not match those that your users will depend on. Responsible development requires that your script works with the latest versions of dependencies satisfied by semver (and yes that includes transitive dependencies).

Comments

[Matthias]

After all, you can easily patch your dependency tree if you need to lock down a specific version for development purposes.

This is basically what a lockfile does... why not just use a lockfile?

[Matthias]

I would personally prefer my environment to break

This is an odd way to put it :) Rather, you probably mean that you want to know when there is a breakage, not that you want to have to drop everything and fix it right now and it is blocking you from your original task at hand. No one wants that kind of interruption.

There are a couple responses:

1. when? When is the break going to be detected? Are you executing these tests daily? hourly? In CI or locally? This is pointed out in the yarn article with babel as an example. In babel's case, consumers logged Issues about the breakage before babel maintainers even detected it themselves. People think a missing lockfile will magically notify them of breakages, but they must realize that they have to actually run the builds and tests to see the breakage.
2. Again, no one wants "surprise broken builds". They do want to fix broken builds, but they want to choose when to do it. Without a tracked lockfile, you can't revert to a "last known good install". You are forced to fix the build before you can work. With a lockfile, you can revert and continue working, and fix the issue later.

[Matthias]

I would support a variation of package-lock.json if it could somehow only apply to devDependencies.

The same reasoning applies to both dependencies and devDependencies, because both are very much used during development.

You described it as "wanting your development environment to not break", but realize this applies to both the build and the tests, and dependencies are used in tests.

[Chen Osipov]

Doesn't make any sense to me, we as a Library produce code, our code will be consumed by others so what?

Our code is not publishing the package-lock.json , that correct cause it's been used to build the code.

What is not ideal is when the CI break because of a 3r'd party and we invest 2-3 days of manpower to find the cause of this....

For example:
I had committed fix for a defect in my code and got a build error because of a 3r'd party changed, can't see any logic.

Libraries produce code that is used in other places so it's not matter is the code used by people or computers, the documentation on NPM documentation is correct and you need to put the package-lock.json in your source control it's there for a reason, and the reason is: stable software is SOLID

Once you don't have reproducible builds your software is not SOLID anymore.

[Nimit Aggarwal]

I think this is like a double edge sword if some dependency in my package's tree is updated with a vulnerable package. That would directly affect my package. The same thing that happened with event-stream snyk.io/blog/malicious-code-found-...

[kgrosvenor]

Always commit your lock files, it speeds up CI builds because npm/composer wont have to locale the packages again since it caches the locations of them in the lock file, there is no place where you should not do this, unless you are inherently a bad programmer and don't like to follow industry practice.

[Guillermo Garcia]

I agree with your arguments about why it should be a developer/maintainer responsibility to keep their dependencies at the latest versions and solve any issue related to some broken/buggy dependency version.

Perhaps npm needs to use a different approach to help us identify when an end-user runs ´npm install´ vs when a developer runs it.

To make this distinction viable we would propose two different kind of lock files. One for development usage and another for final usage (like packages that are themselves dependencies of other packages.).

[Dimitri KOPRIWA]

I am not using lock file on modules, I am now facing a hard issue accross many projects. Terser is breaking on production build of my documentation, having lock file accross the chain of dependencies would have prevented that.

[Daniel Saner]

Aren't you mixing up two things here; committing to a source code repository, and publishing to a registry?

npm pack strips out package-lock.json for publishing. But I believe that even if there were package-lock.json files in dependencies, npm install ignores any but the top-level one, anyway. It wouldn't make sense conceptually to consider them, because if you successfully lock down your dependency versions from your root, you implicitly lock them for all dependencies further down the tree, as well.

On the other hand, nothing speaks against publishing package-lock.json with the source code. In fact, that's half of the reasons for its existence. Because it only has an effect if it's the top-level package, it will help library developers with its intended purpose, while not affecting library consumers.

Please correct me if I'm misunderstanding something here!

[Liran Tal]

This is more of your own opinion Gajus than a best practice.
Mael has pointed out good reasons to use lockfiles.

2 articles I wrote to provide more context on lockfiles are:

1. snyk.io/blog/making-sense-of-packa...
2. if you use lockfiles, there's also a potential security issue that you should know about: snyk.io/blog/why-npm-lockfiles-can...

[Rick van der Meij]

It doesn't matter if you are developing in a team or solo. The package lock provides a clear vision which version of dependencies and dependencies of dependencies are being used,and sets them to a fixed version. You do not have any control of deeper dependencies without a package lock which can be reviewed. npm i updates dependencies without you noticing, that's why you use a npm ci when you want to rebuild your node modules.

When updating all packages you can create a specific feature and check those changes. Or automate it if you have decent test

[Neo]

Thanks i was not aware of this.

[Nahuel]

I understand that even if you have the package-lock it will have no effect on any npm install ran on you machine, docker or CI/CD. That's why it is always updated after an npm install.

It only makes a difference if you ran npm ci, right?

[Guido Bouman]

Almost, npm i always reproduces the same build from package-lock.json. Unless the dependencies are changed. Then it will update the package-lock.json to reflect those changes. It does not ignore the package-lock.json, as that would change your dependencies every time some nested dependency releases a new (patch?) version that satisfies its dependency requirement.

[Jonathan Cousins]

As far as I know, yes. (Although I'm aware you wanted the author to answer you).

npm ci is has been very useful for consistent development and ci-build environments, for me at least.

[Arsen A. Gutsal]

"but it was archived without an action."- probably they just act like you, closing PRs without any notice 😅

[Gajus Kuizinas]

I do provide explanation to every PR I close.

[Attila Fulop]

I'm maintaining PHP packages and we have the same debate about composer.lock in libraries.

I'm lucky to live in the same city as one of the two authors of composer (npm's counterpart in php world) so once I could have a personal discussion about the topic with him.

He also suggested to commit the lock file with the package (which I don't do either) but he also suggested to do this in the CI pipeline:

• test the package with the deps as in the lockfile
• downgrade direct deps to the lowest version allowed by .json file and test against that
• upgrade the direct deps to the highest version allowed by .json file and test against that as well

Here I can see the benefit of the lockfile, albeit I haven't started doing it yet

[kgrosvenor]

Dependency lock files are for fast tracking your dependencies via a file cache, so it doesn't have to look through npm again to find them again, you are meant to commit it yes and i don''t think there is a case for not commiting it?

It also works the same on composer.

Why ignore his advise about that from the author? your PI pipeline versions will eventually mess up because you don't commit it...