Navigating old Node.js dependencies can be confusing. In this article, I'll share how I decoded this challenge in a few-year-old repository using a custom tool—and how you too can benefit from my experience.
tl;dr
Use the tool dependency-time-machine to quickly update dependencies one by one in chronological order. Most dependencies are compatible with other packages from a similar or pastime.
This tool is intended to simulate the typical updating workflow as it was done regularly.npx dependency-time-machine --update
Outdated codebase
Recently I joined a new team with a great product but a highly outdated codebase. So my first task was to update dependencies and made changes so we can migrate from Node 16 to Node 18 and finally to Node 20. Some parts of the code were still relying on Node 14. The initial thought was to use ncu or yarn upgrade-interactive and update all patch versions since they shouldn't break anything if package developers were following the semver. But they didn't.
Dependency void
Many dependencies were not only "old" - this is not the problem, but they were incompatible with new Node versions and other dependencies which features we looked for. At the moment of doing this process, Node 16 becomes deprecated and Node 18 is going into maintenance. Having such an old codebase makes it hard to further develop and with backend part makes it also insecure and vulnerable.
Dependency management is not only about blindly updating, but also about knowing when it's beneficial to use or drop package and having common sense about long term tech debt.
Sometimes, newer versions of a dependency can introduce breaking changes, reduced performance, or other issues. Before updating, developers should check changelogs, understand the implications of the update, and, ideally, test the update in a controlled environment before deploying it.
Over time, some dependencies might become redundant, or better alternatives might emerge. A good practice is to periodically review the project's dependencies to determine if they are still aligned with the project's needs. Additionally, if a package is no longer maintained or has known vulnerabilities, it might be wise to seek alternatives or drop it altogether.
Over-reliance on multiple dependencies can lead to technical debt. If not managed properly, this debt can accumulate, making future changes or updates cumbersome and risky. It's essential to strike a balance between leveraging existing solutions (dependencies) and the potential long-term costs associated with them.
Going back to the case. Thankfully the team wrote plenty of good tests, both unit tests, some integration and a lot of end-to-end so after each update it was possible to check if something broke. At this point, I realized I have to go one-by-one dependency and look at which dependency will be compatible with others and SDK.
Some packages had phantom dependencies. They are indirect dependencies (but also not peerDependencies) and it is expected to have them installed and be available in certain directories. npm since v3 and yarn classic install dependencies in the flat structure of node_modules with nesting only sub-dependencies so the issue with phantom dependencies is hidden until changes in the package structure. pnpm - another package manager installs dependencies in a hidden directory and uses symlinks to make them available at node_modules levels. Also, it hides all sub-dependencies which brokes phantom dependencies early, leaving clean node_modules (IMO, that's good).
Other problems were changes in Node SDK from 16 to 18. In the meantime, Node 17 introduced changes in the open SSL provider which brokes older webpack and DNS resolve mechanism from "by-default-ipv4" to "by-default-ipv6" which brokes usage of some backend and testing libraries.
dependency-time-machine
To maintain all this headache I wrote in my spare time a library called dependency-time-machine which retrieves information about every package listed in package.json and its version. Then it creates a single timeline which is used to determine which library is the most behind and should be updated next.
I found libraries written at certain moments are compatible with other libraries at similar or previous timespan. So, for instance, library a@2.5.1 that came when library b@1.3.0 was already available makes it higher chance compatible than with the same library but with a higher version e.g. b@1.3.2 that was released a year later than a@2.5.1.
npx dependency-time-machine --timeline
Moreover, this replicates the natural process of updating dependencies. When you do this regularly you match existing libs with other existing libraries. So, the main problem that dependency-time-machine tries to solve is to reconstruct the process of natural updating.
And because it may be a lot of work, like in my case where I had to take care of over 200 packages (!) so to ease this process I put into tool auto mode which can find the next dependency to update before another dependency was released, install it and run tests. If tests pass, it will do the same over and over. If the tests fail it will stop so you can look closer and fix either codebase or test.
npx dependency-time-machine --update --install --auto --install-script "yarn install" --test-script "yarn test"
Also, if you don't want to rely on auto-mode you can print the timeline in JSON and make decisions manually of updating dependencies. Other options allow you to exclude dependencies you don't want to update for some reason.
Going back to the story. It still took some time but at least I was more certain about the process and had the linear plan of work. After all, I wrote the document to describe the process so other team members know what's going on with the repository and discussed the process. Deployment went smoothly with only few minor issues in the meantime because most of the problems were spotted early thanks to willing help from team members.
Dependency management can be painful, but done regularly and with patience makes our future work easier. I hope someone finds this article and tool useful :)
Oldest comments (13)
Great work
Impressive work! This will help incrementally update old dependencies that are years behind more easily.
Thanks. That was the goal. I found many times in articles online to suggest blindly updating all at once to the latest patch/minor/major version and #yolo. Then people were surprised they got a bunch of issues at once in the face.
If you're using yarn, you can do yarn --upgrade-intreractive and you'll be able to choose which package to update and if you want a minor,medium or major update for each package. It's a really good native way of solving the update of old dependencies. That's the only reason I prefer yarn over npm!
I agree,
yarn --upgrade-interactiveis a great way to update deps. However, this must be done regularly or in a fairly small repository to quickly catch and fix issues.When a project is huge and outdated by a few years, it makes this really difficult to spot which libs are compatible with which others and some e.g. babel, babel loaders, core-js, webpack, jest plugins and others are making this process even more complicated as they are often base for other dependencies, some left a long time ago.
The tiny tool I wrote about is to fill the missing gap in the dev ecosystem with a huge, highly outdated project as mentioned.
I love it when things are named well: 'dependancy-time-machine' is perfect!
impressive things
Very impressive! I wish I had this a few months ago upgrading from 12 to 16. Now I have to manually update our ORM before I can use this to go from 16 to 20...
Wait a year, and you do the same work again. Is'nt this frustrating? Any hint to ease this pain for the future?
Awesome articles! I'm having a similar situation so this help me quite a bit.
Will definitely try it.
Found this article because I needed to update my React website, which I built a few years ago lol. Shout out to the library you wrote - it's gonna help a lot of people!
When I attempt to run it with the install flag, it fails saying that the list of packages is not being passed into yarn.
I would expect the yarn add to need to be "yarn add [package name]", but it is only running yarn add without the package.
`PM> npx dependency-time-machine --update --install-script "yarn add" --install
New version found: async@0.9.0 (2014-05-16T10:20:22.247Z)
Updating async@0.9.0 in [PROJECT PATH]\package.json...
Installing new version...
npx : error Missing list of packages to add to your project.
At line:1 char:1
node:internal/errors:857
const err = new Error(message);
^
Error: Command failed: cd [PROJECT PATH] && yarn add && cd -
error Missing list of packages to add to your project.`