by author Fernando Doglio
This is not the first time something like this happens, but I wouldn't blame NPM for this. I wouldn't even blame the Node.js community or the creator of such a popular library.
This is the hacker's fault, or rather, 80% of it is, the rest? The rest is the affected project's fault.
Let's understand that for a bit.
What happened?
On October 22nd the owner of the UA-Parser.js package realized his NPM account had been hijacked and that someone else (the attacker) had pushed 3 different new versions (0.7.29, 0.8.0 and 1.0.0) of his package including, within it, malicious code for both Linux and Windows.
The "evil" code was later found to be stealing passwords and Chrome cookies while at the same time, it was running a crypto-miner program.
Why this library? Because it's a very popular one, in fact, so much so that it would seem that even Facebook used it on some of their client-facing code. And don't take me word for it, NPM shows almost 8 million downloads per week.
So yes, I'd say it's used by a few users.
And while the effect of the attack is important and it probably did a lot of damage while it was active, that's not my focus today.
I don't want to focus on the 80% of the problem, but rather on the 20% left which includes you (potentially).
Why are these attacks so effective?
This is not the first time something like this happens. In fact in 2018 something similar happened. A hacker got ownership of a very popular repository through social engineering and proceeded to publish updates with a similar effect as the one we're discussing today.
So why are we seeing these types of attacks happen more often than before?
I believe this is because hackers are starting to realize that developers are lazy when it comes to looking out for the security of their code. Coders are so used to depending on 3rd party libraries that they can't really work without them to the point where some of these libraries even define their career.
Don't know what I'm talking about?
Does "React Developer" ring a bell?
If tomorrow for some reason the React repository got hacked and someone was able to upload malicious code into it, we'd be royally f?cked, trust me on that one.
That being said, this intrinsic trust that we're seeing on these 3rd party libraries coupled with poor dependency management is showing a brand new world for attackers to target. We're literally showing highlighting their targets by relying on them blindly, so how can we fix this?
Do we just stop using 3rd party libraries?
I've had conversations with many developers who claim they don't rely on any 3rd party library. I tend to not believe such a claim because that would make building any piece of software a gargantuan task at this point.
Either that, or they're working with very unstable and untested libraries created by themselves. I don't like either case.
Consider building a simple client-server application, everything that goes into it, aside from the actual business logic of your app:
On the back-end
- The API framework able to simplify the endpoint-definition
- HTTP framework that simplifies header and request management
- The authentication framework that allows you to provide multiple auth strategies out of the box
- The database driver (I'm assuming you're storing data somewhere).
- The bundler
- The testing framework
- Potentially some utility library
On the front-end
- The actual UI framework, because nowadays nobody really builds a complex UI without one (I'm talking about React, Vue, Angular, Svelte, etc).
- The bundler
- The testing framework
- Probably plugins for your UI framework
- HTTP request libraries to simplify sending data to the back-end.
And I'm sure I'm missing a few on each category. So considering that list, are you really suggesting that we should dedicate our own time and effort into building ALL of that, on top of building the actual logic of our application?
The whole point of using them is because they act as accelerators for the development team. They help reduce development time by providing features we'll need.
In return through this massive use by the community, we know that they've been properly tested and vetted. If everyone is using them, chances are, they're good at what they do so we're safe using them too.
How do we solve the security issue then?
The entire problem is not about relying on 3rd party libraries. Focusing on them when trying to solve the problem is exactly what the hackers want, because you miss the whole point and keep the doors open: the real problem is how you're managing your dependencies.
Think about it like this, go to your latest project's package.json file and open it. Check out the dependencies section, do you see something like this:
"dependencies": {
"my-dep": "*",
"my-other-dep": "1.x",
"my-other-dep2": "^1.23.4",
"my-other-dep3": "~4.2.1"
}
They're all security risks.
Look I get it, these libraries are all using semver and you're only allowing for minor or patch-level updates on those dependencies, if they update anything your code won't break.
That's true, in theory.
But let's unpack that for a second. The IT world is filled with standards for everything, we developers love them, but we also love to make our own interpretations of them. This is ironic if you think about it, since standards are meant to be concise and unambiguous so that everyone understands the same thing (tell that to REST btw).
So your first mistake is to assume that the creators of all those dependencies are understanding exactly the same thing you are from the semver standard. Granted, if they state exactly how they update their library on their documentation, you're safe, otherwise, you're not.
If tomorrow they decided to add a breaking change into their code, and either forgot and for some reason, decided that a bump in the major version wasn't required; you're in big trouble.
The second mistake, and the one I want to highlight here is that you're opening the doors of heaven to the hackers. You're literally telling them: "go ahead, put whatever you want and I'll install it blindly".
See the problem yet? Auto-updates should not be an option for production releases.
They add too much instability into the build process and open up a security hole the size of Kansas. Which is big considering we're talking about a whole for attacks to go through.
So what can you do to prevent this from happening? Easy, lock your dependencies.
Instead of leaving fuzzy rules like the ones above on your production build, lock the specific version, all 3 parts of it:
"dependencies": {
"my-dep": "1.0.0",
"my-other-dep": "1.2.3",
"my-other-dep2": "1.23.4",
"my-other-dep3": "4.2.1"
}
It's that simple, you've now solved the security problem because on every single build, you'll be downloading the exact same code. This is also true, because NPM does not allow you to overwrite past versions, so even if the repository of one of your libraries got hacked, you wouldn't be affected by the attack. Imagine that.
Open Source Session Replay
Debugging a web application in production may be challenging and time-consuming. OpenReplay is an Open-source alternative to FullStory, LogRocket and Hotjar. It allows you to monitor and replay everything your users do and shows how your app behaves for every issue.
It’s like having your browser’s inspector open while looking over your user’s shoulder.
OpenReplay is the only open-source alternative currently available.
Happy debugging, for modern frontend teams - Start monitoring your web app for free.
A simple solution for a very complex problem
The very reason why hackers are targetting 3rd party libraries and packages like this, is because they've realized we're a mess when it comes to dependency management. We trust them too much without seeing the security implications of that trust.
But the solution is not to forget about them, but rather to properly vet them first, and then lock our production builds to a set of versions that we know work.
And yes, you'll be missing out on the goodness of those auto-updates and the fixes their creators might be publishing with those minor and patch bumps in the version.
But consider the alternative: your project and your servers being used for crypto-mining or something worse.
Top comments (11)
But this is not our fault.😭
NPM is like wanting a banana and getting a jungle. It was stated in this post that vetting our dependencies is the only way.
If one brings 20 and 20 of those have 200 and those have 2000 dependencies well I'm not sure how many years I would need to check several files with potential for n hundred lines.
But then the only solution is automated checking as npm does (quite poorly depending who you ask), that is where we are right now. Automated checks are unfortunately retroactive, there has to be some victims the ensure the safety of the rest of us. Proactive human checking will always win as long as it's not a large job, I just don't think the pace of sprints tie nicely with this timeline.
It kinda is I'm afraid, yeah npm should have some responsability, but in the end it's up to us to understand how it works, and take messures to securize our applications as much as posible. But we can't do much apart from the solution presented in this post. Although that has it's drawbacks too, minor/patch version sometimes contain security fixes that would not be installed until we do it manually. We would need to be quite responsible and update our dependencies manually each couple of days to get access to those fixes. But we also would need to check what the updates are and check for insecure code... which in my opinion it's not posible (or at least not easy) for smaller teams.
So yeah, I guess we're screwed, damn...
Do your best I suppose it's all we can do. I know in truth it is down to us all to be sucure - thank you for the post Keff :)
Isn’t it what a
package-lock.jsonandnpm cido ?From the node.js documentation
Can I blame society or capitalism?
What if behind every npm package there is a team of well payed developers working really hard to mantain the code? That would be nice.
Also, everyone should add
ignore-scripts=trueto their.npmrc. It will stopnpmfrom running pre-install or post-install scripts.Hmm, true, but in the same time there is growing number of known vulnerabilities in packages which developers already use. That's why there are services like dependabot, which are constantly updating project deps. So what's the better option? To update or not to update? When you are updating, you are opening your project to those attacks, but when you are not updating, hackers may just find known vulnerabilities in your dependencies and exploit them.
It's not about locking your deps and then forgetting about them, instead it should be about locking them for every prod deploy, meaning that if you want to update them, you have to go through the same testing/uat/prod cycle, to ensure that whatever gets into prod actually works.
Because after all, it's true, you do want those other updates, but can you blindly trust them?
I just don't see how humans can check all fo this, its not possible in the timescale of a human lifetime. Id rather pay sombody to write an AI to do it for a fraction of the cost
Start to tame Deno
Okay. Society, capitalism and communism. Is that better?
Yarn