DEV Community

Cover image for Security at Scale: Our npm Incident Response Story
João Pinho for epilot

Posted on

Security at Scale: Our npm Incident Response Story

On September 8th, the npm ecosystem saw what is now called the largest supply-chain compromise in its history. Packages like chalk, debug, and ansi-styles — together downloaded billions of times every week — were hijacked and malicious versions published.

For any SaaS company with Node.js in its stack, this was a moment to pause and act.

At epilot, where we build cloud software for the energy market, we recognised that this had the potential to impact our systems. Here's how we responded.

🚨 First, what happened?

  • A phishing attack compromised the maintainer of several widely used packages.
  • Malicious versions were published and quickly spread via transitive dependencies.
  • The injected code was designed to hijack crypto wallet transactions, but the bigger story is: if attackers could publish once, they could publish anything. Additionally, if attackers could read/intercept our network requests, they would have access to our tokens and thus our platform on behalf of our users.

⚡ Our response — measured in hours

  • As soon as the incident was announced, our engineering team moved fast. Within minutes of posting the alert in our #dev-security slack channel, we had a small response team come together.

  • 172 repositories scanned: we used GitHub Codespaces to access our centralized codebase and systematically audit our product ecosystem.

  • 451 Node.js projects analyzed: every package.json across our domain groups (auth, billing, analytics, 360-portal, and more) was checked for the compromised packages and versions.

  • Automation helped: we cross-checked with our internal tooling and monitoring alerts to confirm no production build had pulled malicious versions.

Result: ✅ zero exposure found across all 451 dependency trees.

🛡 Practices that kept us safe

The npm incident highlighted the value of practices we already had in place:

  • Dependency monitoring with Corge → ongoing visibility into package versions and vulnerabilities across our entire ecosystem.

  • Masked logs in Datadog → all PII automatically obfuscated, ensuring sensitive customer data never leaks, even if dependencies misbehave.

  • LLM anonymisation → using Microsoft Presidio, we automatically detect and anonymise emails, phone numbers, IBANs, credit cards, and other PII before any data reaches AI models for our AI-powered features.

These aren't "nice-to-haves." They are part of how we build trust with customers in a regulated industry like energy.

💡 What we learned

  1. Supply chain attacks are inevitable — being able to respond fast is what matters.

  2. Modern tooling makes a difference — GitHub Codespaces let us audit 172 repositories and 451 Node.js projects in hours, not days.

  3. Good hygiene compounds — monitoring dependencies, masking logs, and anonymizing LLM data aren't glamorous, but when incidents like this happen, they prove their worth.

🔭 Looking forward

Incidents like the npm compromise will continue. At epilot, we're committed not just to reacting fast, but to building resilient, privacy-first systems that earn customer trust every day.

Because in the energy market — where trust and compliance are everything — resilience is the real competitive advantage.

Top comments (0)