DEV Community

brian austin
brian austin

Posted on

The LiteLLM supply chain attack: what developers using AI proxies need to know

#claude #security #ai #programming

The LiteLLM supply chain attack: what developers using AI proxies need to know

Last week, Mercor disclosed they were hit by a cyberattack tied to a compromise of the open-source LiteLLM project. LiteLLM is one of the most popular open-source LLM proxy routers — used by thousands of teams to route requests between OpenAI, Anthropic, Cohere, and other providers.

If you're using LiteLLM (self-hosted or managed), here's what happened and what to do about it.

What happened

The attack was a supply chain compromise — malicious code inserted into a dependency, not LiteLLM's core codebase directly. Mercor's infrastructure ingested the compromised package, and attackers gained access.

This is the same attack vector as:

  • The xz-utils backdoor (2024)
  • The axios NPM compromise (2026)
  • The event-stream incident (2018)

Supply chain attacks are effective because developers trust packages they've been using for months. The exploit happens silently during an innocuous npm install or pip install --upgrade.

Why AI proxy stacks are especially vulnerable

AI proxies are high-value targets for a few reasons:

1. They hold API keys

Your LiteLLM instance typically has your ANTHROPIC_API_KEY, OPENAI_API_KEY, and potentially billing credentials stored in environment variables or config files. A compromised proxy = all your API keys exfiltrated.

2. They process sensitive prompts

Every user query flows through the proxy. A malicious package can log requests silently — including anything your users send.

3. Self-hosted = your responsibility

When you self-host LiteLLM, you own the security surface. Keeping up with dependency audits, CVEs, and supply chain integrity on top of building your product is a lot.

The attack surface of a typical self-hosted proxy 

[Your App]
    ↓
[LiteLLM proxy - self-hosted]
    ↓ (has your API keys)
[Anthropic / OpenAI API]
Enter fullscreen mode Exit fullscreen mode

The proxy sits between your app and the LLM provider with elevated trust on both sides. It's a natural pivot point for attackers.

What to do if you're running LiteLLM

Immediate steps:

# Check your installed version
pip show litellm

# Update to latest patched version
pip install --upgrade litellm

# Audit your dependencies
pip-audit  # or: safety check

# Rotate your API keys immediately
# Anthropic: console.anthropic.com → API Keys → Revoke + Regenerate
# OpenAI: platform.openai.com → API Keys → Revoke + Regenerate
Enter fullscreen mode Exit fullscreen mode

Audit your logs:

# Check for unusual outbound connections from your proxy host
ss -tupn | grep ESTABLISHED

# Look for unexpected DNS queries (if you have DNS logging)
cat /var/log/syslog | grep -E '(curl|wget|nc|bash)'

# Review API key usage in provider dashboards for anomalous spend
Enter fullscreen mode Exit fullscreen mode

Harden your setup:

# Lock down outbound traffic from your proxy
# Only allow connections to specific Anthropic/OpenAI IPs
iptables -A OUTPUT -d 23.102.140.112 -j ACCEPT  # anthropic
iptables -A OUTPUT -j DROP

# Use Docker secrets instead of env vars for API keys
docker secret create anthropic_key ./anthropic_key.txt
Enter fullscreen mode Exit fullscreen mode

The ANTHROPIC_BASE_URL alternative

If you're using LiteLLM primarily to:

  • Avoid Anthropic's rate limits
  • Get a stable API endpoint
  • Reduce per-call costs

There's a simpler option. Anthropic's Claude supports a ANTHROPIC_BASE_URL environment variable that redirects all requests through a custom endpoint:

export ANTHROPIC_BASE_URL=https://api.simplylouie.com
export ANTHROPIC_API_KEY=your-simplylouie-key

# Works with claude CLI, Python SDK, Claude Code — everything
claude "explain this code"
Enter fullscreen mode Exit fullscreen mode

This approach:

  • No self-hosted infrastructure to maintain
  • No dependency chain to audit
  • No supply chain attack surface on your side
  • $2/month flat rate, no per-token billing surprises

The broader lesson

Every dependency you add is a trust relationship. Open-source LLM proxy stacks are incredibly useful, but they come with a security cost that isn't always visible until something like Mercor's disclosure.

The LiteLLM team responds quickly to security issues. But you're still responsible for running pip install --upgrade before the compromise reaches you.

If your primary use case is Claude API access without rate limits, the lightest-weight option is often the most secure one — a single API endpoint swap via ANTHROPIC_BASE_URL, no infrastructure to own.

TL;DR:

  • Rotate your API keys now if you run LiteLLM
  • Run pip-audit on your proxy dependencies
  • Consider whether self-hosting a proxy is worth the attack surface
  • ANTHROPIC_BASE_URL redirects all Claude SDK traffic with zero proxy infrastructure

SimplyLouie is a flat-rate Claude API proxy — $2/month, no per-token billing, works as a drop-in ANTHROPIC_BASE_URL replacement.

Top comments (0)