DEV Community

Jonna Fassbender
Jonna Fassbender

Posted on

Your package was compromised. How do you prove which version you actually shipped?

#security #python #supplychain #devops

Last week, LiteLLM got owned.

Not the company. Not the code. The publishing pipeline. An attacker compromised a vulnerability scanner in their CI/CD, used it to grab PyPI credentials, and pushed a malicious version that stole API keys from every engineer who installed it.

Three days later, the same thing happened to Telnyx.

If you work with AI, you probably had LiteLLM installed. The question isn't whether you were affected. It's whether you can prove what version you were running, and when.

The real problem isn't the hack

Supply chain attacks aren't new. SolarWinds. Codecov. ua-parser-js. The pattern is always the same: something between the developer and the user gets compromised.

What's new is the response problem.

When LiteLLM was hit, every team using it had to answer: "Were we running the compromised version?" Most couldn't answer with certainty because their evidence came from the same systems that were potentially compromised.

Your CI logs live in your CI. Your registry timestamps come from the registry. Your internal records are... internal.

The system under dispute is also the source of the evidence. That's not proof.

What anchoring does

Anchoring creates a cryptographic fingerprint of your artifact (a package, a model, a build) and timestamps it in Bitcoin's blockchain. Not a copy. Not a backup. A mathematical proof that this exact artifact existed at this exact moment.

artifact.whl (your package)
    ↓ SHA-256
a]7c3f9e...  (fingerprint)
    ↓ anchor
origin_id: ec8eead9-...  (your proof)
    ↓ Bitcoin
anchored to Bitcoin block 889,412
Enter fullscreen mode Exit fullscreen mode

The proof is independently verifiable. It doesn't depend on Umarise, your CI provider, or any single system staying honest.

What this looks like in practice

In your build pipeline (one line):

umarise anchor dist/package-1.2.0.whl
Enter fullscreen mode Exit fullscreen mode

In GitHub Actions:

- name: Anchor build artifact
  uses: AnchoringTrust/anchor-action@v1
  with:
    file: dist/package-1.2.0.whl
  env:
    UMARISE_API_KEY: {% raw %}${{ secrets.UMARISE_API_KEY }}{% endraw %}
Enter fullscreen mode Exit fullscreen mode

In Python:

import hashlib
from umarise import UmariseCore

with open("dist/package-1.2.0.whl", "rb") as f:
    sha256 = "sha256:" + hashlib.sha256(f.read()).hexdigest()

client = UmariseCore(api_key="um_...")
result = client.attest(hash_value=sha256)
print(result.origin_id)
Enter fullscreen mode Exit fullscreen mode

That's it. One call. The proof is permanent.

The timeline that matters

Here's what the LiteLLM attack looked like, and what changes with anchoring:

Day 0: Team publishes litellm 1.52.6 (legitimate)
       → Without anchoring: version exists on PyPI
       → With anchoring: SHA-256 fingerprint anchored to Bitcoin

Day 3: Attacker compromises CI vulnerability scanner
       → Attacker extracts PyPI credentials
       → Pushes malicious litellm 1.52.7

Day 4: Malicious version steals API keys from .env files
       → 1000+ engineers affected

Day 6: Attack discovered, version yanked

Day 7: Incident response begins
       → Without anchoring: "Our logs say we had 1.52.6"
       → With anchoring: "Here's the Bitcoin-anchored proof
         of exactly what we were running"
Enter fullscreen mode Exit fullscreen mode

The difference: one is a claim. The other is a proof that anyone can verify, independently, forever.

Why this matters for your team

You don't need to have been hit by LiteLLM to care about this. Ask yourself:

  • Can you prove which version of a dependency you deployed last Tuesday?
  • If your CI provider gets compromised, does your evidence survive?
  • If a regulator asks for proof of what was in production, can you provide something that doesn't come from your own systems?

Most teams answer no to all three.

The pattern is bigger than packages

The LiteLLM attack targeted a Python package. But the same vulnerability exists everywhere artifacts move through a pipeline:

  • ML models: Can you prove which weights were in production when a decision was made?
  • Training data: Can you prove the dataset wasn't modified after the audit?
  • Docker images: Can you prove the image that ran in prod matches what was built?
  • Firmware: Can you prove the binary that shipped is the binary that was signed?

Every artifact that moves through a pipeline without independent proof is vulnerable to the same class of attack.

Getting started 

pip install umarise
umarise anchor <your-artifact>
Enter fullscreen mode Exit fullscreen mode

The LiteLLM attack wasn't exotic. It exploited a dependency in a CI pipeline. This will happen again.

The real question is whether you'll be able to prove what you shipped, what you installed, and when.

Right now, for most teams, the answer is no.

Adding temporal proof to your publish pipeline takes one line.

Before anchoring: "Our logs say so."
After anchoring: "Verify it yourself."

Links:

Top comments (0)