DEV Community

Cover image for Has anyone else run into this problem while building an AI SaaS?
aliezat
aliezat

Posted on

Has anyone else run into this problem while building an AI SaaS?

You launch your product, a few customers start using it, and then you realize...

How do you know which tenant is spending your OpenAI/Anthropic/Gemini budget?

And even if you know, how do you stop them before they burn through hundreds of dollars?

I looked around, and most tools focus on observability—they tell you what happened after the API call. I wanted something that could also enforce spending limits before the next request goes out.

So over the past few weeks, I built token-limit, a Python SDK that:

✅ Monkey-patches the official LLM SDKs (OpenAI, Anthropic, Gemini, DeepSeek, OpenRouter)

✅ Automatically tracks every LLM request per tenant

✅ Batches usage events in the background (no changes to your LLM call sites)

✅ Lets you set per-tenant spending limits and raises a LimitExceededException before another paid request is sent

The setup is intentionally simple:

from token_limit import Meter, MeterConfig

meter = Meter(MeterConfig(api_key="..."))
meter.patch_all()

with meter.for_tenant("acme"):
    client.chat.completions.create(...)
Enter fullscreen mode Exit fullscreen mode

I'm still actively improving it, and I'd genuinely love feedback from people building AI products.

A few questions:

  • How are you tracking LLM usage today?
  • Do you enforce budgets per customer, or just monitor costs?
  • Is monkey-patching something you'd be comfortable using in production, or would you prefer another approach?
  • Which provider or framework would you want to see supported next?

If you'd like to try it or review the implementation, here's the repo:

GitHub: https://github.com/AliEzatyar/token-limit

I'd really appreciate any feedback—positive or critical. If there's something that makes you think, "I wouldn't use this because...", I'd especially like to hear it.

Top comments (0)