DEV Community: John Medina

Why AI Agencies are flying blind (and how to fix your LLM margins)

John Medina — Wed, 22 Apr 2026 14:30:49 +0000

If you're running an AI agency, you're probably building some
variation of RAG or agentic workflows for your clients.

You deliver the project, it works great, and then the first OpenAI bill hits.

Most agencies I talk to are still in the "winging it" phase when it
comes to API costs. They use one master key for dev, one for prod, and
maybe—if they're feeling fancy—one key per client.

But fwiw, per-client keys are a maintenance nightmare. And if you're
using a single master key for multiple clients, you're flying blind.

The "Averages" Trap

You might think: "I'll just charge a flat $100/mo for API usage."

Then one client decides to run a bulk ingest of 5,000 PDFs. Your
margin on that client just went negative, and you won't even know it
until the end of the month when you see a spike in the dashboard that
you can't explain.

Averages don't work for LLMs. Usage is too spiky.

3 ways to handle client attribution

1. The Metadata Header (The "Minimum Viable" way)

OpenAI, Anthropic, and OpenRouter all allow you to pass a user or
metadata header.

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [...],
  user: `client_acme_corp`
});

This is the bare minimum. It lets you export a CSV at the end of the
month and spend 4 hours in Excel trying to pivot-table your way to a
client invoice. tbh, it's better than nothing, but it's not real-time.

2. The Custom Proxy

You build a middleman. Every request from your client's app goes to
your proxy first, you log the tokens, then you forward it to OpenAI.
Pros: Absolute control.
Cons: You just added a single point of failure and 200ms of latency to
every request. Unless you're a DevOps wizard, this is usually
over-engineering for a boutique agency.

3. Real-time Attribution (The "Sanity" way)

You keep your direct provider connection but fire an async event for
every request.

This is why I built LLMeter. I needed a way to see exactly which
client was spending what, in real-time, without adding latency to the
main request.

We use it at Simplifai for our own tools. It tracks:

Cost per client (tenant)
Cost per model
Daily burn rates

If a client starts hitting the API harder than expected, I get an
alert immediately. I can then decide to upsell them, cap their usage,
or adjust the billing.

Why this matters for your agency

Clients don't like "surprise" bills. If you can show them a dashboard
(or a report) with their exact usage and cost, the trust level goes up
10x. It moves you from "freelancer with a script" to "professional AI
partner."

LLMeter is open-source (AGPL-3.0) and you can self-host it for free.
Check it out: llmeter.org

How are you billing your clients for LLM usage right now? Flat fee?
Pass-through? Or just eating the cost?

Why LLM Cost Dashboards Are Not Enough — The Runtime Enforcement Gap

John Medina — Thu, 16 Apr 2026 16:20:56 +0000

I've been looking at how teams handle LLM API costs in production, and there's a weird gap in the tooling right now. Everyone is building observability — logs, traces, dashboards. But almost no one is actually enforcing budgets at runtime.

If you are running multi-step agents or letting users chat indefinitely, discovering a $4,000 OpenAI bill at the end of the month via a dashboard doesn't help. The money is already gone.

The problem breaks down into three layers:

Attribution (knowing which user/tenant caused the cost)
Alerting (getting warned when a threshold is near)
Enforcement (blocking requests at runtime)

Most teams are stuck at layer 1. You can't enforce a per-customer budget if you don't even know what each customer is costing you.

I built LLMeter because I needed to solve that first layer. It's an open-source dashboard that tracks OpenAI, Anthropic, DeepSeek, and OpenRouter costs per user and per day. It also handles budget alerts.

Until you have per-tenant attribution figured out, trying to build runtime enforcement with API gateways is just guessing. Get the data first, then block the requests.

LLM prices dropped 80% — but are you actually saving money?

John Medina — Thu, 16 Apr 2026 16:17:17 +0000

veryone is cheering about Anthropic and OpenAI dropping API prices by 80%.
It sounds great on Twitter. But if you look at your actual billing dashboard, your costs probably haven't moved that much.

Why? Because cheaper tokens usually just mean you start wasting more tokens.

Here is the thing:

1- Context bloat
When GPT-4 was expensive, we carefully truncated histories and compressed prompts. Now that it's cheap, devs just throw the entire 128k context window at it on every single retry. The cost per token dropped, but you are sending 10x more tokens per request.

2- Agent loops
Cheaper models make agentic workflows viable, but a poorly configured while loop can still burn through your budget in minutes. When an agent gets stuck and retries 40 times, cheaper tokens don't save you—you still bleed cash.

3- Lack of per-customer attribution
It's easy to see your total OpenAI bill. But if you don't know which specific tenant or user is driving the cost, you can't optimize it. You just eat the cost.

tbh, the raw price per token is only half the story. If you can't attribute the cost per-user or per-model, you're still flying blind.

fwiw I built LLMeter to fix this for my own projects. It tracks costs per model and per user, and sets budget alerts—without a proxy in the middle. It's open-source (AGPL).

Check it out if you're tired of guessing your AI bills: https://llmeter.org?utm_source=devto&utm_medium=article&utm_campaign=2026-04-16-llm-prices-dropped-are-you-saving

AI Observability Bill Shock — 200% Cost Increase

John Medina — Wed, 15 Apr 2026 16:09:31 +0000

Got hit with a 200% increase on my OpenAI bill last month. Thought it was a rogue script or a bug in my code. It wasn't. It was just silent failures and inefficient prompt testing by the team that no one was tracking.

We were basically flying blind. A lot of startups are doing this right now with LLM APIs. You hook up OpenAI or Anthropic, throw some prompts at it, and look at the total bill at the end of the month. When it's $50, you don't care. When it spikes to $800 and you have zero attribution, it hurts.

I tried looking into the dashboard. It gives you an aggregate. That doesn't tell me if it was the customer service bot eating tokens or the new RAG feature. No idea who the "tenant" was driving the cost.

This isn't a rare thing either. Talked to a few other devs and everyone just accepts the billing page as truth and moves on. Tbh, it's a terrible way to run production.

If you are using LLMs, you need to track cost per model, per user, per day. And you need budget alerts before the end of the month.

I got tired of parsing raw logs and built LLMeter to solve this for myself. It tracks costs per tenant and sends budget alerts before you get hit with a surprise bill. It's open-source (AGPL-3.0) so you can self-host it or use the cloud version.

Stack is Next.js, Supabase, Inngest, and Vercel. Supports OpenAI, Anthropic, DeepSeek, and OpenRouter.

You can check it out here: LLMeter

Fwiw, even if you don't use LLMeter, start logging your token usage by user ID. You will thank yourself later when the usage scales.

Token Prices Are Dropping, So Why Is My AI Bill Going Up?

John Medina — Thu, 09 Apr 2026 14:31:57 +0000

Everyone's cheering the latest token price drops from OpenAI and Anthropic. Great. But my cloud bill doesn't seem to care. It's still climbing.

What gives?

It's the "agentic" workflow trap. We've moved past simple text-in, text-out chatbots. Now we're building agents that think, loop, and run multiple steps to complete a task.

A simple chatbot call might use 2k tokens. An agent figuring out a multi-step problem? I've seen them burn through 50k-100k tokens for a single task. The reasoning loops, error correction, and tool usage stack up fast.

Gartner just put out a warning about this. They're saying agents can use 5x to 30x more tokens than a standard chatbot call. So while the per-token price is 80% lower, our usage is quietly exploding by 500% or more. The math isn't in our favor.

The second part of the problem is per-customer attribution. If you have a multi-tenant SaaS, how do you know which customer's agent just went rogue and spent $50? Most basic monitoring just shows a single, terrifying number going up. You can't bill it back, you can't warn the user, you can't do anything but pay it.

This is the stuff that kills margins in AI products.

fwiw, I've been dealing with this by building better monitoring. I built LLMeter to get per-user cost attribution. It's open-source (AGPL). It hooks into OpenAI, Anthropic, etc. and lets me see exactly which user ID is responsible for which costs.

At least now when the bill spikes, I know who to blame.

Agent loops are eating your API budget

John Medina — Wed, 08 Apr 2026 17:08:37 +0000

Everyone's shipping agents right now. ReAct, tool-calling loops, whatever.

Looks great in demos. But nobody mentions the billing dashboard the morning after.

Agent loops are entirely unpredictable. A simple task might take 2 LLM calls. Or the model gets confused, tries a failing tool 20 times, and burns 40 calls before timing out.

Local test: $0.05.
Prod: user triggers a loop, agent hallucinates, costs $4 for a single request. Multiply by 100 users.

Devs treat LLM calls like standard API calls. They aren't. They're variable-cost compute disguised as a REST endpoint.

If you run agents in prod, you need defensive monitoring:

Hard iteration caps. Never let an agent run "until complete". max_iterations=5. Return an error instead of a massive bill.
Per-tenant attribution. Global tracking is useless. When your Anthropic usage spikes 300%, you need to know exactly which userId caused it so you can rate-limit them.
Budget alerts. Set up webhooks that fire the second a user crosses their daily quota.

tbh I got tired of building this from scratch for every project, so I built LLMeter.

It's an open-source dashboard (AGPL-3.0) for multi-tenant LLM cost tracking. Works with OpenAI, Anthropic, DeepSeek, and OpenRouter. You pass the user ID, it tracks the cost per user, per day, per model.

Code is on GitHub (https://llmeter.org). fwiw, running agents without per-user monitoring is just asking for a denial-of-wallet attack. ymmv.

Per-customer cost attribution without a proxy

John Medina — Wed, 08 Apr 2026 13:44:49 +0000

Most AI cost tracking solutions force you to route all your LLM traffic through their proxy. Tbh, that's an architectural nightmare waiting to happen. You're adding latency, introducing a single point of failure, and giving some third-party service the keys to your entire prompt stream.

If their proxy goes down, your app goes down. If their proxy gets slow, your users think your app is slow. And let's not even talk about the compliance headache of sending sensitive customer data through an intermediary just to track API costs.

You don't need a proxy to figure out which customer is burning your OpenAI budget. You just need proper attribution at the request level, handled asynchronously.

The Problem with LLM Billing

When you look at your billing dashboard on OpenAI or Anthropic, you just see total tokens used and a massive dollar amount at the end of the month. You don't see that user_123 ran a massive batch extraction job that cost you $40 in API calls, while your other 100 users cost $2 combined.

Multi-tenant SaaS apps need unit economics. If you charge a flat $20/mo subscription but a power user is burning $50/mo in Claude 3.5 API costs, you are actively losing money. But to fix it, you need to know exactly who is spending what.

Why Proxies Are a Bad Idea for This

A lot of dev tools in the AI space right now tell you to just swap your base URL from api.openai.com to proxy.theirservice.com.

Here is what happens when you do that:

Every request adds 50-200ms of network overhead.
If the proxy goes down, your production app fails to serve requests.
You are sending raw PII and proprietary data to a vendor just to count tokens.

It's massive overkill. Cost tracking should be out-of-band. It should never be in the critical path of your application's request/response cycle.

The Async Logging Approach

The correct way to handle this is logging costs asynchronously after the request completes. Your app talks directly to the provider (OpenAI, DeepSeek, OpenRouter, Anthropic), gets the token usage from the response, and fires a background job to log it against the customer ID in your own database.

Here is the flow:

User triggers an action.
Your backend calls the LLM provider directly using their official SDK.
Provider responds with the completion and usage stats (prompt_tokens, completion_tokens).
Your backend returns the response to the user immediately.
Your backend fires a non-blocking async event (e.g., using Inngest, BullMQ, or standard background workers) with the user ID, model used, and token count.

This gives you zero added latency. Zero third-party risk. Your app stays fast and reliable even if your cost-tracking database goes down.

Implementing the Calculation

Calculating the cost is straightforward but tedious. You need to maintain a pricing table for every model you support.

For example, if the payload from OpenAI says:
{ "prompt_tokens": 1500, "completion_tokens": 400 }

The background worker calculates the cost based on current model pricing and writes it to your database.

This gives you zero added latency. Zero third-party risk. Your app stays fast and reliable even if your cost-tracking database goes down.

I Got Tired of Surprise OpenAI Bills, So I Built a Dashboard to Track Them

John Medina — Sat, 04 Apr 2026 02:09:49 +0000

A few months ago, I got a bill from OpenAI that was about 3x what I was expecting. No idea why. Was it the new summarization feature we shipped? A single power user going nuts? A cron job gone wild? I had no clue. The default OpenAI dashboard just gives you a total, which is not super helpful for finding the source of a spike.

This was the final straw. I was tired of flying blind.

The Problem: Totals Don't Tell the Whole Story

When you're running a SaaS that relies on multiple LLM providers, just knowing your total spend is useless. You need to know:

Which provider is costing the most?
Is gpt-4o suddenly more expensive than claude-3-sonnet for the same task?
Which feature or user is responsible for that sudden spike?

I looked for a tool that could give me this visibility without forcing me to proxy all my API calls through their servers. I didn't want to introduce another point of failure or add latency. I just wanted to see my costs.

Nothing quite fit what I needed, so I did what any self-respecting developer does: I started building my own thing.

The Build: A Glorified Cron Job

I started simple. The core of the idea was a background job that would run every hour, hit the usage APIs for my providers (OpenAI, Anthropic, etc.), and store the normalized data in a Postgres database.

The stack was pretty straightforward:

Backend: Inngest for the hourly polling jobs. It's reliable and has great logging.
Database: Supabase for the Postgres DB and auth.
Frontend: Next.js and Shadcn UI to build the dashboard.

But a simple stack doesn't mean a simple build. I hit a few interesting technical challenges.

First, normalizing the data was a bigger pain than I expected. OpenAI's API returns usage in tokens. Anthropic's returns it in characters for some models and tokens for others. Their JSON structures are completely different. I had to write a flexible adapter layer to ingest these varied formats into a single, unified Postgres schema. The goal was to have one usage_data table where I could query cost in USD across all providers without complex joins or transformations on the fly.

Second, handling API rate limits and errors required some care. When you're polling multiple services, one of them is bound to fail or rate-limit you eventually. I built a simple exponential backoff mechanism into the Inngest function. If a provider's API call fails, it retries up to three times with increasing delays. This makes the data pipeline resilient to transient network issues and API hiccups without sending me a million error alerts.

One of the most important parts was security. I couldn't store the provider API keys in plain text. So, I made sure they were encrypted at rest with AES-256-GCM before being saved to the database.

The Result: LLMeter

After a few weekends of work, I had a working dashboard. I called it LLMeter.

It does exactly what I needed:

It connects directly to the provider APIs. No code changes, no proxies.
It pulls usage data every hour and shows me the actual costs, not just estimates.
It breaks down the costs by provider and model, so I can see exactly where my money is going.
I can set budget alerts to get an email if my daily or monthly spend goes over a certain threshold.

The impact was immediate. After running LLMeter for a month, I discovered that nearly 70% of my costs were coming from a single background job that was mistakenly using gpt-4o for a simple classification task. Claude 3 Haiku could do the job just as well for a fraction of the price. A five-minute fix ended up saving me an estimated $200/month. That's the power of real visibility.

It now supports OpenAI, Anthropic, DeepSeek, and even OpenRouter, which is great for tracking costs across hundreds of smaller models.

Lessons Learned Along the Way

This small project reinforced a few key lessons for me:

Observability is Non-Negotiable. You can't optimize what you can't see. My initial guess about the summarization feature being the cost culprit was completely wrong. Without granular data, I would have wasted time "optimizing" the wrong part of my application.
Make Onboarding Frictionless. The decision to avoid a proxy and use direct API integrations was critical. It meant I could adopt LLMeter for my own projects in minutes, without changing a single line of application code. For a developer tool, ease of integration is a core feature, not a nice-to-have.
Security From Day One. When you're dealing with API keys that can literally burn money, you can't afford to be sloppy. Encrypting keys at rest wasn't a "v2 feature," it was a requirement for the first line of code.

Open Source, Of Course

I figured other indie hackers and small teams probably have the same problem, so I made the whole thing open source (AGPL-3.0). You can self-host it on your own infrastructure if you want to.

If you're tired of surprise LLM bills, you can check it out here:

Website: llmeter.org (there's a generous free tier)
GitHub: github.com/amedinat/LLMeter

Let me know what you think. And if you have any other horror stories about your LLM bills, I'd love to hear them in the comments.

The hidden cost of GPT-4o: what every SaaS founder should know about per-user LLM spend it

John Medina — Wed, 01 Apr 2026 23:53:51 +0000

So you're running a SaaS that leans on an LLM. You check your OpenAI bill at the end of the month, it's a few hundred bucks, you shrug and move on. As long as it's not five figures, who cares, right?

Wrong. That total is hiding a nasty secret: you're probably losing money on some of your users.

I'm not talking about the obvious free-tier leeches. I'm talking about paying customers who are costing you more in API calls than they're giving you in subscription fees. You're literally paying for them to use your product.

The problem with averages

Let's do some quick, dirty math. GPT-4o pricing settled at around $3/1M tokens for input and $10/1M for output. It's cheap, but it's not free.

Say you have a summarization feature. A user pastes in 50,000 tokens of text (around 37.5k words) and gets a 1,000 token summary back.

• Input cost: 50,000 / 1,000,000 * $3.00 = $0.15
• Output cost: 1,000 / 1,000,000 * $10.00 = $0.01
• Total cost for one summary: $0.16

If a user on a $19/mo plan does this just four times a day, every day, their usage looks like this:

• Daily cost: $0.16 * 4 = $0.64
• Monthly cost: $0.64 * 30 = $19.20

You just lost twenty cents on that customer. And that's one feature. What if your app is a chatbot? What if they're running complex agentic workflows? It's easy to see how a single "power user" can quietly burn through their subscription fee and start eating into your margins.

Your monthly bill averages this out. You see the total, you see your total MRR, and if one is bigger than the other, you think you're fine. But you're flying blind. You have no idea which customers are profitable and which are financial dead weight.

You can't fix what you can't see

The real issue is attribution. The OpenAI invoice is just a number. It doesn't tell you that customer-123 on the Pro plan cost you $45 last month while customer-456 cost you $1.50. Without that breakdown, you can't make smart decisions.

• You can't identify users who need to be moved to a higher tier.
• You can't set fair rate limits.
• You can't detect abuse.
• You can't accurately price your service.

You're just guessing.

To give you a clearer picture, let's look at how the main providers stack up. Prices are always in flux, but as of early 2026, here's the landscape for the flagship models per million tokens:

Model	Input Cost / 1M tokens	Output Cost / 1M tokens
OpenAI GPT-4o	~$3.00	~$10.00
Anthropic Claude 3.5 Sonnet	~$3.00	~$15.00
Google Gemini 1.5 Pro	~$3.50	~$10.50

As you can see, output costs for a model like Claude 3.5 Sonnet are 50% higher than for GPT-4o. If your application is write-heavy (generating long reports, articles, etc.), that difference will show up on your bill. Without per-user tracking, you'd have no idea if a profitable GPT-4o user would become a loss-leader on a different model.

4 ways to stop the bleeding

Okay, so tracking is the first step. But once you can see the problem, how do you fix it? Here are a few practical strategies. This isn't rocket science, but it's amazing how many startups ignore the basics.

Strategic Rate Limiting
This is the simplest tool in your arsenal. Don't offer an unlimited buffet. Set generous but firm limits based on your tiers. A free user might get 10 complex summaries per day, while a Pro user gets 100. This prevents a single user from running up a massive bill, accidentally or maliciously.
Introduce Usage-Based Tiers
Flat-rate subscriptions are simple, but they're a poor fit for variable costs like LLM APIs. A better model is to include a generous token allowance with each plan (e.g., 5 million tokens/month for $19) and then charge for overages. This ensures your power users pay for what they use, keeping your business profitable.
Implement Smart Caching
Is your tool summarizing popular articles? Are multiple users asking the same question to your chatbot? Cache the results. Hitting a database is orders of magnitude cheaper than hitting an LLM API. A simple Redis cache layer can save a surprising amount of money on redundant queries.
Use Cheaper Models for Simpler Tasks
Not every task needs a flagship model. For things like text classification, basic formatting, or simple Q&A, a cheaper and faster model like Claude 3 Haiku or Gemini 1.5 Flash can do the job for a fraction of the cost. Route tasks intelligently based on complexity. Use the expensive scalpel, not the expensive chainsaw, for delicate work.

A simple logging wrapper (example)

You don't need a complex system to get started. Here’s a conceptual JavaScript snippet showing how you could wrap your OpenAI calls to log usage per customer.

// This is a simplified example, not production code.

async function callOpenAIWithCostTracking(prompt, customerId) {
  // Your existing OpenAI API call logic
  const response = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: prompt }],
  });

  const usage = response.usage; // { prompt_tokens: 123, completion_tokens: 456 }
  const inputCost = (usage.prompt_tokens / 1000000) * 3.00; // GPT-4o input pricing
  const outputCost = (usage.completion_tokens / 1000000) * 10.00; // GPT-4o output pricing
  const totalCost = inputCost + outputCost;

  // Log it to your database
  console.log(`- LOGGING: Customer ${customerId} request cost $${totalCost.toFixed(4)}`);
  // await db.logLLMUsage({ 
  //   customerId: customerId,
  //   model: 'gpt-4o',
  //   promptTokens: usage.prompt_tokens,
  //   completionTokens: usage.completion_tokens,
  //   cost: totalCost
  // });

  return response.choices[0].message.content;
}

// When a user makes a request:
// const result = await callOpenAIWithCostTracking("Summarize this for me...", "customer-123");

Start tracking today

Building your own logging wrapper is a solid first step, but maintaining it at scale gets annoying fast. fwiw, I use a simple open-source tool called LLMeter that does exactly this — it wraps the provider APIs and logs costs per user to a dashboard, no proxying required. Might be worth a look if you're in the same boat and don't want to build the tracking yourself.

But honestly, whether you build it, use a tool, or just run a script, the important thing is to start tracking your per-user LLM spend today. Your bottom line will thank you.

5 Ways I Reduced My OpenAI Bill by 40%

John Medina — Wed, 01 Apr 2026 21:19:50 +0000

When you first start using LLMs in your product, the costs seem manageable. But as you scale, they can quickly become one of your biggest expenses. A few months ago, my OpenAI bill was getting out of hand. I
knew I had to do something about it.

After a few weeks of focused effort, I managed to cut my monthly LLM spend by over 40%. Here are the five most impactful changes I made.

Caching is Your Best Friend

This one might seem obvious, but it's amazing how many people don't do it. I found that a significant number of my API calls were for the exact same prompts. I set up a simple Redis cache to store the results of
common prompts. If a prompt is already in the cache, I just return the cached response instead of hitting the OpenAI API.

This is especially effective for things like summarizing the same article for multiple users, or for common customer support questions. It's a quick win that can save you a surprising amount of money.

In my own application, I have a feature that generates a market analysis for specific keywords. I noticed that popular terms like "AI in Healthcare" were being requested hundreds of times a day by different

users. By implementing a simple Redis cache with a 24-hour TTL for the generated analysis, I achieved a cache hit rate of over 60% for the feature. This single change cut the feature's operational costs in half
with zero impact on the user experience.

Use Cheaper Models for Simpler Tasks

Not every task requires the power (and cost) of GPT-4o. I was using the most expensive model for everything by default. I did an audit of all my API calls and realized that many of them were for simple tasks

like sentiment analysis, keyword extraction, or basic summarization.

I switched to using cheaper, faster models like gpt-3.5-turbo for these tasks. I even use claude-3-haiku for some things. The cost difference is huge, and the quality is more than good enough for simpler use

cases. The key is to build a simple router that sends prompts to the right model based on the task's complexity.

You Can't Optimize What You Can't Measure

This was the biggest one for me. I had no idea where my money was actually going. I just had a single number at the end of the month.

To get a handle on it, I built a cost monitoring dashboard called https://llmeter.org. It connects to my OpenAI, Anthropic, and other provider accounts and gives me a detailed breakdown of my spend by model, by
feature, and even by user.

Within the first week of using it, I found a single user who was responsible for almost 20% of my total costs. I was able to optimize their usage. This one insight saved me over $200 in the first month.

If you don't have visibility into your costs, you're just guessing.

Prompt Engineering is Cost Engineering

The shorter and more efficient your prompts are, the less you'll pay for both input and output tokens. I spent a few days going through my most common prompts and optimizing them for brevity and clarity.

For example, instead of a verbose prompt like:
▎ "Please analyze the following customer feedback and tell me if the sentiment is positive, negative, or neutral. Also, please extract the key topics of the feedback. The feedback is: [text]"

I changed it to a more concise, system-style prompt:
▎ "Analyze sentiment (positive/negative/neutral) and extract key topics. Input: [text]"

This simple change reduced my average prompt size by about 30%, which adds up to significant savings at scale.

Set Budgets and Alerts

This is your safety net. Most LLM providers don't have great built-in budget alerting. You usually find out you've overspent when you get the bill at the end of the month.

I set up daily and monthly budget alerts in LLMeter. If my spend goes over a certain threshold, I get an email and a webhook notification. This lets me catch any unexpected spikes in usage before they become a

major problem. For instance, I set a daily budget of $50. Last week, I got an alert at noon that I had already hit $45. I quickly discovered a runaway script in a new deployment that was making thousands of

unexpected API calls. I disabled the feature, fixed the bug, and redeployed. Without that alert, the script would have run all day and cost me over $100 instead of just $45. Simple, but it gives me peace of

mind.

Controlling your LLM costs is all about being intentional. By caching, using the right models, measuring everything, optimizing your prompts, and setting up alerts, you can make your AI features much more

profitable and sustainable.