DEV Community: John Medina

Your prompt is getting longer without you knowing it (and it's killing your margins)

John Medina — Tue, 12 May 2026 21:39:59 +0000

I've been looking at LLM billing patterns lately, and there's a silent killer that creeps up on almost every team: prompt inflation.

When you first build an AI feature, your prompt is tight. Maybe 500 tokens for the system instructions and 100 for the user query. The math looks great. "This will cost us fractions of a cent per call," you tell the team.

Fast forward three months.

Someone added conversation history to make the bot "smarter." Another dev added a massive RAG context block because the model hallucinated once. Product asked for formatting instructions, so now the system prompt is a 2,000-word essay.

Suddenly, your baseline request is 8k tokens.

The worst part is that user value doesn't scale linearly with prompt size. But your OpenAI bill sure does. If you're running at scale, you're suddenly paying $0.05+ per request for a feature you modeled at $0.005.

If you just look at your monthly total on the provider dashboard, it just looks like you're getting more usage. You think "growth is good" until the Stripe payout hits and you realize your margins are gone.

You need to track cost per user and cost per feature, not just total spend. If you see specific users driving crazy costs, they're probably accumulating massive context windows that you need to truncate.

fwiw, I ran into this exact issue, which is why I built LLMeter (https://llmeter.org?utm_source=devto&utm_medium=article&utm_campaign=2026-04-21-prompt-inflation-margin-killer). It's an open-source, proxy-free way to track this stuff. It attributes costs down to the user ID level so you can actually see who is dragging around a 10k token history.

Stop assuming your prompt is the same size it was on day one. Track it.

The Hidden 43% — How Teams Are Wasting Almost Half Their LLM API Budget

John Medina — Fri, 08 May 2026 23:20:51 +0000

You look at your provider dashboard and see one number: the total bill. It's like getting an electricity bill that just says "$5,000" with no breakdown of whether it was the AC, the fridge, or someone leaving the lights on all month.

tbh, most AI startups are flying blind right now. We recently looked into the cost breakdown for several teams and found something crazy: almost 43% of LLM API spend is completely wasted. It’s not about paying for usage; it’s about paying for bad architecture.

Here’s where the leaks are actually happening:

Retry Storms (34% of waste)
Your agent fails to parse a JSON response, so it retries. And retries. Sometimes 5-10 times in a loop. You aren't just paying for the failure, you are paying for the massive context window sent every single time.
Duplicate Calls (85% of apps have this issue)
Multiple users asking the exact same question, or internal systems running the same RAG pipeline on the same document. Without caching at the provider level, you're paying OpenAI to generate the identical tokens twice.
Context Bloat
Sending the entire 50-page document history when the user just asked "what's the summary of page 2". RAG is great, but shoving everything into the prompt "just in case" is burning your runway.
Wrong Model Selection
Using GPT-4o or Claude 3 Opus for simple classification tasks when Haiku or GPT-3.5-turbo would do it for a fraction of the cost.

You can't fix what you can't see. That's exactly why I built LLMeter (https://llmeter.org?utm_source=devto&utm_medium=article&utm_campaign=hidden-43-percent-llm-waste). It's an open-source dashboard that gives you per-customer and per-model cost tracking. Stop guessing who or what is draining your API budget.

Fwiw, just setting up basic budget alerts and seeing the breakdown by tenant usually drops a team's bill by 20% in the first week. Give it a try, it's open source (AGPL-3.0) and you can self-host or use the free tier.

The week your AI coding tier got smaller

John Medina — Wed, 06 May 2026 23:15:24 +0000

In 48 hours this week, two of the biggest AI coding platforms confirmed the same thing: your unlimited subscription was never sustainable for how you actually use it. The provider will be the one who decides when to cut you off.

Anthropic silently removed Claude Code from Pro on a "2% A/B test" (later reversed). Their Head of Growth justified it saying "usage has changed a lot and our current plans weren't built for this." GitHub paused new Copilot Pro signups and dropped Opus from Pro entirely.

One dev on HN said sending 3-4 messages to Opus 4.7 blew through their $20 plan limits and consumed $10 of extra usage.

Simon Willison framed the trust break: "Should I be taking a bet on Claude Code if I know that they might 5x the minimum price of the product?"

The structural takeaway for any team shipping AI features: the invoice is the governance boundary, not the plan page. The provider's unit economics are now public. Every user is a small loss when they exceed the pricing assumption, and no vendor has found the pricing floor yet.

Teams that cannot meter their own spend per-customer, per-agent, per-task are now one pricing memo away from being unprofitable overnight.

The concrete fix:

track your tokens (not the invoice's)
use per-customer attribution (so you know whose usage is killing you)
implement hard budget caps at the agent level. Alerts don't stop a runaway loop.

This is exactly what LLM Budget Guard is being built for.

Here is how a wrapper around the SDK produces per-customer token attribution without waiting for invoice day:

import { wrapOpenAI } from 'llmeter';
import OpenAI from 'openai';

const openai = wrapOpenAI(new OpenAI(), {
  projectId: 'prod-cluster',
  tenantId: 'cust_883'
});

// Cost is now tracked per customer automatically
const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Generate report' }]
});

Track your costs early. Check out LLMeter to get started with attribution.

Bun Is Porting from Zig to Rust — Here's Why That Matters If You Run LLM Workloads

John Medina — Wed, 06 May 2026 17:11:01 +0000

This week Bun published its internal Zig→Rust porting guide — a signal that the runtime is migrating core components from Zig to Rust. The HN thread hit 700+ points in 24 hours.

The Rust move is a reasonable technical bet. Rust's ecosystem maturity, tooling, and contributor onboarding advantages at Bun's scale are real. But for teams running LLM workloads in production, the migration surfaces a question worth thinking about: what does it mean that your JavaScript runtime is now an Anthropic asset?

Background: December 2025

Anthropic acquired Bun in December 2025. For most developers this was a minor footnote. For teams running AI pipelines, it quietly changed the vendor dependency structure.

If you're using Bun as your JS runtime, Claude Code as your AI CLI, and Anthropic as your LLM provider, all three now share the same balance sheet. The Rust port doesn't change that — it makes Bun faster and easier to contribute to, but it doesn't change who owns it.

Why the shared balance sheet matters at the billing layer

The last 90 days produced 6 separate LLM billing incidents across the major providers:

Date	Incident	HN
Mar 2026	Anthropic Pro A/B — silent tier reclassification	47854477
Mar 2026	Cursor per-token surprise — $200→$500 overnight	47847849
Apr 2026	GitHub Copilot 7.5x billing multiplier	#192911
Apr 2026	GitHub Copilot 27x billing trap	47923357
Apr 2026	OpenClaw trigger-word charges	47963204
Apr 2026	HERMES.md rate reclassification	47952722

None of these were announced in advance. All of them hit teams that thought they had controls in place — dashboards, alerts, rate limits set through the vendor's own UI.

The pattern: vendor-side controls fail at the worst moment because they live inside the system making the billing decision.

What out-of-band enforcement looks like

The durable fix isn't switching runtimes or providers. It's moving cost enforcement outside the vendor stack.

Enforcement that runs before the API call goes out — synchronously, without a network round-trip — can't be overridden by a policy update on the vendor side. The call either goes out or it doesn't. No spend is committed until your own cap logic says it's safe.

import { BudgetGuard, wrapAnthropic } from '@simplifai/budget-guard';

const guard = new BudgetGuard({
  global_cap_per_day_usd: 50,
  per_customer_cap_per_day_usd: 2,
});

const anthropic = wrapAnthropic(new Anthropic(), guard);

// BudgetCapError thrown before the call if cap exceeded
// The call never goes out. No spend incurred.
await anthropic.messages.create({ ... });

When a cap is hit you get structured data — scope, spend_usd, cap_usd, retry_after — not a Slack alert 10 minutes after the damage is done.

The Rust port is good news

Genuinely. Faster startup, better memory safety, easier for external contributors. If you're building on Bun, the migration is a net positive for stability.

The vendor dependency concern is a separate question from runtime quality. Bun being well-engineered in Rust doesn't change that it's now part of the same vendor stack as your LLM calls. For teams where that matters, the answer isn't a different runtime — it's enforcement that lives outside all of them.

SDK (TypeScript, zero deps, 29 tests): npm install @simplifai/budget-guard

Managed version waitlist → simplifai.tools/validate/budget-guard

27 days to the DeepSeek V4-Pro cliff: what a 4x price jump looks like in production

John Medina — Tue, 05 May 2026 13:48:06 +0000

So here's the thing about the deepseek v4-pro pricing schedule that has been making the rounds on hn this week (thread 48002136, 494 pts / 198 comments) — the 75% promotional discount everyone has been routing to is calendar-deterministic. it expires 2026-05-31 at 15:59 UTC. that's 27 days from today. on june 1 the same agent that costs you $87 in tokens will cost you $348.

most teams i've talked to about this have a vague awareness ("yeah we've been saving with deepseek") and zero plan for the cliff. nobody has a runbook for what happens when a long-running agent session starts before the price flip and finishes after it. nobody has tested whether their billing dashboard refreshes mid-session or only on the next invoice. and nobody has checked if their fallback provider is actually configured to take over at 16:00 UTC on the 31st.

this isn't a "scandal" the way openclaw or hermes.md or the copilot 27x pivot were. deepseek published the schedule openly. but the failure mode is identical to a stripe price-update event delivered with no buyer-side notification: you've been routing 60-80% of agent traffic to v4-pro since the 04-26 launch, the meter ticks up at 4x without an alert, and the line on june's bill is the first thing you see.

what's actually happening

the published schedule (api-docs.deepseek.com/quick_start/pricing) is unambiguous:

line item	promo (now → may 31 15:59 UTC)	post-cliff (june 1 onward)	multiple
output tokens	$0.87 / 1M	$3.48 / 1M	4x
input cache miss	$0.435 / 1M	$1.74 / 1M	4x
input cache hit	$0.03625 / 1M	$0.145 / 1M	4x

every line is exactly 4x. the cache-hit tier is the one that catches people — at $0.036/M it's basically free and most teams stopped instrumenting after the first week. at $0.145/M it's still cheap but cache-hit volume is usually 5-10x cache-miss volume, so the absolute dollar delta on cache-hit can dwarf the headline output-token delta.

the deepclaw thread (claude code → deepseek wrapper, hit hn front page same week) is full of "$0.06 per task with v4-pro" anecdotes. every one of those becomes $0.24 on june 1. the $30/mo dev-side budget becomes $120. the $4k/mo agent fleet becomes $16k. nothing on the deepseek side of the wire changes — same model, same endpoint, same response — just a 4x multiplier on the meter.

what we learned doing the math on our own usage

i ran our last 30 days of llm spend through the cliff scenario assuming we keep the same routing distribution. three observations worth sharing:

1. cache-hit dominance flips the headline number. our v4-pro mix is ~70% cache-hit / 25% cache-miss / 5% output-heavy. on the promo schedule cache-hit is ~6% of total spend; post-cliff it's ~22%. the "output tokens went 4x" story misses where the dollars actually sit.

2. mid-session pricing flips are a real failure mode. we have agent runs that span 4-6 hours. on may 31 a session that starts at 12:00 UTC and finishes at 18:00 UTC will see the promo rate for 3h59m of tokens and the post-cliff rate for the rest. nothing in the deepseek api response surfaces which tier the request was billed at — you find out from the invoice. for a long-running summarization or repo-analysis job that's a $0.40 → $1.60 swing on a single run, but the agent has no way to detect it and slow down.

3. multi-provider failover is necessary but not sufficient. the obvious move is "fail over to anthropic / openai if deepseek crosses some price threshold." but the threshold isn't price (deepseek hasn't changed prices, the schedule has) — it's calendar. you have to encode "after 2026-05-31 15:59 UTC, if i'm still routing to v4-pro, stop" in the routing layer itself. and you have to test it before the day, because if your e2e test environment hits real apis on june 1 you've lost the dry run.

practical implications (things you can do this week)

these are the actions, in priority order, that close the cliff exposure for a typical team running 60-80% deepseek-v4-pro traffic:

audit your last 30 days of v4-pro spend by tier. group by cache-hit / cache-miss / output. multiply each line by 4x and look at the dollar delta, not the percent. the result is your post-cliff monthly run-rate floor. if it scares you, the rest of this list is mandatory; if it doesn't, you can stop here.
add a hard cutoff in your router by may 30, not may 31. "switch off v4-pro at 15:59 UTC on the 31st" is the obvious answer and the wrong one — it puts a billing event on the same calendar minute as a routing event with no headroom. switch off at midnight UTC may 30, run on the post-cliff equivalent (deepseek-v4 base, anthropic haiku, gpt-4.1-mini) for 24-48 hours, validate that quality + latency hold, then make a deliberate "do we want v4-pro at the new price" call with the actual numbers in front of you.
instrument cache-hit-rate dollar impact, not just request count. most observability stacks count cache hits as a vanity metric. on june 1 cache-hit becomes 4x its promo price and the volume doesn't change. plot dollar-cost-per-cache-hit over time so the cliff shows up as a step function and not a slow drift.
kill any long-running session that starts after may 31 12:00 UTC. for the 4-hour window before the cliff, force agent runs to stay under 30 minutes wall-clock. it's annoying for one shift but it eliminates the mid-session-pricing-flip class of bug entirely. cheaper than discovering on june 2 that your overnight run got billed at the post-cliff rate for 90% of its tokens.
set an out-of-band budget cap that survives provider misbehavior. the cliff is the deterministic case. the worse case is a provider you've never used before issuing a similar schedule on a tuesday with no warning. the only durable defense is a hard $/period ceiling on a layer the provider doesn't control. that's the slot llmeter sits in — count tokens before they leave your network, refuse to forward requests once the cap is hit, fail closed when the meter is uncertain. fwiw it's the same architecture every team rolls themselves on the 2nd of the month after the first surprise bill.

tldr

Deepseek did the honest thing and published the schedule. the failure mode is still that 99% of teams won't read it until june 1. you have 27 days. the cheap version of preparing is item 1 (run the numbers); the durable version is item 5 (own the cap). the rest is somewhere in between.

if you're running llm spend across deepseek + anthropic + openai and want a dashboard that flags the pre-cliff routing exposure today rather than after the invoice, that's what i've been building at https://llmeter.org/?utm_source=devto&utm_medium=article&utm_campaign=hn-week-deepseek-may-31-cliff. open-source, agpl-3.0, no proxy in the request path. drop the sdk in, point it at your providers, see the cliff exposure as a number.

happy to compare notes if you've already done the audit on your stack — what was the cache-hit delta vs the headline?

You Vibe-Coded Your SaaS Landing Page — Google Can't See It

John Medina — Mon, 04 May 2026 21:42:46 +0000

Look, I get it. You used Lovable or Bolt, shipped a beautiful landing page in 3 hours, and felt like a god. The UI is slick, animations are butter.

But it's been 3 weeks and you have 0 organic traffic.

Here is why: you shipped a Client-Side Rendered (CSR) React app. Google hates those.

I was digging into indexation stats recently. Google takes about 9x longer to index JS-heavy pages. If your site exceeds their rendering budget, indexation drops by up to 40%.

When Googlebot hits your vibe-coded site, it sees a blank HTML file and a massive JS bundle. It has to queue it for rendering, parse the JS, execute it, and only then sees the content. Most indie devs don't realize this until their Search Console stays flat for months.

Vibe-coding tools default to SPA (Single Page Application) architectures because they're easier to generate and feel faster to the user. But for a landing page? It's SEO suicide.

If you want organic users, you need SSR (Server-Side Rendering) or SSG (Static Site Generation). When Google hits the URL, the HTML needs to be there instantly.

I built LLMeter (open-source LLM cost tracking) using Next.js on Vercel specifically for this reason. SSR out of the box. No waiting for Googlebot to execute JS. It just indexes.

If you're stuck with a CSR landing page right now:

Prerender.io is an option, though it's a band-aid.
Rebuild the marketing pages in Next.js/Astro. Keep the CSR app for the actual dashboard on a subdomain (app.yoursaas.com).

Don't let the AI tools trick you into bad architecture. Ship fast, but make sure Google can actually read it, tbh.

Check out LLMeter if you're dealing with LLM API costs

GitHub Copilot's 27x Billing Trap is Closing — The Budget Guard Deadline

John Medina — Sun, 03 May 2026 01:55:07 +0000

GitHub Copilot is shifting to usage-based billing on June 1, 2026. If you've been relying on flat-rate pricing to shield you from the realities of AI coding costs, your runway just evaporated.

We've been tracking a nasty trend in the wild: the "27x billing trap." When an AI coding assistant gets stuck in a recursive loop—hallucinating a fix, failing the test, and trying again—it burns tokens at maximum velocity. Under flat-rate billing, this was an invisible annoyance. Under usage-based billing, it's a catastrophic financial event.

We saw one dev hit a -$563 bill on a $5 prepaid account because provider-side limits don't enforce in real-time. They are eventually consistent, and by the time the provider shuts you down, the damage is done.

The clock is ticking. You have less than 30 days to implement hard enforcement before the new billing model takes effect.

You don't need another dashboard that sends you a Slack alert after your budget is blown. You need an enforcement layer that kills the request before the network call is made.

That's why we open-sourced LLM Budget Guard. It's a dead-simple, local SDK enforcement layer that cuts the cord the millisecond your hard cap is reached. No proxy routing latency, no cloud gateway single points of failure. Just deterministic budget enforcement.

Self-host it and protect your runway: llmeter.org

Your LLM budget alerts won't save you if you can't map costs to users

John Medina — Mon, 27 Apr 2026 19:41:33 +0000

Most devs think they have AI costs under control because they set a $500 hard cap on OpenAI.

tbh, that's not cost control. That's just setting a timer for when your app goes down.

Here's the problem: when you hit $400 and get that warning email, what do you do? You look at the dashboard and see a massive spike in GPT-4o usage. But who caused it?
Was it your new enterprise client onboarding? (Good, increase the limit).
Was it a free-tier user who figured out how to loop your agent? (Bad, ban them).
Was it a bug in your own RAG pipeline retrying the same chunk? (Very bad, fix the code).

The provider dashboard won't tell you. It just shows a giant wall of tokens.

If you are building a multi-tenant SaaS, you need to map every single LLM call to a specific user ID. If you can't attribute the cost, you don't know your unit economics. You might be losing $2 on every $1 you make from your power users.

We built LLMeter exactly for this. It's an open-source dashboard that sits between your app and the provider. You pass the user ID in the metadata, and we track the cost per user, per day, across OpenAI, Anthropic, DeepSeek, and OpenRouter.

No proxies, no Stripe lock-in. Just raw data.

You can self-host it (AGPL-3.0) or check it out here: https://llmeter.org?utm_source=devto&utm_medium=article&utm_campaign=2026-04-25-devto-cost-to-user-mapping

The Hidden 43% — How Teams Waste Half Their LLM API Budget

John Medina — Fri, 24 Apr 2026 23:24:11 +0000

The provider dashboards show you one number — your total bill. That's like getting an electricity bill with no breakdown. You just see the total and hope nobody left the AC on.

Tbh, if you look closely at your API logs, you are probably wasting around 43% of your budget. I spent the last few weeks analyzing LLM usage across different teams, and the same leaks happen everywhere.

Here is where your money is actually going:

1. Retry Storms (34% of waste)

Your prompt fails to return valid JSON. The agent retries. It fails again. Next thing you know, your while-loop has fired 40 times. At 10k tokens a pop on Claude 3.5 Sonnet, that single user interaction just cost you a lot.

2. Duplicate Calls

Users ask the same questions. Without semantic caching, you are paying OpenAI to generate the exact same answer 100 times a day.

3. Context Bloat

Sending the entire chat history in every single request without truncation. You only need the last few turns, but your wrapper is sending 50k tokens "just in case."

4. Wrong Model Selection

Using GPT-4o for basic routing or classification tasks when a much smaller, cheaper model could do it 10x faster.

How to stop the bleeding

You can't fix what you can't see. If you don't have per-tenant cost attribution, you are flying blind. You need to know exactly which user, model, and feature is burning tokens.

I built LLMeter (open-source AGPL-3.0) to solve this. It tracks costs per model, per user, per day. It connects directly to OpenAI, Anthropic, DeepSeek, and OpenRouter to give you the exact breakdown without needing to route your traffic through a proxy.

Stop guessing. Track your per-user costs: https://llmeter.org?utm_source=devto&utm_medium=article&utm_campaign=devto-hidden-43-percent

Why AI Agencies are flying blind (and how to fix your LLM margins)

John Medina — Wed, 22 Apr 2026 14:30:49 +0000

If you're running an AI agency, you're probably building some
variation of RAG or agentic workflows for your clients.

You deliver the project, it works great, and then the first OpenAI bill hits.

Most agencies I talk to are still in the "winging it" phase when it
comes to API costs. They use one master key for dev, one for prod, and
maybe—if they're feeling fancy—one key per client.

But fwiw, per-client keys are a maintenance nightmare. And if you're
using a single master key for multiple clients, you're flying blind.

The "Averages" Trap

You might think: "I'll just charge a flat $100/mo for API usage."

Then one client decides to run a bulk ingest of 5,000 PDFs. Your
margin on that client just went negative, and you won't even know it
until the end of the month when you see a spike in the dashboard that
you can't explain.

Averages don't work for LLMs. Usage is too spiky.

3 ways to handle client attribution

1. The Metadata Header (The "Minimum Viable" way)

OpenAI, Anthropic, and OpenRouter all allow you to pass a user or
metadata header.

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [...],
  user: `client_acme_corp`
});

This is the bare minimum. It lets you export a CSV at the end of the
month and spend 4 hours in Excel trying to pivot-table your way to a
client invoice. tbh, it's better than nothing, but it's not real-time.

2. The Custom Proxy

You build a middleman. Every request from your client's app goes to
your proxy first, you log the tokens, then you forward it to OpenAI.
Pros: Absolute control.
Cons: You just added a single point of failure and 200ms of latency to
every request. Unless you're a DevOps wizard, this is usually
over-engineering for a boutique agency.

3. Real-time Attribution (The "Sanity" way)

You keep your direct provider connection but fire an async event for
every request.

This is why I built LLMeter. I needed a way to see exactly which
client was spending what, in real-time, without adding latency to the
main request.

We use it at Simplifai for our own tools. It tracks:

Cost per client (tenant)
Cost per model
Daily burn rates

If a client starts hitting the API harder than expected, I get an
alert immediately. I can then decide to upsell them, cap their usage,
or adjust the billing.

Why this matters for your agency

Clients don't like "surprise" bills. If you can show them a dashboard
(or a report) with their exact usage and cost, the trust level goes up
10x. It moves you from "freelancer with a script" to "professional AI
partner."

LLMeter is open-source (AGPL-3.0) and you can self-host it for free.
Check it out: llmeter.org

How are you billing your clients for LLM usage right now? Flat fee?
Pass-through? Or just eating the cost?

Why LLM Cost Dashboards Are Not Enough — The Runtime Enforcement Gap

John Medina — Thu, 16 Apr 2026 16:20:56 +0000

I've been looking at how teams handle LLM API costs in production, and there's a weird gap in the tooling right now. Everyone is building observability — logs, traces, dashboards. But almost no one is actually enforcing budgets at runtime.

If you are running multi-step agents or letting users chat indefinitely, discovering a $4,000 OpenAI bill at the end of the month via a dashboard doesn't help. The money is already gone.

The problem breaks down into three layers:

Attribution (knowing which user/tenant caused the cost)
Alerting (getting warned when a threshold is near)
Enforcement (blocking requests at runtime)

Most teams are stuck at layer 1. You can't enforce a per-customer budget if you don't even know what each customer is costing you.

I built LLMeter because I needed to solve that first layer. It's an open-source dashboard that tracks OpenAI, Anthropic, DeepSeek, and OpenRouter costs per user and per day. It also handles budget alerts.

Until you have per-tenant attribution figured out, trying to build runtime enforcement with API gateways is just guessing. Get the data first, then block the requests.

LLM prices dropped 80% — but are you actually saving money?

John Medina — Thu, 16 Apr 2026 16:17:17 +0000

veryone is cheering about Anthropic and OpenAI dropping API prices by 80%.
It sounds great on Twitter. But if you look at your actual billing dashboard, your costs probably haven't moved that much.

Why? Because cheaper tokens usually just mean you start wasting more tokens.

Here is the thing:

1- Context bloat
When GPT-4 was expensive, we carefully truncated histories and compressed prompts. Now that it's cheap, devs just throw the entire 128k context window at it on every single retry. The cost per token dropped, but you are sending 10x more tokens per request.

2- Agent loops
Cheaper models make agentic workflows viable, but a poorly configured while loop can still burn through your budget in minutes. When an agent gets stuck and retries 40 times, cheaper tokens don't save you—you still bleed cash.

3- Lack of per-customer attribution
It's easy to see your total OpenAI bill. But if you don't know which specific tenant or user is driving the cost, you can't optimize it. You just eat the cost.

tbh, the raw price per token is only half the story. If you can't attribute the cost per-user or per-model, you're still flying blind.

fwiw I built LLMeter to fix this for my own projects. It tracks costs per model and per user, and sets budget alerts—without a proxy in the middle. It's open-source (AGPL).

Check it out if you're tired of guessing your AI bills: https://llmeter.org?utm_source=devto&utm_medium=article&utm_campaign=2026-04-16-llm-prices-dropped-are-you-saving