Why Making AI Answer Faster Is Worth $1.5 Billion

Alexander Kopylkov — Fri, 14 Aug 2026 23:01:08 +0000

Keeping an AI model fast enough to use is turning out to be the expensive part of building one.

Fireworks AI just raised $1.5 billion at a $17.5 billion valuation. The company doesn't build AI models. It takes other companies' models and makes them run faster and cheaper. That round is a clear signal about where the money in AI is actually moving right now: not just toward smarter models, but toward making the models that already exist usable at scale.

Most people size up an AI company by how smart its model seems. That's the wrong first question.

Fireworks is now doing more than $1 billion a year in revenue, up five times from last year. It moves more than 40 trillion tokens (roughly, pieces of text) through its systems every day, nearly triple what it was moving a year earlier. Its investors, including Index Ventures, TCV, and Nvidia, didn't back the company because the underlying models got smarter. They backed it because someone had to solve the unglamorous problem of making AI usable at scale, once it already works.

In my experience, that's the question most people skip. Everyone asks whether a model is good. Almost nobody asks, up front, whether it can be served fast enough and cheap enough to keep a paying user around.

A slow answer becomes a cost that grows with every user.

Running an AI model isn't like running a website. A slow-loading webpage gets a shrug. An AI model that takes three extra seconds to respond loses the user, and every one of those seconds is also a bill for the computer that was busy generating the answer.

At real scale, the cost of running a model (called inference) now eats up more than 80% of a company's AI hardware budget. Building the model is mostly a one-time cost. Running it for millions of users, every day, keeps costing money the whole time. That's the part of the bill that never levels off.

There are three things you can't have all at once: speed, volume, and cost.

Running AI models at scale comes down to a tradeoff between three things: how many requests you can handle at once, how fast each one comes back, and how much it costs you. Push on one, and the other two get worse. A chatbot needs to feel instant, so it needs speed above everything. A system processing a million documents overnight can afford to be slower, but it needs to be cheap per document. No setup wins on all three at the same time, and that tradeoff is the actual product Fireworks sells.

AI agents make the problem worse. A simple AI chat might make one call to a model. An AI agent doing a real task, like writing code or researching a topic, can make 50 to 200 calls before it's done. Prices per request dropped roughly 80% over the past year, but agents are making so many more requests that the total bill keeps climbing anyway. Getting the same job done for less money, at the same speed, is a small technical trick with a very large business attached to it.

The boring layer of AI is where the real value sits.

When I evaluate a company now, I ask a question I used to skip: once this works, can it actually run at a price and speed that people will keep paying for? A model that's too slow or too expensive to serve is still just a demo with a bill attached.

That's the real takeaway from the Fireworks round. Someone just paid $1.5 billion to guarantee a model shows up fast, cheap, and on time, and that's the layer of AI worth watching most closely right now.

Why Token Cost Became a Real Line Item I Track

Alexander Kopylkov — Sun, 05 Jul 2026 22:09:04 +0000

A founder showed me a dashboard last month with a metric I hadn't seen before: cost per completed task, the dollar amount it takes an agent to answer one real request, start to finish. Not total spend. Not cost per user. Cost per task. Once I understood why he tracked it that specifically, I couldn't stop noticing how many AI products don't.

Gross margin stops being a useful signal once compute enters the unit economics.

For most of software's history, 80% gross margin meant healthy, and 40% usually meant a services business dressed up as software, because the cost of serving one more user was close to zero. AI-native products are running closer to 52% gross margin on average now, against the 70-80% range that used to be the baseline for traditional SaaS. That's the shape of the whole category, a direct result of every response calling out to a model that costs real money to run, every single time.

The tell is that the margin number stops telling you anything on its own. Two products can report the same 55% gross margin, one well-optimized and improving, the other one growth spurt away from watching its economics worsen with scale. You can't see the difference without going one level deeper.

A single agentic task is rarely one model call. It's usually five to ten.

A chat response is one call in, one response out. An agent that plans, picks a tool, executes it, checks its own work, and then responds chains five calls together at minimum, and agents in general make three to ten times the model calls a simple chatbot does for what looks, from the outside, like the same request.

That compounding is why costs for agentic products rarely scale in a straight line with users. A team that budgets for its bill to track headcount gets blindsided when it tracks agent loops instead, growth that looks linear on a user chart can look exponential on an inference invoice. Chatbot-era intuition, where more users meant roughly proportional cost, doesn't prepare you for how quickly those extra calls compound.

Falling token prices raised the ceiling on usage.

The price of a single token has genuinely fallen. Anthropic's current Opus pricing sits at $5 per million input tokens, a 67% cut from the $15 per million tokens Opus cost two generations back. That's the kind of price drop that should shrink a bill. Instead, every agentic task multiplies that lower price by three to ten calls instead of one, so a cheaper token still buys a bigger invoice. Somewhere in that expansion sits a real threshold: below roughly 50 million tokens a month, a managed API is usually still the cheaper option; above 100 million, self-hosting starts winning on unit economics.

Most of that cost is recoverable through caching, a straightforward engineering fix.

Semantic caching, recognizing when a new request means roughly the same thing as one already answered, took one documented team from an 18% to a 67% cache hit rate and cut their bill by 73%. Prompt caching, reusing repeated context instead of recomputing it, prices cache reads at a tenth of a fresh input token, up to a 90% saving on that portion of the bill. Neither technique requires a bigger GPU budget. Both are ordinary engineering discipline applied to a cost category that used to get ignored until the invoice arrived.

I ask for this number now because it predicts more than a margin ever did.

I still spend most of my time on product and market, same as always. But I've added one question to every technical conversation: what does it cost you to finish one task, end to end, right now? The teams that can answer without opening a spreadsheet are usually the same ones who saw the cost cliff coming before it hit them.

DEV Community: Alexander Kopylkov

Why Making AI Answer Faster Is Worth $1.5 Billion

Keeping an AI model fast enough to use is turning out to be the expensive part of building one.

Why Token Cost Became a Real Line Item I Track