TL;DR — BI systems were architected around human query patterns: a handful of dashboards, refreshed on a schedule, heavily cached. AI agents issue a much larger volume of semantically similar but syntactically distinct SQL, which defeats warehouse caching and explodes compute cost and latency. The fix isn't a bigger warehouse — it's a semantic query cache that canonicalizes intent before it ever reaches the database.
Every analytics stack has an implicit assumption baked into its cost model: a small number of humans issue a small number of distinct queries, repeatedly, on a schedule. Dashboards refresh every fifteen minutes. Analysts run the same cohort query every Monday. Caching layers, materialized views, and warehouse autoscaling were all tuned against that pattern. It worked, for two decades, because the thing generating the SQL was a person with limited patience and a finite number of questions.
Agents don't have that limitation, and that's the problem nobody priced in.
The usage pattern nobody priced for
An AI agent answering "how did revenue trend in the Northeast last quarter" doesn't ask that question once. It asks it as part of a chain: first to scope the region, then to validate the date range, then to cross-check against a slightly different grain because the user's next message implies a different breakdown. Multiply that by every agent session, every retry, every self-correction loop where the agent isn't sure its first query was right so it tries a second, slightly different one just to sanity-check the number.
None of these queries are identical at the SQL text level. They differ in whitespace, column order, a CTE the agent decided to add for clarity, a WHERE clause phrased as region = 'Northeast' in one pass and region IN ('Northeast') in the next. To a human analyst these are the same question. To a warehouse's query cache, which typically keys on exact statement text or a normalized hash of it, they are completely different queries, each one a full compute job.
Dashboards were built for repetition, agents are built for variation
This is the part that's easy to miss if you're only thinking about correctness. BI tools like dashboards and semantic layers solved a trust problem — making sure everyone sees the same number for "revenue." They were never asked to solve a throughput problem, because throughput was never the bottleneck. A dashboard viewed by five hundred people still issues one query, cached once, served five hundred times.
An agent fleet inverts that ratio. One user intent can fan out into dozens of queries, and those queries are generated fresh by a model each time rather than retrieved from a fixed library. The semantic layer can still enforce that "revenue" means the same aggregation everywhere — that problem is real and worth solving — but it does nothing to stop the warehouse from re-executing a full scan-and-aggregate every time the phrasing changes. You can have perfectly governed metric definitions and still watch your compute bill triple, because governance was never the cost lever here. Volume and cache-miss rate are.
Why caching breaks: semantic similarity vs syntactic identity
Query caches, result caches, and materialized views all operate on the premise that "the same question" produces "the same bytes." That premise held when queries were authored by a small set of humans using a small set of BI tools with fixed query templates. It falls apart the moment an LLM is the one writing the SQL, because an LLM has no incentive to converge on a canonical form. It will happily write a correct, semantically equivalent query in a dozen different syntactic shapes depending on temperature, prompt phrasing, or which tool call preceded it.
This is a strictly different failure mode from text-to-SQL accuracy. The query can be completely correct and still be a cache miss. You can have a well-tuned agent, a clean semantic layer, and a fast model, and still see your warehouse behave as if every single query is novel — because syntactically, it is. The result is a cost curve that scales with the number of agent interactions rather than the number of distinct questions being asked, and those are not the same number by an order of magnitude or more.
The fix: canonicalization before the warehouse, not caching inside it
The instinct is to throw more caching at the database layer — better plan caches, result-set memoization, materialized views tuned for common filters. Those help marginally, but they're fighting the wrong layer of the stack. The warehouse can only cache what it can recognize as identical. The fix has to happen before the query reaches the warehouse: a canonicalization layer that maps an agent's generated SQL, or better yet its underlying intent, onto a normalized representation before execution.
Concretely, this looks like intercepting agent-generated queries and rewriting them against the semantic layer's metric definitions rather than letting them hit raw tables directly — not for governance reasons this time, but because a metric call like revenue(region=Northeast, period=last_quarter) is trivially cacheable in a way that five differently-phrased raw SQL statements are not. It also looks like embedding-based query deduplication at the application layer: before a query executes, check whether a semantically equivalent request has run recently, using the query's parsed intent rather than its text, and serve the cached result if the underlying parameters match within an acceptable staleness window.
This is a genuinely different engineering problem from building a semantic layer for correctness. It's a routing and caching problem that sits between the agent and the warehouse, and almost nobody's analytics stack has it yet because almost nobody needed it until agents started generating query volume that no human team ever would.
What analytics engineers should actually build now
The teams that get ahead of this aren't the ones buying bigger warehouses. They're the ones instrumenting query intent before SQL generation — logging what an agent was actually trying to find out, not just the statement it emitted — and using that intent log to build a cache keyed on meaning rather than text. They're also the ones putting hard limits on agent query fan-out, the same way API gateways rate-limit retries, because an agent that self-corrects three times per answer is tripling your compute cost for a marginal accuracy gain that's often not worth it.
None of this is exotic infrastructure. It's closer to a CDN for analytical queries than it is to a new database feature. But it requires analytics engineers to think about agents as a traffic pattern, not just a new consumer of the semantic layer. The semantic layer answers "what does revenue mean." The canonicalization layer answers "have we already answered this, in some other words, ten minutes ago." Right now almost every AI-plus-BI stack has the first and is missing the second entirely, and that gap is where the warehouse bill is quietly going.
Top comments (0)