A GitHub repo I starred in January does not run anymore. The tutorial I followed in February has a pinned issue that says "keys getting banned." The Mac mini I was going to buy is still on the wish list, which turns out to be the only part of this I got right.
This is not an anti-AI post. The tools are real, the tokens are real, the productivity wins are real. But most AI tutorials in 2026 are selling layers 1 and 4 of a four-layer stack and hoping you do not look at layers 2 and 3 until after your credit card is on file.
This post is a teardown of those four layers, with real numbers, and a checklist you can run on any tool before you commit hardware or a subscription to it.
The four layers of the hype stack
| Layer | What it is | Who pays for it |
|---|---|---|
| 1. UI wrapper | The app you interact with | You (subscription) |
| 2. Orchestration | Agent loop, tool calls, memory, retries | You (eventually, in tokens) |
| 3. Inference | The model behind the curtain | You (tokens) or the provider (loss) |
| 4. Hardware | Whatever runs the inference | Data center, or your desk |
Most hype cycles concentrate on layers 1 and 4 because those are the layers a YouTube thumbnail can sell. A slick dashboard on layer 1. A glowing Mac mini on layer 4. Layer 2 is engineering and hard to video. Layer 3 is invisible until the bill arrives.
Rule of thumb: If a tutorial spends more time unboxing a Mac mini than discussing the inference bill, layer 3 is where your surprise is going to land.
Layer 1: The wrapper
Call it what you want. Agent, copilot, assistant, IDE extension, self-hosted dashboard. The UX sits on top of someone else's model. That is not a bad thing. Wrappers are legitimate businesses. The problem is when the pitch implies the wrapper is the product, which it almost never is.
Audit questions for layer 1:
- Can I swap the model underneath without rewriting my workflow?
- What happens to my prompts, context, and memory if the provider changes ToS?
- Is there a ToS clause in the provider's terms that names this tool, or its category, as prohibited?
That last one bites harder than people realize. As of April 2026 there are several major provider ToS updates that explicitly prohibit certain third-party agentic tools. Users have reported keys being revoked. A wrapper that routes through a non-permitted path is one enforcement pass away from being a paperweight.
Layer 2: Orchestration
This is where the real engineering lives. Tool calling, retries, memory, sandboxing, error recovery, concurrency, cost control. Most OSS projects that show off a "watch it book my flight" demo are one degraded model or one retry storm away from the same flow taking forty minutes and four dollars in tokens.
Agent loops are where cost explodes. A single user message can trigger ten or fifteen tool calls, each with a full context window of system prompt, tool descriptions, and prior turns. Input tokens dominate. You do not see this on the demo.
# Simplified agent loop cost surface
def agent_run(user_msg):
context = load_memory() # N input tokens
while not done:
response = llm.call( # N input + M output tokens
system + tools + context
)
tool_result = run_tool(response)
context.append(tool_result) # grows N each iteration
return final_answer
Notice the loop. Each iteration includes the full context of every previous iteration. A ten-step agent run with a 30K-token system prompt and 4K-token tool results runs roughly 300K input tokens before you count output. At current Claude Sonnet rates that is about $0.90 for a single user request. Do that twenty times a day and you have replaced a $20 subscription with a $540 month.
Real numbers from the All-In podcast in February 2026:
- Jason Calacanis: his team's AI agents were running at $300 a day, roughly $110K annualized.
- Chamath Palihapitiya: started imposing token budgets on his developers. Direct quote, paraphrased: "I'll run out of money."
- Mark Cuban: eight Claude agents at $300 a day plus a human maintaining them at $200 a day costs more than the employee they replace.
These are people who can afford it. They are flinching.
Audit questions for layer 2:
- What is the worst-case token spend on a single request loop?
- Does the agent have a hard cap on tool-call iterations?
- Is retry logic fixed-interval or exponential? Exponential plus a flaky tool is where budgets die.
- Does the memory layer grow unbounded, or is it summarized?
Layer 3: Inference
Every tool you are looking at, self-hosted or otherwise, runs on someone's inference bill. When a subscription "includes" a feature, somebody is eating the difference. That works until it does not.
Recent, concrete, April 2026 examples:
- April 4, 2026: Anthropic revoked subscription billing for third-party tools. OpenClaw and similar tools that previously rode on a Pro/Max subscription now bill to Extra Usage.
- April 21, 2026: Anthropic A/B tested removing Claude Code from the $20 Pro plan for 2% of new signups. The pricing page briefly reflected the removal before clarification.
- 2025 and 2026 economics: OpenAI reportedly burned about $8 billion against $13 billion in revenue in 2025, with roughly $14 billion in losses projected for 2026. Sam Altman has publicly said OpenAI loses money on the $200/month ChatGPT Pro tier.
- GTC 2026: Jensen Huang put a gigawatt data center, amortized over 15 years, at roughly $40 billion empty.
Per-token prices have collapsed. GPT-4 input tokens went from $30 per million to $2.50 per million in eighteen months, an 80 to 90 percent drop. The floor is not zero, and consumption keeps rising faster than price falls.
| Metric | 2023 | 2025/2026 | Change |
|---|---|---|---|
| GPT-4-class input tokens | $30/M | ~$2.50/M | -90% |
| Avg enterprise AI spend | — | $85K/month | +36% YoY |
| Orgs spending >$100K/month | baseline | 2x baseline | doubled |
| Daily tokens processed (China alone) | 100B (Jan 2024) | 140T | ~1400x |
The structural problem: per-token costs are down 80-90% in 18 months, per-user consumption is up faster. Your effective monthly bill is not going down.
Audit questions for layer 3:
- If the provider moved every call to strict pay-per-token tomorrow, what would my monthly bill look like?
- Can I cap spend at the provider level, not just the app level?
- Does the tool work with a local fallback that preserves 80% of function, or does it hard-fail without the frontier model?
Layer 4: Hardware
The Mac mini posts, the "$2000 gets you your own AI" guides, the affiliate-stuffed "best LLM server" roundups. This is not a scam. Apple Silicon really is good for local inference. Unified memory, no PCIe copy penalty, low idle power, silent. All genuinely useful.
But the honest read on local models for agentic workflows in April 2026:
| Model | RAM needed | Speed on consumer Apple Silicon | Usable for agents? |
|---|---|---|---|
| Kimi K2 (1.8-bit quantized) | 245GB+ | 1-2 tokens/sec on Mac Studio 512GB | No, too slow |
| Qwen3-Coder-30B (MoE, 3B active) | 64GB | 10-15 tokens/sec on M4 Pro | Yes, for simple flows |
| GLM-4.7-Flash (9B active, 128K ctx) | 24GB | ~20 tokens/sec on M4 | Yes, for coding subset |
| GPT-OSS-20B | 24GB | ~15 tokens/sec on M4 | Yes, general purpose |
Local models are usable. They are not the same as frontier cloud models for multi-step agentic work, especially when the task needs long-horizon planning or serious tool use. A local 30B is a working tool. A local Kimi K2 is a science project unless you own a Mac Studio cluster.
Anyone telling you otherwise is usually selling you something (affiliate link, course, the same course with a new thumbnail).
Audit questions for layer 4:
- Do the benchmarks in the guide match the task I actually want to do, or are they summarization benchmarks being used to imply coding performance?
- What is the tokens-per-second floor for the model at the context length I'll actually use?
- If I only use the hardware four hours a day, is the cloud API still cheaper at my real usage?
Four signals a stack is on borrowed time
Run these before you commit.
- The pricing page changed silently in the last 30 days. Check the Wayback Machine. Compare.
- The tutorial's recent comments are mostly "this doesn't work for me." Filter by newest. If the top new comments are bug reports, the snapshot decayed.
- The GitHub issues tab has an open "API keys being banned" or "provider blocked us" thread. Search by keyword. This one is terminal.
- The tool routes through a provider whose ToS explicitly names it, its category, or its pattern. Find the ToS. Ctrl-F the project name.
Three of four true: the stack is late cycle, budget accordingly or skip.
One or two true: yellow flag, keep watching.
Zero: you might be early, which is also risky, but at least the risk is legible.
What to actually build on
Boring but it holds up:
- Prefer tools where the provider bills you directly. The bill is honest. The less abstraction between you and the token meter, the fewer surprises.
- Prefer stacks where swapping the underlying model is a config change, not a rewrite. Portability is the cheapest insurance you can buy.
- Treat GitHub stars like follower counts. Not meaningless, but not the product.
- Assume any third-party tool that rides a major provider's consumer subscription instead of an API key is temporary, and price that temporariness into your decision.
- Run the 30-day test. If a tutorial still has fresh comments saying "this works" a month out, it is probably safe. If not, skip.
None of this is anti-AI. It is pro not-getting-caught-out. The tools are real. The wins are real. The token bill is also real. The layer you are not looking at is usually the one that bites.
What signals have you seen that a tool is late cycle? Drop them in the comments. I am genuinely collecting patterns, because the next wave of this is already starting.
Top comments (0)