I am Jarvis Osorio (Jarvis Nelson Osorio)—tech entrepreneur, AI systems architect, author and artist based in Chile. For several years the dominant narrative in software was simple: bigger is better. Throw a frontier model at the problem, route every prompt through a heavy cloud API, and accept latency, token bills and privacy tradeoffs as the price of intelligence.
I challenged that story on LinkedIn while discussing local, high-speed coding agents—see JetBrains Mellum2.1 Thinking: Why Local, High-Speed AI Agents Are the Future of Software Engineering. Specific model names will age. The architecture choice will not.
What the giant-cloud default quietly costs
Cloud frontier models are extraordinary for open-ended reasoning. They are a blunt instrument for the thousands of micro-decisions inside an IDE, a review bot or an ops agent:
- Round-trip latency that breaks flow
- Token costs that punish tight loops
- Code and secrets leaving the machine
- Fragility when the network or vendor blips
Teams were told those costs were inevitable. They are not. They are a product of routing everything to the largest remote brain.
A better split: system-one speed, system-two depth
In a related note on micro-decision models—Why the Next Generation of AI Workflows Isn't About Generating Long Paragraphs—I argued that the next wave of agentic workflows is less about essays and more about sub-100ms choices: which file to open, whether a diff looks safe, which test to run, whether a claim needs a judge.
That is a local job. Keep a fast on-device or on-prem agent for the hot path. Escalate to a larger remote model when the task truly needs breadth, long context or unfamiliar domains. Multi-model routing—not worship of one "god model"—is the engineering future I sketched in earlier writing on fusion-style architectures.
Institutions still apply
Local does not mean unsupervised. A fast agent with shell access is still an agent with power. Apply the same institutional checklist from Breaking the Centralized Order:
- Narrow roles and credentials
- Allowlists for tools
- Human or independent-model approval for irreversible steps
- Audit logs outside the agent's write reach
Speed without bounds recreates the yes-machine problem from The Compliant Mind: fluent, helpful and hard to interrupt.
Where local agents shine first
Start where feedback loops are tight and data is sensitive: code completion and review, log triage, personal knowledge search, on-device drafting against private notes. In those zones a mid-size local model with tool allowlists often outperforms a frontier call that arrives after the developer has already context-switched away.
Save the large remote model for unfamiliar domains, long-document synthesis and adversarial review—the moments when breadth beats latency. That split is curation, the same skill I describe for agentic product orchestration.
Practical questions for your stack
- Which decisions must complete inside a keystroke?
- Which ones can wait for a frontier round-trip?
- What data must never leave the machine?
- Who can pause the local agent when it is confidently wrong?
Answer those before you buy another cloud quota.
More work and links: jarvisosorio.com · hub.
The future of AI engineering is not one giant brain on a faraway GPU farm. It is the right brain, in the right place, with limits you can prove.
Originally explored on my LinkedIn. More: https://jarvisosorio.com
Top comments (0)