Day 1 of 7 — The Agentic Engineer’s Field Notes
The demo always works.
Picture it. A small audience, a slide deck, a prompt typed into a textbox. The agent reads a Jira ticket, opens a browser, scrapes a competitor’s pricing page, drafts a slide, posts it to Slack. Applause. Calendar invites for follow-up calls. Someone in product asks how soon it can ship.
Six months later, that agent is still not in production. Maybe it never will be.
In May 2026, this is no longer just an anecdote. Gartner is now publicly forecasting that over 40% of agentic AI projects will be cancelled by 2027. Industry surveys of shipped agent projects show a brutal split: narrow-scope projects (single workflow, defined boundaries) deliver on time at roughly 65%, with a median slip of about two months. Broad-scope projects (multiple workflows, multiple integrations) deliver on time at 16%, with a median slip approaching ten months. The gap is not the model. The gap is everything around the model.
If you are building agents in 2026 and your demo-to-production ratio looks like everyone else’s, this series is for you. Over the next seven days I’m going to walk through the system I use to build agentic systems that actually ship, hold up under load, and survive the next model upgrade. Today is the mental shift that has to happen before any of the rest of it lands.
Agents are not chatbots with tools bolted on
The most expensive mistake I see is treating an agent as “a chatbot, but it can also call functions.” That framing is intuitive and almost entirely wrong.
A chatbot is a stateless transformation: text in, text out. The user is in the loop. They read the answer, decide if it’s any good, and either accept it or rephrase. The chatbot’s mistakes are transient. A wrong answer in turn three is forgotten by turn five.
An agent is a closed loop where the model is making decisions, taking actions, and feeding the consequences of those actions back into its own context. The user is no longer the corrector. The model itself is the corrector. Mistakes don’t evaporate — they compound. A bad tool call at step three poisons the context at step four, which produces a worse decision at step five, and by step nine the agent is confidently wrong in ways that would have been caught instantly by a human in the loop.
Andrej Karpathy, in his 2025 LLM year-in-review, put a useful frame on what’s happening. We are not building “smarter chatbots.” We are summoning what he calls ghosts — entities optimized differently than biological minds, with what he labels jagged intelligence: simultaneously polymath-genius and grade-schooler-confused. They live on your computer. They make decisions. And, in his words, “it’s not just a website you go to like Google, it’s a little spirit/ghost that ‘lives’ on your computer.”
When you bolt tools onto a chatbot, you are not building a chatbot with hands. You are giving a ghost a loaded weapon.
The demo-to-production cliff
Most agent demos work because the demo path has been implicitly hand-tuned. The prompt has been iterated on the demo input. The tools have been tested on the demo input. The expected trajectory is two or three steps. The data is clean. The user is forgiving. The cost doesn’t matter because nobody is counting tokens during a slide.
Production looks nothing like that. A year of production agent deployments has surfaced a now-canonical list of failure modes. They show up in conference talks, post-mortems, and academic workshops (ICML even ran a Failure Modes in Agentic AI workshop earlier this year). The recurring themes:
- Silent failures. Systems appear healthy. Outputs look reasonable. The agent has been confidently wrong for six weeks and nobody noticed.
- Black-box decisions. When something does go wrong, no one can explain why the agent did what it did, because nobody instrumented the trajectory.
- Permission explosion. Agents accumulate access over time. By month four, the customer-support agent has read access to half your data warehouse and nobody remembers approving it.
- Runaway execution. A loop that should have stopped at iteration five runs to fifty. The bill arrives. The CFO has questions.
- Compounding errors. Each step of a long-running agent introduces small errors. By step forty, the cumulative drift is fatal.
- Context degradation. Surveys put this at the top of the list — 65% of developers now cite missing or rotted context as the leading cause of poor AI code quality.
- Assumption-filling. When the agent doesn’t know something, it doesn’t ask. It guesses. The guess looks plausible. The downstream code calls a method that doesn’t exist.
- Self-review blindness. Ask the agent if its work is good and it will tell you yes.
Each of these is solvable. None of them gets solved if your team’s mental model is “make the LLM smarter.” The model isn’t your bottleneck. The system around the model is.
Stop optimizing the artifact you’re staring at
When teams hit the production wall, their instinct is almost always to keep tuning the prompt. They add another bullet point to the system message. They reword the instructions. They paste in three more examples. They beg the model not to hallucinate.
This is a category error. The prompt is not the artifact. The system is the artifact.
The system has at least four moving parts: the model, the tools, the memory, and the loop that ties them together. The prompt is, at most, a configuration file for one of those. If your agent is unreliable, the prompt is one of maybe a dozen places the bug could live, and probably not the most likely. I have spent days reworking system prompts on agents whose real bug was a tool that returned a 200-OK with an empty body when the real answer was “not found.” The model dutifully reasoned over an empty body and produced confident nonsense. No prompt was going to fix that.
Once you internalize that the system is the artifact, your day-to-day work changes. You stop arguing about the wording of a sentence in the system prompt. You start instrumenting the loop. You start logging every tool call and every model output. You start asking: when the agent fails, which subsystem failed? That question is answerable. “Why is the LLM dumb today?” is not.
The shift, in one sentence
Here is the framing that changed how I work, and the through-line for everything in this series:
You are not building a smarter chatbot. You are building software whose runtime behavior is decided by a probabilistic component, with all the engineering rigor that statement deserves.
Read it twice. Every word in there is doing work.
- Software — not a magic box. It has interfaces, state, error modes, and tests. It can be debugged. It has a deployment story.
- Whose runtime behavior is decided by a probabilistic component — the model’s outputs are not deterministic and never will be. Your system has to be robust to that, the way distributed systems are robust to network failure.
- With all the engineering rigor that statement deserves — observability, evals, rollback strategies, capability boundaries, cost budgets, incident response. Boring, unsexy infrastructure. The same things that make any production system work.
The teams that ship agents successfully are the teams that treat agentic systems as a new substrate for software engineering, not as a clever chatbot trick. Their mental model is closer to “I’m building a service that internally consults a brilliant but unreliable contractor on every call” than “I’m building a magic AI that does the job.”
This shift is not optional. It is, increasingly, the difference between the 16% who ship the broad-scope projects and the 84% who don’t. Anthropic’s 2026 Agentic Coding Trends Report tracks the inflection in real time: software development is shifting from writing code to orchestrating agents that write code. One case study profiles a team at Rakuten that had an agent implement a complex feature across a 12.5-million-line codebase in a single seven-hour run. That is not “AI helps engineers code faster.” That is a different shape of engineering work entirely.
The engineers who learn to do that work well — who can move fluently between the four subsystems, design tools an LLM speaks, build memory that doesn’t rot, run evals that catch silent regressions — those are the engineers who will be enormously valuable for the rest of the decade.
What the next six days look like
If you accept the shift, the natural follow-up question is: what does the engineering rigor actually consist of? That is what the next six days are about.
- Tomorrow (Day 2): I’ll dissect the anatomy of an agent that actually works in production — the four moving parts, why frameworks often hurt before they help, and the simplest agent worth shipping. I’ll also walk through Anthropic’s five composable patterns (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) and where each one actually fits.
- Day 3: Context engineering. Why Karpathy is right that this has eaten prompt engineering, the four-strategy framework that’s emerging as the canonical model, and why teams that figured this out a year ago are pulling away from everyone else.
- Day 4: Tools. The most undervalued lever in agent design. Plus the lethal trifecta — Simon Willison’s framework that has predicted four major production AI vulnerabilities in the first two weeks of 2026 alone.
- Day 5: Memory. The hardest unsolved problem in agentic systems. We’ll get into hierarchical memory architectures (Letta’s three-tier model, the LLM-as-OS paradigm), and why most “memory features” you’ve seen are theater.
- Day 6: Evals. Hamel Husain’s discipline, codified. Error analysis, binary-not-Likert judgment evals, and the single habit that separates the teams whose agents quietly improve every quarter from the teams whose agents silently rot.
- Day 7: The multi-agent debate — and the plot twist. Cognition famously published Don’t Build Multi-Agents in 2025. They published a follow-up in 2026 that significantly updates that position. We’ll go through both, plus Anthropic’s Planner/Generator/Evaluator architecture and what production multi-agent systems actually look like now.
Each piece stands alone. Together, they’re the playbook I wish someone had handed me three years ago.
A small ask before you go
If you take one thing from today, take this: stop describing your agent project by its prompt. Start describing it by its system. The next time someone on your team says “the agent is broken, can we tweak the system prompt?”, ask them which of the four subsystems they think is failing and what the evidence is. If they can’t answer, you’ve found your real problem — and it’s not the prompt.
Tomorrow I’ll show you what the four subsystems are, how they interact, and the diagram that has saved me more debugging hours than any tool I’ve ever installed.
See you then.
This is Day 1 of a 7-day series on agentic development. If this resonated, follow along — the rest of the week goes deep on context engineering, tool design, memory, evals, and multi-agent systems. No fluff, no hype, just what’s working in production in 2026.
Sources & further reading:
- Anthropic, Building Effective AI Agents
- Anthropic, 2026 Agentic Coding Trends Report
- Karpathy, 2025 LLM Year in Review
- ICML 2026 Failure Modes in Agentic AI workshop
First published at codewithrashid.com.
Top comments (0)