Building an AI agent is easy. Keeping one alive for six months is not.
You can wire up an LLM and a tool in an afternoon. What's hard — what actually separates a demo from a production system — is an agent that runs for hours, touches dozens of tools, survives a restart, refuses the dangerous command, and doesn't quietly forget everything it learned yesterday.
None of that comes from the model. It comes from what's around the model.
A production agent isn't User → Prompt → LLM → Tool → Answer. It's a harness — context assembly, tiered memory, permissioning, subagents, lifecycle hooks — with the LLM sitting inside it as the reasoning component, not the whole system. Below are 12 patterns for building that harness, with notes on which ones I've actually shipped.
┌─────────────────┐
│ User │
└────────┬─────────┘
↓
┌────────────────────────┐
│ Agent Harness │
│ Context · Memory │
│ Planning · Permissions │
│ Subagents · Hooks │
└───────────┬─────────────┘
↓
┌───────────┐
│ LLM │
└─────┬─────┘
↓
┌────────────┼────────────┐
↓ ↓ ↓
Tools Subagents APIs
These patterns generalize past coding agents — I've applied versions of most of them on an enterprise sales copilot, and they map cleanly onto retrieval agents, workflow copilots, and autonomous ops tooling too.
1. Persistent Instruction File
Every new session, you re-explain: use pnpm, run tests before finishing, never push to Git. The agent forgets by tomorrow. So stop telling it — store it.
project/
├── AGENTS.md ← loaded automatically on session start
├── package.json
├── src/
AGENTS.md becomes infrastructure, not conversation:
SessionStart → Load AGENTS.md → Build Context → LLM
The trade-off: this file rots exactly like production config rots. "Use MongoDB" outlives the MongoDB → PostgreSQL migration by six months, and now the agent is confidently wrong. Treat it like code — it needs an owner and a review cycle, not a "write once" mentality.
2. Scoped Context Assembly
One AGENTS.md is fine for a small repo. In a monorepo with a React frontend, a FastAPI backend, and a Terraform infra layer, one giant file becomes a giant, mostly-irrelevant context blob.
Instead, cascade it:
Organization → Repository → Directory → Subdirectory → Task
company/AGENTS.md
company/backend/AGENTS.md
company/backend/auth/AGENTS.md ← agent working here gets all three, merged
This is the same idea as namespace-scoped config in any large system — the closer the instruction is to the code, the more specific and less stale it stays.
3. Tiered Memory
The most common architecture mistake I see: memory = vector database. It's not. Memory needs temperature.
Memory
│
┌───────────┼───────────┐
↓ ↓ ↓
Hot Warm Cold
(always (topic- (full session
loaded) loaded) history, searched
only on demand)
- Hot — "user prefers TypeScript," "never auto-push" — a handful of facts, always in context.
-
Warm — topic files (
database.md,deployment.md) pulled in only when relevant. - Cold — full session logs, searched, never bulk-loaded. > From my own build: on the Customer 360 Sales Copilot, we run Mem0 as the memory layer specifically because "load everything" doesn't scale past a handful of accounts — a rep working 200+ accounts needs this account's renewal-risk history surfaced, not last quarter's entire transcript archive dumped into context. Tiering isn't an optimization here, it's the difference between a usable copilot and an unusable one.
4. Dream Consolidation
Six months in, memory looks like this:
"We use MongoDB." → "We migrated from MongoDB." → "PostgreSQL is primary." → "MongoDB still used for some services."
Duplicates, contradictions, stale facts. Left alone, this actively degrades the agent — it's not neutral clutter, it's misinformation the agent will confidently cite.
Raw Memory → Deduplicate → Detect Contradictions → Remove Stale
→ Merge Related Facts → Update Index
Humans consolidate memory during sleep. An idle agent can do the equivalent — a background job that runs when nothing else is happening and leaves memory smaller and more accurate than it found it.
5. Progressive Context Compaction
Long-running agents eventually fill the context window. Deleting old context or killing the agent are both bad options — so compress in layers instead:
Recent turns → full detail
Older turns → light summary
Very old turns → aggressive summary
Ancient history → key facts only
Worth being precise about the distinction here, because it trips people up: memory answers what should survive across sessions; compaction answers what should survive inside this session once it gets too big. Related problems, different mechanisms — conflating them is how you end up compacting things that should have been promoted to long-term memory instead.
6. Explore → Plan → Act
A naive agent edits first and discovers the system is different than it assumed halfway through. The fix is boringly effective:
Explore (read-only) → Plan (written, reviewable) → Act (write access) → Verify
Give the agent Read / Search / Grep before you give it Edit / Write / Bash. The plan step matters as much as the read-only restriction — a written plan is something a human (or another agent) can veto before code changes exist, not after.
7. Context-Isolated Subagents
One agent doing research + planning + coding + testing accumulates context nobody needs all at once. Split it:
Parent Agent
│
┌──────────────┼──────────────┐
↓ ↓ ↓
Researcher Planner Tester
(read-only) (read-only) (test-only)
A research subagent returns structured findings, not prose — {"rootCause": "...", "evidence": [...]} — that the planner consumes directly. Smaller context per agent, smaller reasoning problem per agent.
From my own build: this is close to how we split research/planning/execution roles in the copilot — each subagent gets exactly the tool surface it needs and nothing else, which also happens to make permissioning (see #10) dramatically simpler, since you're scoping access per role instead of per giant do-everything agent.
The honest trade-off: coordination overhead is real, and nuance gets lost in handoffs. This pattern earns its complexity on genuinely large problems — don't reach for it on a two-step task.
8. Fork-Join Parallelism
Twenty independent services need the same migration. Sequential is slow; if the work is truly independent, fork it:
Parent
│
┌─────────────┼─────────────┐
↓ ↓ ↓
Agent A Agent B Agent C
(svcs 1-7) (svcs 8-14) (svcs 15-20)
│ │ │
└─────────────┼─────────────┘
↓
Join → Integration Tests
The catch that actually matters: this only works when tasks don't share state. Agent A changing an API contract while Agent B changes the consumer isn't parallelism, it's a race condition with extra steps. Verify independence before you fork — don't assume it from "these look like separate services."
9. Progressive Tool Expansion
If your MCP gateway exposes 5,000 tools, do not send 5,000 tool definitions to the LLM on every call. You'll create a tool-selection problem before you've solved the user's actual problem — and burn a large chunk of context on definitions alone.
5,000 tools → Tool Registry → Discovery → Relevance Filter
→ Permission Filter → Top-K → LLM
"Create a Jira ticket" should activate jira.createIssue, jira.searchIssues, jira.getProject — not expose AWS, Salesforce, Slack, and Kubernetes tooling the model has no reason to see. Treat tool selection as a retrieval problem, not a prompt-stuffing problem.
10. Command Risk Classification
ls and rm -rf / are both "shell commands." They are not remotely the same risk. Don't ask the LLM to self-assess danger — build a deterministic layer that decides for it:
LLM → Tool Request → Risk Classifier → Permission Policy → Execute / Ask / Deny
| Operation | Risk | Action |
|---|---|---|
ls, cat, git diff
|
Low | Allow |
npm install |
Medium | Allow / Ask |
External API call, git push
|
High | Ask |
| Delete production DB | Critical | Deny |
The principle: the LLM decides what it wants to do; the harness decides whether it's allowed. That split is the actual security boundary — not a system prompt asking the model to be careful.
From my own build: we enforce this with SPIFFE/SPIRE for workload identity and OPA for policy — deliberately outside the model's control plane entirely. A prompt telling the model "be careful with destructive commands" is a suggestion. A policy engine that structurally can't be talked out of denying
DROP TABLEis a boundary. Don't rely on the former where you need the latter.
11. Single-Purpose Tools
Bash(command) is flexible and nearly impossible to govern. Compare:
{"tool": "Bash", "command": "cat src/auth.ts"}
{"tool": "ReadFile", "path": "src/auth.ts"}
The second is trivial to validate, authorize, audit, and restrict — you can enforce "no writes outside src/" against a typed EditFile(path, changes) call in a way you simply cannot against arbitrary shell strings. Typed tools aren't just cleaner API design; they're what makes #10's permission layer enforceable at all.
12. Deterministic Lifecycle Hooks
"After every edit, run prettier" in a system prompt is a request the model can forget. A hook is not:
EditFile → PostToolUse Hook → Run Prettier (no model decision required)
SessionStart → Load config → Load memory → Agent execution
→ SessionEnd → Persist state
The rule that ties this whole list together: if something must happen every time, it doesn't belong in the prompt. It belongs in the runtime.
What this looks like assembled
USER
│
┌────────────────┐
│ Agent Gateway │
└───────┬─────────┘
┌───────────────────┐
│ Context Assembler │
└─────────┬──────────┘
┌────────────┬─────┴──────┬────────────┐
↓ ↓ ↓
Instructions Memory History
└────────────┼────────────┘
┌──────────────┐
│ Agent Runtime │
└──────┬────────┘
Explore/Plan → Tool Discovery → Permission Check
┌─────┴─────┐
↓ ↓
Allowed Denied
↓
Execute → Post Hook → State Update → loop
What I'd actually build first
Not all 12 at once. In order:
- Explore → Plan → Act — biggest reliability win for the least architecture.
- Progressive Tool Expansion — non-negotiable once you're past ~20 tools.
- Deterministic Lifecycle Hooks — moves the highest-stakes behavior outside the LLM's control entirely.
- Tiered Memory + Context Compaction — buys you long-running sessions without runaway context cost.
- Permission + Risk Classification — the actual security boundary. Subagents, fork-join, and memory consolidation are real wins — but they're complexity you earn once the first five are solid, not complexity you start with.
The takeaway
The old mental model was LLM + Tools. The current one is:
Agent Harness (memory, context, planning, state, permissions, tools, hooks)
↓
LLM
↓
Actions
The LLM stopped being the whole agent a while ago. It's the reasoning component inside the agent — the harness is what makes it trustworthy enough to run unattended.
Top comments (1)
Strong list. The part I’d add pressure to is testing the harness in a messy runtime, not only in a clean single-user task.
A lot of agent systems look solid when the flow is User -> Agent -> Tool -> Answer. The failures get more interesting when there are interruptions, stale memory, multiple agents touching adjacent state, public feedback, or incentives that make the agent want to keep talking instead of stopping.
That’s where memory, permissioning, lifecycle hooks, and risk classification stop being architecture diagrams and start becoming behavior controls.