I set out to build a small GenAI app. I ended up doing context engineering twice — once inside the app, and once inside the pipeline of AI agents that built it — and the second one is the part nobody warned me about.
The app is Dev, a Hinglish AI motivation coach https://dev-desi-ai-coach.web.app/ — live, and rough in places; it's a learning artifact, not a product). Flutter + Firebase + Gemini. The stack is unremarkable. The two context problems are where it got interesting.
Upfront honesty: most of the code was written by AI agents under my direction. What I'm claiming is the architecture, the decisions, and the orchestration — which turned out to be the actual work.
Layer 1: context engineering inside the app
The naive way to make an LLM feel "personal" is to stuff every prompt with everything — full history, all the user's goals, their whole profile. It works, then your latency and bill both blow up.
So Dev assembles context selectively per turn:
Tiered persona — a lightweight core personality loads by default; the heavier, expanded version only activates when a turn actually needs depth. Same character, fraction of the tokens.
Extracted memory — instead of replaying the whole conversation, a background step distills a few durable facts into a small capped store, and only those get re-injected. (To be precise: this is selective context assembly + a capped array, not vector RAG — I won't dress it up as something it isn't.)
Hard caps on history and output. Boring, but caps are what make cost bounded and predictable.
The trick I'm proudest of — a $0 emotion channel. Dev's mascot reacts to the emotional tone of each reply. The obvious build is a second sentiment call: more latency, more tokens. Instead I have the model emit a tiny inline tag as it streams:
[TONE:hype] Arre wah! Streak intact. Keep going. 🔥
The client parses [TONE:hype], drives the mascot animation, and strips the tag before render. Real-time emotional UX, zero extra inference. The model was already generating tokens — I just gave it one more cheap job.
(And because a limited LLM tier will throttle you: the AI layer retries with backoff honoring the API's own retry hints, and never leaks a raw error — a quota failure becomes an in-character line, not a red stack trace. Engineer the failure path like you mean it.)
Layer 2: context engineering for the agents — the part that surprised me
I didn't hand-write most of this. I ran a multi-agent pipeline and directed it like a small team: a PM agent (Claude "Fable" 5) turned my decisions into tight specs; a developer agent (Claude Sonnet 5) implemented them; an evaluator agent (Claude Opus 4.8) independently reviewed every diff before merge; a second coding agent (Cursor) ran in parallel on separate branches. They coordinated through a shared claims file, and every change had to clear the same gates — static analysis, 363 tests, a clean production build — before it merged. That discipline is the only reason agents can move fast without breaking things: they're confidently wrong sometimes, exactly like junior devs, so the gates are non-negotiable.
Here's the failure mode I didn't see coming: agents read your project's docs, and those docs have a token cost. My specs, eval notes, architecture docs, and running board — the agents' shared "memory" — quietly grew to ~258k tokens, most of it shipped, irrelevant specs. Every session, the agents were wading through a swamp of closed work. Slower, costlier, and they'd anchor on stale decisions.
So I did to the tooling exactly what I'd done to the app — tiered its context:
HOT — always-loaded essentials (current board, workflow). Budget: ≤40k tokens.
WARM — task-relevant specs, loaded only when working on that task.
COLD — an archive of shipped specs, never auto-loaded, with a one-line index so nothing's lost, just not ambient.
And I made it enforced, not aspirational — a CI step computes each tier's token weight and fails the build if the corpus exceeds budget (I capped the total at 400k). The rule: ship a feature → archive its spec in the same PR. Now the agents' context physically can't rot, because the pipeline won't let it.
That's the takeaway I didn't expect: if you build with AI agents, their context window is a resource you engineer as deliberately as your app's. Same discipline, different layer — and CI is where you make it stick.
Honest caveats (because this is dev.to)
It's a well-built single-user app on a standard stack — not distributed-systems wizardry. The value is in the practices.
The agents needed guardrails, not trust — the eval agent and the gates caught real issues the dev agent shipped. Remove either and quality drops immediately.
If you want to poke at the live thing, it's here. But the two ideas I'd actually love feedback on are the inline tone-tag pattern and the CI context-budget gate — I haven't seen either written up much, and I suspect they generalize.
Top comments (0)