"Agentic AI" is the most hyped phrase in software right now, and interviews have noticed. The junior answer to "have you built agents?" is a LangChain demo that worked twice. The senior answer knows why that demo would burn $400 in a loop on Tuesday, why the fix is a step cap and not a better prompt, why tool design matters more than tool count, why the model must never execute anything itself, and what MCP changes about all of it.
This is 50 questions at that second level — the agent loop, planning, tool design, MCP, memory, multi-agent orchestration, guardrails, evals, and production operations. Each with a short answer, a deep dive with real production code, and the senior signal interviewers listen for. The running example is Mattrx Insights — an agentic analytics copilot (110k MAU; C# .NET 9 for orchestration/governance, Python FastAPI for the agent), tools served over MCP by three servers (analytics/read, reports/actions, admin/locked), ~85,000 governed tool calls a day. Real numbers: agent p95 4.2s → 1.8s, cost per run $0.90 → $0.12, tool-call error rate 6% → 0.8%, zero runaway loops since hard caps shipped.
The architecture these 50 questions live in
User (React dashboard)
|
AI GATEWAY (authn, model routing, budgets, audit)
|
AGENT RUNTIME (Python): plan -> act -> observe loop,
step/token caps, typed state + checkpoints, HITL interrupts
|-- tool calls (typed, validated, least-privilege) --
|
MCP SERVERS (C# .NET 9): analytics(read) reports(actions) admin(locked)
OAuth 2.1, per-tenant scopes, tenant bound from token
|
GOVERNED BUSINESS APIs -> deterministic execution, audit log
|
OUTPUT GUARDS -> grounded answer / queued action -> user
traces + evals on every run (run -> turn -> tool spans)
The one law that organizes everything: the model proposes; deterministic code executes; humans approve what matters.
Block A — agent foundations
What is an agent — and what is it not? An LLM in a loop with tools and state: it decides the next step, acts through a tool, observes, repeats until done or capped. Not an agent: a single call, a fixed chain, or RPA. Define it by who owns control flow — the LLM, not you — and use the least autonomy that solves the problem.
When is an agent the wrong choice? Whenever the path is knowable in advance: deterministic transforms, fixed pipelines, single judgments, latency-critical paths, or irreversible actions without a human gate. The rarest senior signal in this whole topic is a story about not building an agent because a SQL query + one LLM call solved the ticket.
The production agent loop is the same plan-act-observe everyone writes — plus hard step/token/time caps, typed state, validated tool calls, checkpointing, and an explicit "couldn't finish" path. The caps and the honesty are the production part.
Plan-and-execute vs reactive (ReAct): reactive for open-ended exploration; plan-and-execute for multi-step tasks with a knowable shape (up-front structure cuts cost and wandering).
Durable execution / checkpointing: a long agent run must survive a restart. Persist typed state per step so the run resumes instead of starting over — essential once human approvals span hours or days.
Block B — tool design
What makes a good tool? "Intents, not endpoints." ~10 task-shaped tools beat 40 CRUD endpoints — too many tools confuse the model and bloat the prompt. A production tool is a typed function + a clear schema + strict validation + errors that teach the model how to recover.
The description IS the prompt. The model picks tools by their docstrings — write them like instructions, not code comments. A vague description is a wrong-tool bug waiting to happen.
Side-effect annotations (read-only / destructive / idempotent) let the runtime gate behavior: auto-run reads, require approval for destructive, make retries safe for idempotent.
Who authorizes the tool call? Not the agent. A valid token is authentication, not authorization. Every action is checked against the user's actual permissions, deterministically, server-side. The tenant is bound from the token — never a model argument.
Least privilege for agents: the agent gets exactly the tools this task needs. An analytics agent has no delete tool; it can't misuse what it was never given.
Block C — MCP (Model Context Protocol)
What problem does MCP solve? It standardizes how agents discover and call governed tools across hosts, clients, and servers — so you stop hand-wiring every integration. Three primitives: tools (actions), resources (data), prompts (templates).
MCP vs plain function calling: the protocol adds discovery, a transport, auth, and a server boundary you can govern and operate independently — not just a JSON schema.
Auth for a production MCP server: OAuth 2.1, per-tenant scopes, the token carries identity, and the server binds the tenant from the token. A valid token still isn't authorization (see Block B) — the server enforces what this identity may do.
MCP-specific attacks: tool poisoning (malicious tool descriptions), confused-deputy, and indirect injection via tool results. Defenses: pin/verify tool definitions, least privilege, and treat every tool result as untrusted data.
Block D — memory & context
What memory does an agent need? Working memory (the scratchpad for this run), episodic/long-term (facts across sessions, often a vector store), and durable state (checkpoints). Each lives in a different place; don't conflate them.
Context budget mid-loop: long runs flood the window. Summarize or trim old steps, keep the goal and rules pinned (never trimmed), and watch the token budget like cost — because it is.
Agentic retrieval vs pipeline RAG: the agent decides when and what to retrieve as a tool call, mid-reasoning — vs a fixed retrieve→generate pipeline. More flexible, but you must bound the retrieval loop.
Context poisoning: a bad tool result or injected instruction corrupts the running context and every subsequent step. Contain it by fencing tool outputs as data and validating what re-enters the loop.
Block E — multi-agent orchestration
When does multi-agent beat a single agent? Only when the task genuinely exceeds one agent's coherence — decompose by capability, like microservices. It adds coordination cost and new failure modes, so reach for it late.
Orchestrator-worker is the production default: a planner routes sub-tasks to specialist workers and combines results. What travels in a handoff is a structured task + just-enough context — and what gets lost is the nuance you didn't serialize.
Multi-agent failure modes single agents don't have: routing to the wrong specialist, context loss across handoffs, cascading errors, and shared-state races. Assign models by role (small model routes, frontier reasons).
Block F — guardrails, evals & ops
The guardrail checklist: step/token/time budgets, tool allowlist/least privilege, human-in-the-loop for destructive actions, input + output validation, tool-result-as-untrusted, timeouts/fallbacks, and full tracing.
Prompt injection against agents is worse than against chatbots — an injected instruction can trigger a real action, not just a bad sentence. The architecture answer: the model proposes, deterministic authorized code executes, humans approve what matters.
Observability: every run is one trace of run → turn → tool spans, each with cost, latency, args, and result — so "the agent felt off" becomes a debuggable timeline.
Evaluating agents is harder than evaluating answers — you grade the trajectory (did it take sane steps, call the right tools, stay in budget), not just the final output. Golden runs + LLM-as-judge on the trace + a CI gate.
Idempotency & retry-safety: every destructive action carries an idempotency key so a retry after an ambiguous failure can't double-charge or double-create.
Cost levers in order: fewer steps (better planning/caps) → cheaper model per role → caching → shorter context. $0.90 → $0.12 per run came from structure, not a smaller model.
The through-line across all 50: an agent is an action system, not a chat system. The engineering is the loop caps, the typed tools, the authorization boundary, the human gate, and the trace on every run — not a cleverer prompt. The model proposes; deterministic code executes; humans approve what matters.
This post was originally published on PrepStack. The full original has all 50 with complete production code.
Top comments (0)