DEV Community

Bhavik Shah
Bhavik Shah

Posted on Originally published at Medium on

Agent Architecture vs. Agentic Platform Architecture: Why Your Second Agent Is a Different Problem

#machinelearning #architecture #ai #agenticplatform

Building one agent is the easy part. The real architecture questions show up when you try to run several.

Every few days a recruiter or hiring manager reaches out about an “AI Architect” role, and the goal is usually some version of the same idea:

“We want to build agents to automate different parts of the business — support triage, supply-chain tasks, code migrations, some finance ops…”

It’s a reasonable goal, and the enthusiasm is real. But when the conversation reaches the operational questions — how will agents authenticate to each other? who caps token spend when something misbehaves? how do events move between agents? where does a human approve the risky actions? — it usually becomes clear those parts haven’t been scoped yet.

That’s not a knock on anyone. The distinction at play is genuinely easy to miss — I’ve watched experienced teams miss it, and I’ve gotten it wrong myself. Agent architecture (how a single autonomous entity thinks and acts) and agentic platform architecture (the infrastructure that lets several of them run reliably) are different problems that happen to share a word. Build a handful of agents without the second one and things tend to work fine in the demo — then get fragile and expensive in production.

This post walks through both: what’s inside each, where each shines and hurts, real platform examples (AWS Bedrock AgentCore and its peers), the failure modes that show up in production, and the practices that help avoid them.

The core distinction: the car and the highway system

An agent architecture describes one autonomous worker from the inside: how it reasons, what it remembers, which tools it can reach, and what limits bind it. It is the design of a high-performance car.

An agentic platform architecture is everything that lets hundreds of those cars share the road safely: the road network, the traffic rules, the fueling stops, and the licensing office. It treats each agent as a small, replaceable service and carries — as shared infrastructure — everything the agents shouldn’t have to.

One answers “how does this specific agent accomplish its task?” The other answers “how does an organization run, watch, and govern a growing fleet of agents that different teams keep shipping?”


Figure 1 — The same building blocks, coupled inside one app vs. provided as shared, governed platform services.

Part 1: Inside a single agent

Strip away the hype and a production-grade agent is five parts working together:

1. The model — the reasoning core. The engine that decides what to do next, sequences the steps, and recovers when a step goes wrong. Tune it for task accuracy before you worry about cost or latency.

2. Instructions — the operating contract. Your team’s procedures translated into the prompt: a tightly scoped brief, worked examples, the edge cases spelled out, and what the agent should do when each thing fails.

3. The reasoning loop — the cognitive pattern. The agent doesn’t just answer; it plans, acts, observes the result, and reflects, looping until the task is done. This is where patterns like ReAct (Reason + Act), Plan-and-Solve, and self-correction live. A useful test for whether you’ve built an agent at all: step away from the keyboard. If the task keeps advancing without you, it’s an agent; if it stalls waiting for your next message, it’s a chatbot.

4. Memory — and it’s three things, not one. Short-term working memory holds the current task state and recent steps. Long-term / episodic memory persists past interactions, user preferences, and previously successful runs. Semantic memory is your enterprise knowledge, chunked into a vector store (pgvector, Qdrant, Pinecone) and retrieved via RAG. Skip the distinction and your agent is brilliant for one turn and amnesiac the next.

5. Tools — the hands. Capabilities exposed via strict schemas: a web-search tool, a code executor, database queries, an email/Slack API, a browser, a CRM updater. Increasingly these are wired through the Model Context Protocol (MCP), the emerging standard for describing tools so any agent can discover and call them. Alongside tools sit skills  — packaged, reusable instructions the agent loads on demand, like a report-writer, a code-reviewer, or an invoice-parser.

You rarely build all of this from scratch. Agent frameworks  — LangGraph (graph-based, stateful orchestration), CrewAI (role-based multi-agent teams), Microsoft’s AutoGen, AWS Strands Agents, LlamaIndex — give you the loop, tool bindings, and memory hooks as libraries. But note what they are: authoring tools. They help you write the agent. They do not run, govern, or scale it — that is the platform’s job, and conflating the two is where teams get burned.

Advantages of the single-agent approach

  • Speed to value. You can ship something genuinely useful in days, not quarters.
  • Full control. Every prompt, retry, and tool binding is yours to tune for one specific job.
  • Low ceremony. No message buses, no multi-tenant identity, no platform team required.
  • Cheap to run — at first. One process, one model bill, one deploy pipeline.

Disadvantages

  • Nothing is reusable. The loop, the logging, the tool wiring — all rebuilt for the next agent.
  • Observability is whatever you remembered to log. Debugging means grepping print statements and guessing what the model did.
  • Scaling is vertical. A traffic spike means a bigger box; one slow request stalls the rest.
  • Governance by good intentions. Keys in the codebase, access rules in the prompt, budgets in someone’s head.

Where a single agent is genuinely the right call

  • Prototypes and proofs-of-concept where the question is “does this work at all?”
  • A single, well-scoped internal task: triaging one inbox, summarizing one report type, migrating one codebase.
  • Teams of one or two engineers with no platform organization behind them.

Structural challenges at the agent level

  • Loop runaway. The most common way agents fail in production: a tool errors or returns something ambiguous, the agent retries, the retry fails the same way, and the cycle spins all night on your token bill. The runtime — not the prompt — must enforce a ceiling on steps per task and a hard spend limit per request.
  • Context-window decay. On long runs, accumulated reasoning traces crowd the window and answer quality drifts downward. You need compaction — summarize and prune as you go — to hold quality steady.
  • Brittle structured extraction. Relying on the LLM to format complex API arguments works until a model upgrade subtly changes its output habits. Validate every tool call against its schema; never trust, always parse.

The Monolithic Agent Trap

Here’s how the single-agent approach fails in slow motion. You build agent #2 and copy the loop. Agent #3, you copy it again — but tweak the retry logic, so now they’ve quietly drifted apart. By agent #10, infrastructure concerns — connection pools, secret keys, homegrown logging, a hard-pinned model endpoint, security rules — are baked into every agent’s code, each slightly differently.

Then the company switches model providers, or legal updates one compliance rule — and the change fans out into a hand-edit of every agent you own. Nothing stops the marketing team’s agent from calling an internal finance API — no matter what its prompt says — because access control was never a system property, only a prompt suggestion.

The trap in one sentence: infrastructure baked into agent code works for one agent, and fails combinatorially for twenty.

None of these are agent problems. They’re platform problems — and you’ve been solving them one agent at a time, which doesn’t compose.


Figure 2 — Layer by layer: the pain with one agent, and how a platform removes it.

Part 2: The agentic platform — infrastructure for a fleet

An agentic platform is the shared runtime an enterprise puts underneath all of its agents — hosting them, brokering their communication, auditing their actions, and enforcing its rules on every one. The agents become thin — mostly a prompt plus a policy — and everything hard moves down into shared services:

  • Unified front door & API gateway. Multi-tenant boundaries, functional routing, incoming webhooks and events. Departmental “front doors” (HR assistant, finance copilot, support agents) all land on the same platform.
  • Agent runtime / harness. One standardized, versioned reasoning scaffold — loop, retries, timeouts, context assembly, step caps — that every agent inherits instead of reinventing.
  • Orchestration & agent-to-agent communication. Managed handoffs between agents — supervisor, pipeline, or peer-to-peer — whether over an event bus (Kafka, Redis streams) or the emerging Agent-to-Agent (A2A) protocol, which standardizes how agents from different teams (or vendors) exchange structured work.
  • Data & knowledge layer. Centralized knowledge graphs, vector stores and semantic indexes, streaming data — shared grounding instead of per-agent silos.
  • Memory services. Short-term, long-term/episodic, and semantic memory offered as managed services with retention control.
  • Tool & API fabric. Tools registered once — increasingly via the Model Context Protocol (MCP) — permissioned centrally, reused by any agent that needs them.
  • Model gateway / inference router. Routes each sub-task to the right model (frontier model for hard reasoning, cheap fast model for parsing), with caching, rate-limiting, and automatic failover between providers. A provider outage becomes a config change, not a 2 a.m. refactor.
  • Governance, identity & guardrails. IAM/RBAC so agents have real identities and scoped permissions; centralized input/output guardrails; audit logs; enforced budgets.
  • Human-in-the-loop service. One approval engine for the whole fleet: when any agent reaches for a high-risk action — a refund, a production change, an outbound customer email — the platform freezes that run until a person approves, then resumes it.
  • Observability & evaluations. End-to-end tracing of every step, tool call, token and dollar (OpenTelemetry, LangSmith, Arize Phoenix), plus automated evals that catch quality regressions before they ship.
  • Lifecycle management. Provision, version, pause, and deprecate agents like any other production service, with CI/CD and staged rollouts.

A worked example: AWS Bedrock AgentCore

If the platform concept feels abstract, look at how AWS productized it. Amazon Bedrock AgentCore  — generally available since October 2025 — is essentially the platform column of this article sold as composable managed services. Its building blocks map almost one-to-one:

  • Runtime  — serverless, session-isolated execution with long-running windows (up to eight hours) and Agent-to-Agent (A2A) protocol support. Works with any framework (LangGraph, CrewAI, Strands, LlamaIndex) and any model, inside or outside Bedrock.
  • Gateway  — turns APIs and Lambda functions into agent-compatible tools and connects to existing MCP servers, acting as one secure endpoint where agents discover and use tools.
  • Memory  — managed session and long-term memory with pluggable extraction/consolidation strategies.
  • Identity  — agents get real identities, OAuth/IAM-based authorization, and secure token vaults so they can act on behalf of users with scoped access.
  • Policy  — centralized, natural-language or policy-as-code controls over what agents are permitted to do (reached GA in early 2026).
  • Observability & Evaluations  — step-by-step execution tracing plus built-in evaluators for response quality, safety, task completion, and tool usage.
  • A managed harness  — the newest piece (GA mid-2026): define an agent with one API call, invoke it with another, with the loop, tools, memory and tracing handled by the platform.

The strategic read: a hyperscaler looked at what every enterprise was hand-rolling — runtime, gateway, memory, identity, policy, observability — and shipped exactly that list. That’s strong independent confirmation of where the platform boundary sits.

Other platforms worth knowing

  • Google Vertex AI Agent Builder / Agent Engine  — Google Cloud’s managed agent infrastructure, with native RAG and A2A support; natural fit for BigQuery/Gemini-centric shops.
  • Azure AI Foundry Agent Service  — Microsoft’s developer-grade managed runtime with per-agent Entra identity, private networking, and multi-framework support (LangGraph, OpenAI Agents SDK, Claude Agent SDK).
  • LangGraph Platform  — the managed deployment layer for LangGraph agents, with state persisted at every execution step; maximum control over the execution graph, portable across clouds.
  • Salesforce Agentforce  — the CRM-native agentic platform: agents built directly on Salesforce data, workflows, and permissioning, aimed at sales and service automation for organizations already living in that ecosystem.
  • CrewAI Enterprise  — the managed tier of the fastest route to multi-agent prototypes, when role-based collaboration is your primary pattern.
  • Custom platforms  — asynchronous runtimes built on distributed task queues (Celery + Redis) over Kubernetes/EKS, for teams that need full control and accept full operational burden.

A pragmatic 2026 heuristic: if you’re committed to one cloud, use that hyperscaler’s platform for deployment — but keep your agent logic in portable framework code so the authoring layer isn’t locked in.

Frameworks vs. platforms, in one line: LangGraph and CrewAI help you write an agent; AgentCore, Vertex AI Agent Builder, and Agentforce help you run a fleet of them — with identity, guardrails, observability, and scale handled for you. Most production stacks use one of each.

Advantages of the platform approach

  • Write-once infrastructure. The loop, guardrails, tracing, and tool fabric are built once and inherited by every agent.
  • Governance as a system property. Access control, budgets, and approvals are enforced by the platform — not requested politely in a prompt.
  • Organizational leverage. Data scientists tune prompts and reasoning; the platform team owns scale, security, and cost. Different teams ship agents without re-solving infrastructure.
  • Resilience by default. Model failover, worker isolation, and staged rollouts mean one bad agent or one provider outage doesn’t take down the fleet.

Disadvantages

  • Upfront cost and ceremony. A platform is real engineering. Built too early, it’s expensive scaffolding around a product that doesn’t exist yet.
  • New failure surface. Message buses, identity systems, and gateways can themselves fail — and platform bugs affect every agent at once.
  • Potential lock-in. Managed platforms trade portability for convenience; choose where you accept that trade deliberately.
  • Organizational overhead. Someone has to own the platform — its roadmap, its SLAs, its upgrades.

Platform-level engineering challenges

  • Multi-tenant state management. Keeping every tenant’s data walled off from every other’s — while long-running agent sessions still reach their own state quickly.
  • Unsustainable scale costs. Token bills compound across a fleet. Survival requires caching (prompt and semantic) plus routing discipline: reserve frontier models for the reasoning that needs them, and push parsing and extraction to small, cheap ones.
  • Distributed auditing complexity. When a workflow fails three handoffs deep, per-agent logs won’t tell you where the payload went bad. You need traces that follow one request across every agent it touched.

The shield is deeper than it looks: four layers of defense

“We added guardrails” usually means one moderation call on the output. In production, protection works better as defense in depth — four layers, each guarding a different thing, with every request crossing all of them:

  1. Inbound defense — screen what reaches the model. Request and file validation, injection and jailbreak screening, isolation of system instructions, rate and size limits.
  2. Knowledge integrity — trust what the agent reasons over. Source vetting for anything RAG pulls in, grounding and freshness checks, PII redaction in retrieval, and memory hygiene: clear rules for what gets stored, recalled, and retained.
  3. Action control — bound what the agent can do. Least-privilege tool scopes, sandboxed execution, transaction and write ceilings, and human sign-off on high-risk calls.
  4. Runtime assurance — watch how the whole run behaves. Loop and anomaly detection, step caps and spend budgets, output policy validation, and an end-to-end audit trail. This layer never sleeps: it wraps every stage of every run.

In a single agent you hand-roll pieces of these layers and hope. On a platform all four live once, as shared, versioned policy every agent inherits — the difference between “we added some safety checks” and “safety is enforced by default, everywhere.”


Figure 3 — Defense in depth: a request crosses three gates to reach the agent core, inside an always-on assurance layer.

When agents team up: three topologies

Once a platform exists, agents start collaborating — and the shape of the collaboration is an architectural decision:

  • Supervisor (hierarchical). A lead agent breaks the goal apart, farms the pieces out to specialists, and assembles the final result. Suits complex, cross-functional work — a product-spec agent feeding a coding agent feeding a QA agent.
  • Pipeline (sequential). Work flows in one direction: each agent’s structured output becomes the next one’s input. Suits predictable, linear back-office flows — invoice intake, then audit, then entry.
  • Peer-to-peer (network). Autonomous peers coordinate directly, pulling each other in when they need help. Suits open-ended, dynamic problems like supply-chain negotiation.

Reach for multiple agents only when the work truly needs parallel effort or distinct specialisms. A single well-instrumented agent beats a poorly governed swarm every time.


Figure 4 — Supervisor, pipeline, and peer-to-peer: match the topology to the work.

Best practices — and the pitfalls they prevent

Best practiceThe pitfall it prevents Standardize the harness first. One versioned reasoning scaffold — loop, retries, context assembly — shared by all agents.Ten hand-rolled loops that quietly drift apart, each with different bugs. Hard step caps and per-request budgets, enforced by the runtime. Loop runaway — the #1 production failure — burning tokens against a failing tool all night. Trace everything from day one. Every step, tool call, token and dollar, with distributed tracing across agent handoffs.Debugging by grepping print statements; being unable to explain why Agent B received garbage from Agent A. Evals in CI, not vibes in prod. Golden test sets, behavioral assertions, expected tool sequences — run on every prompt or model change.Silent quality regressions shipped because “the demo still looked fine.” Route models through a gateway. Cheap models for parsing, frontier models for hard reasoning, automatic failover.Emergency refactors on a provider outage; a cost curve that kills the project at scale.Give agents real identities (IAM/RBAC), not prompt-based trust.The marketing agent calling the finance API because nothing but a sentence in a prompt said it couldn’t. Centralize human-in-the-loop as a service. One approval engine, risk thresholds defined per action class.High-risk mutations (refunds, prod writes, customer emails) executing with no human gate — or every team building its own inconsistent one.Register tools once (MCP), permission them centrally.Bespoke integrations and API keys duplicated across codebases. Design memory in three tiers (short-term, episodic, semantic) with retention rules.Amnesiac agents, unbounded context windows, and sensitive data retained forever. Start simple; extract the platform when you feel the pain twice. Build agent #1 as an app. When agent #2 needs the same plumbing, that’s the platform’s birthday.Both failure modes at once: premature platform ceremony, or the Monolithic Agent Trap.

So which do you need?

If you’re shipping your first agent, or a single well-scoped one — build the agent. A platform you don’t need yet is expensive ceremony.

But if your roadmap has ten agents on it, then building each as a standalone app means maintaining ten reasoning loops, ten logging schemes, ten copies of your security rules, and zero shared visibility. At that point the platform isn’t over-engineering. It’s the thing that keeps the fleet debuggable, affordable, and safe.

In an agent architecture, the cross-cutting concerns — the harness, observability, scalability, governance — are plumbing you hide inside the app. In a platform architecture, they are the product.

Build the agent first. Just know the second one is a different kind of problem — and design for it before it designs for you.

Are you building a multi-agent system right now? I’d love to hear how you’re handling agent runtime isolation, global guardrails, and cost control in your stack — drop a comment.

References & Further Reading

This article builds on established research and tooling from the AI engineering community. Key references:

Research & Patterns

Frameworks & Standards

Platforms & Services

This article reflects current best practices in agentic AI architecture as of 2026. The field evolves rapidly; for the latest on frameworks, standards, and platforms, refer to their official documentation.

Top comments (0)