DEV Community: Bhavik Shah

From MLOps to AgentOps: Redrawing the Three Lines of Defense

Bhavik Shah — Thu, 10 Sep 2026 18:11:12 +0000

In my last post I made the broader case that agentic AI breaks every governance playbook built for traditional software and traditional models. That argument was deliberately wide. This one is narrow on purpose.

I have spent years building and reviewing MLOps systems, including model inventories, validation pipelines, drift monitoring, and approval workflows across enterprise and financial services environments. None of that gets thrown out when an organization moves to LLMs and agents. What changes is more specific than most people assume, and that’s what this post is about.

Picture a model risk team getting a routine intake request. Their form asks the standard questions: what data was this trained on, what’s the version number, what’s the approval status, when was it last checked for drift. They’ve processed multiple of these. Then they read the description of the system in front of them: four agents, six tools between them, a retrieval layer that refreshes daily, and a decision that comes out the other end of a chain where no single step was wrong on its own. The form has no field for that. McKinsey’s 2026 AI Trust survey of roughly five hundred organizations found something consistent with what that team is about to discover: only about a third of enterprises report governance maturity levels adequate for the autonomous agents they are already running. Most of the rest are filling out the same intake form and hoping it’s close enough.

It usually isn’t. Here’s specifically where it stops being close enough, and what closing that gap actually requires.

What MLOps governance is, and how it fits into MLOps

Start with what the intake form already gets right.

MLOps governance is the set of controls that make sure a traditional machine learning model stays trustworthy from training through retirement. In practice that means model versioning so you always know which model is in production and why, drift monitoring that watches for performance degradation as input data shifts, data lineage back to what a model was trained and validated on, approval gates before anything reaches production, and the CI/CD and change management discipline any production software system needs. Software governance sits inside this, not off to the side as a separate track. None of these controls enforce themselves, though. Someone has to own each one, and someone else has to check that the owner actually did the job.

It helps to be precise about who is responsible for each of these controls, and the model I use is the three lines of defense, a framework banking and other regulated industries have used for years to assign accountability. One accuracy note: the Institute of Internal Auditors renamed this the Three Lines Model in 2020, shifting the emphasis toward value creation rather than pure defense. The older term has stuck in finance and model-risk circles regardless, so I use it here, but it is worth knowing the name has technically moved on even if the industry has not.

Applied to a traditional MLOps program, the first line is product, data, and engineering: they own model development, testing, deployment, and documentation. The second line is compliance and risk: they set model use policy, monitor for bias, and review high-risk applications before anything ships. The third line is internal audit, and their job is narrower than either of the other two: independently verify that the first two lines are doing what they claim, not build or approve anything themselves. This structure is mature, well understood, and it works, for the systems it was designed around: models with a fixed structure that gets validated once and monitored between review cycles.

What AgentOps governance is, and how it fits into AgentOps

AgentOps governance starts from the same three-line structure as MLOps, but the substance each line has to check is different, because the systems themselves behave differently.

A traditional model is one artifact. The four-agent system has the underlying model, four system prompts, six tool definitions, and a retrieval layer that updates on its own schedule. The first line now has to version and test all of that, not just the model. In practice this looks less like a model registry entry and more like treating prompts and tool schemas as code: stored in version control, reviewed through pull requests, and run through canary tests against a holdout set before a change is promoted to production, the same discipline used for any other production software change. Version the model alone in a system like this and you have covered maybe a quarter of what actually determines behavior.

A traditional model is deterministic given the same input, which is what makes validation straightforward. An LLM can give two different, both-defensible answers to the same prompt on two different runs. The second line’s review process has to shift from checking whether an output is correct to checking whether it falls within an acceptable range most of the time. Concretely, that means an evaluation gate with explicit thresholds, for example requiring a defined pass rate against a labeled test set before promotion, plus a trigger for human review whenever an agent’s output confidence or evaluation score drops below a set floor. Writing that threshold, and getting compliance to sign off on what “acceptable most of the time” means for a specific use case, is its own project. It is not a policy update you make in an afternoon.

A single model making a single prediction has a clear owner. The four-agent system does not. When the outcome is wrong and no individual step was wrong on its own, tracing which step actually caused it requires a decision log that records not just the final output but the intermediate handoffs: what each agent received as input, what it decided, and what it passed to the next agent in the chain. Without that log, “which of the four agents is accountable” is a question nobody can actually answer, and the first line and second line end up arguing about it after the fact instead of before deployment.

The third line’s job changes too. Drift monitoring watches whether predictions are degrading against a known baseline. It does not watch what an agent actually does while it runs: which tools it called, what actions it took, whether it stayed inside its intended scope. A newer category of agent observability platforms has emerged specifically for this: they capture the full execution trace of an agent run, not just the final output, including which tools were called, what parameters were passed, and how one agent’s output became another agent’s input. Many build on open instrumentation standards like OpenTelemetry so traces stay portable across tools rather than locked into one vendor’s format. For our four-agent system, that trace is the only artifact that would actually let an auditor reconstruct what happened after the fact. Without it, the third line is auditing a black box and calling it independent verification.

Last, the retraining cadence itself breaks. Traditional models retrain on a calendar: monthly, quarterly, whenever there is enough new data to justify it. A system built on RAG or continuous fine-tuning can shift its effective behavior as often as the knowledge base underneath it updates, which can be daily. If the four-agent system’s retrieval layer refreshes every night, a third line built around a quarterly review cadence is checking the system roughly ninety times less often than the system is actually changing.

The comparison, the gap, and what closes it

Laid side by side, the two programs share a spine and diverge on substance. Both assign the same three lines. Both need versioning, testing, review, and independent audit. What differs is granularity, frequency, and what each control actually has to look at.

The gap is not that MLOps governance is wrong for the four-agent system. It is that MLOps governance answers a set of questions that assumed a model’s behavior stays fixed between review cycles, and a system with a nightly-refreshing retrieval layer does not hold still long enough for that assumption to work. Every one of the five deltas above traces back to that single broken assumption: a model that could change monthly is now a system that can change behavior daily, sometimes without anyone changing the model itself.

Closing the gap does not mean building an entirely new governance function from scratch. It means extending the existing three lines with capabilities they did not previously need, and being honest about what each one costs. The first line needs prompt and tool version control with canary testing before promotion, which is mostly an engineering practice change, not a new team. The second line needs an evaluation gate with explicit acceptance thresholds and a documented escalation path for chained decisions, which is a genuine policy-writing project that will take real negotiation with legal and compliance the first time through. The third line needs runtime observability tooling plus the skill set to audit a decision log rather than a static model card, and for most model risk functions that is a new hire or a real training investment, not a checkbox added to an existing review template. None of these are optional if the goal is to actually answer, credibly, which of the four agents made the call.

Where the regulators stand

No single regulatory framework has fully caught up to agentic AI yet, but three existing models, from three different industries, are worth watching for where this is heading.

The EU AI Act takes a horizontal approach that applies across sectors. Its Article 50 transparency obligations take effect August 2, 2026 regardless of other developments. Its heavier Annex III high-risk obligations were originally set for the same date, but a proposed Digital Omnibus package, agreed in principle in May 2026 but not yet formally adopted, would push those specifically to December 2027. Worth treating as pending rather than settled if you are building a compliance calendar around it.

The FDA’s approach to AI-enabled medical devices is a genuinely different model: vertical, product-based, and built around a mechanism called a Predetermined Change Control Plan. A PCCP lets a manufacturer pre-authorize specific future model changes within defined performance bounds as part of the original approval, paired with Good Machine Learning Practice guidance and quality system alignment toward ISO 13485. It is essentially a change-control gate built directly into the regulatory pathway rather than a compliance requirement layered on after the fact.

Finance offers a different kind of case, partly because of what it does not cover. I have written elsewhere in this series about SR 26–2, the interagency guidance that replaced SR 11–7 in 2026, and its explicit exclusion of generative and agentic AI from formal scope. That gap is itself instructive. A mature, well-established regulatory framework is acknowledging that the tools it built for one generation of models do not automatically extend to the next. Enterprises operating under it are, in practice, already building governance for agentic systems ahead of formal supervisory guidance catching up to them.

None of these three is a template to copy wholesale. Together they point at where things are heading: pre-authorized change boundaries, tiered risk classification, and an acknowledged gap between existing rules and what agentic systems actually do.

A checklist for both generations

Back to the model risk team. They don’t need to throw out their intake form. They need a second one for the multi-agent system, mapped against the same three lines they already trust, and they need to accept that filling it out honestly is real work, not a formality. If you are running LLM or agent-based systems through an existing model risk function, I am curious which step in this roadmap your governance program has actually caught up to first. My guess is versioning gets solved early, because it looks like a version control problem engineers already know how to solve. Accountability for chained decisions is the one I would bet stays unresolved the longest, because it is an org design problem wearing a technical costume, and no evaluation gate fixes an org design problem.

References

McKinsey & Company. State of AI Trust in 2026: Shifting to the Agentic Era. AI Trust Maturity Survey of approximately 500 organizations, 2026.
European Commission. Timeline for the Implementation of the EU AI Act. AI Act Service Desk.
EU AI Act Update: Timeline Relief, Targeted Simplification, and New Prohibitions. Inside Global Tech, May 28, 2026.

4. IntuitionLabs. FDA AI/ML SaMD Guidance: Complete 2026 Compliance Guide.

5. Institute of Internal Auditors, Three Lines Model (2020 revision).

The Data Layer Behind the Agent

Bhavik Shah — Fri, 21 Aug 2026 02:36:45 +0000

Data usually moves through a few stages before anything gets to reason over it. It starts in transaction systems, servicing platforms, SaaS tools, and document repositories. From there it lands in a lake, typically in three stages: bronze, silver, and gold. Bronze is raw data, barely touched. Silver is cleaned up and standardized. Gold is the finished version, ready to use.

At every one of those stages, someone is making governance decisions. Who’s allowed to see this. How long it gets retained. Whether it needs to be masked before it moves forward. Whether it’s even trustworthy enough to promote to the next layer. That work is supposed to happen quietly in the background, so that by the time data reaches gold, an analyst building a report doesn’t have to think about any of it.

Agents are starting to sit right where that analyst used to sit, sometimes at the gold layer, sometimes reaching further upstream than that. And almost none of the governance built into that pipeline was designed with an autonomous actor on the other end. It assumed a person, a query, and a dashboard. Not something that reads the data and then goes and does something with it.

Most of what’s being written about agentic AI right now is orchestration frameworks, memory, tool-calling, multi-agent patterns. Almost none of it asks the more basic question: can the agent actually trust the data it’s reaching into? If the answer is no, none of the orchestration sophistication matters. You’ve just built something that fails faster and with more confidence.

Why this is a different problem than “AI-ready data”

Enterprises have been chasing AI-ready data for years now. Clean it, catalog it, govern it, make it available for models to train on or reason over. That work matters, but it was built around a specific assumption: a human or a model is going to read this data and produce an answer.

Agents break that assumption. An agent doesn’t just read data and summarize it. It reads data and then acts on it, approves something, routes something, updates a record, kicks off a downstream process, often without a human checking its work first. That changes what “ready” actually means.

A stale customer record is a minor annoyance in a dashboard. It’s a real problem when an agent acts on it. A dataset with unclear lineage is fine for a BI report nobody’s betting money on. It’s not fine when an agent’s decision needs to be explained to a regulator six months later, and nobody can say for certain where the underlying number came from.

So the bar for “AI-ready” quietly moves. It’s not just about quality and structure anymore. It’s about whether the data can support an autonomous action, and whether you can reconstruct what the agent saw and why afterward.

A walkthrough: the hardship review agent

Let’s take an example. Picture an agent built to triage loan hardship requests, something a servicer might use to decide whether a request gets fast-tracked for a forbearance plan or routed to a person for review.

To do this well, the agent needs to reason across three different kinds of data, and of the three, only the structured data actually made the bronze to silver to gold journey I described above in any meaningful way.

The structured layer is the loan servicing system: payment history, current balance, delinquency status. Clean rows, clear schema, row-level security and audit logs behind it in most shops. This is the layer governance programs are genuinely built for.

The semi-structured layer is the hardship application itself, usually a form submission or an income verification feed. JSON or something close to it, with fields that aren’t always filled in the same way twice (income entered as “$4,500" in one submission and “4500.00" in another). Manageable if the schema is documented and versioned, though in practice it usually stops between bronze and silver, since reaching gold takes modeling investment that operational data like this rarely gets ahead of an actual consumer asking for it. Nothing about JSON prevents it from being modeled all the way to gold. It’s a backlog problem, not a technical ceiling, and an agent that needs to reason over this data is exactly the kind of consumer that finally justifies doing that modeling work.

The unstructured layer is everything else. The hardship letter the borrower actually wrote. A pay stub uploaded as a PDF. Maybe a transcript from a call with a servicing rep last month. This layer typically bypasses the medallion pipeline entirely. It goes from a document repository straight into a vector index, because that’s the path that makes it retrievable, and retrievability is what the agent project needed. That bypass is where the trouble starts, and it’s worth being precise about why.

What actually happens to access control when you embed a document

When a document gets chunked and embedded, you’re creating a derived artifact that lives in a different system, under a different storage model, with a different access path than the source ever had. Whatever ACL sat on that pay stub in the document repository doesn’t follow it into the index unless someone deliberately carried it across.

There are roughly four ways teams handle this, and they fail differently.

None of these four are wrong exactly. The point is that “we handle access control on the vector store” is not a complete answer, and the follow-up question is which of these you’re doing and where the enforcement actually sits.

Then there’s re-embedding. Documents get updated, chunking strategies get tuned, embedding models get swapped for better ones, and each event regenerates the derived artifacts. Access metadata has to survive that regeneration. I’ve seen setups where the original ingestion carefully propagated ACLs, and the re-indexing job, written six months later by someone else, didn’t.

Scoping access when the consumer has a task instead of a job

Most data access models were built around people and their job functions. An analyst in finance sees finance data, and that grant is durable, tied to identity, reviewed on some quarterly cadence.

An agent doesn’t have a job function in that sense. It has a task. Back to the hardship agent: on one particular run, it needs this borrower’s payment history, this borrower’s application, and this borrower’s uploaded documents. It does not need every borrower’s records, which is what it almost certainly has, because the service account it runs under was provisioned against the servicing domain rather than the invocation.

That gap between what the task requires and what the identity permits is where most of the real exposure sits, and it’s not obvious how to close with the tools most enterprises have.

The direction I think this has to go is attribute-based access control evaluated per invocation rather than per session: the agent doesn’t hold a standing grant to the servicing domain, it presents a token scoped to loan number X for the duration of this task, and the data layer enforces that scope on every read regardless of which system the read lands in.

That’s clean to describe and hard to build, for three reasons.

I don’t think anyone has fully solved this. But I’d rather see a team say “we know our agent’s identity is over-provisioned relative to its task, here’s our compensating control” than assume the service account model covers it.

Lineage that has to survive a question six months later

Lineage has always been a governance artifact. Someone produces a diagram, it goes in a catalog, it’s reviewed when an auditor asks. With agents it becomes something closer to a runtime obligation, because the question changes from “where does this table come from” to “what did this specific agent run see, and why did it act the way it did.” For the hardship agent, that’s the difference between a diagram of the servicing pipeline in general and being able to say exactly what it read before recommending one borrower’s forbearance decision.

Answering that means capturing, per invocation, the task context, the queries issued and against what, the specific chunks retrieved with source identifiers and versions, the tool calls made, and the decision produced. Not a summary. Durable references you can resolve later.

The structured side of this is tractable. Queries are loggable, tables are versioned, a warehouse can usually tell you what a query returned at a point in time.

The unstructured side is where it falls apart, and the reason is the re-embedding problem again. If you log chunk identifiers for that pay stub or hardship letter, and the corpus gets re-chunked eight months later, those identifiers point at nothing, or worse, at different text. Reconstructing what the agent actually read requires either immutable chunk storage with real versioning, or logging the retrieved content itself rather than pointers to it. The first is operationally heavier than most teams expect. The second creates a second copy of your sensitive data sitting in your audit store, which is its own governance problem.

That tension is real and I don’t think there’s a free answer. But it’s worth deciding deliberately rather than discovering it during an examination.

Where This Fits in Build, Scale, Govern

This is the same Build, Scale, Govern pattern, just one layer down from the agent. In Build, teams give agents whatever data access is easiest to set up. Getting it working matters more than getting the access right. In Scale, that shortcut starts to hurt. New agents copy the same broad access, and the gaps add up. In Govern, this is what gets checked. Not “is your data clean” but “did this system know only what it should have.”

The data layer doesn’t move through Build, Scale, Govern on its own schedule. It follows the agent layer, usually a step behind.

What I’d actually watch for

I don’t think the answer here is a new platform category, and I’d be skeptical of anyone selling one. Most of what’s needed is existing governance machinery applied at a granularity it wasn’t designed for.

The question worth asking isn’t lakehouse versus warehouse, or which vendor has the best vector search. Plenty of systems can log what an agent retrieved. Plenty can check permissions at the moment of retrieval. Far fewer can put those two things together into evidence that still holds up later, showing exactly what unstructured content was touched and under what access rules, months after the fact. That’s the real test, and for unstructured data, most platforms today aren’t built for it.

That gap gets more expensive as agents move from pilots into things that run unattended. It’s the layer I think deserves more attention than it’s getting. Not the agent. What’s underneath it.

References

Board of Governors of the Federal Reserve System, OCC, FDIC. SR 26–2, “Revised Guidance on Model Risk Management,” April 17, 2026. https://www.federalreserve.gov/supervisionreg/srletters/SR2602.htm (full text: https://www.federalreserve.gov/supervisionreg/srletters/SR2602a1.pdf)

NIST Special Publication 800–162, “Guide to Attribute Based Access Control (ABAC) Definition and Considerations.” https://csrc.nist.gov/pubs/sp/800/162/upd2/final

OWASP Cheat Sheet Series, “RAG Security Cheat Sheet.” https://cheatsheetseries.owasp.org/cheatsheets/RAG\_Security\_Cheat\_Sheet.html

Databricks, “What is the medallion lakehouse architecture?” https://docs.databricks.com/aws/en/lakehouse/medallion

The Govern Phase: AI Agents Break Every Governance Playbook

Bhavik Shah — Mon, 03 Aug 2026 05:04:40 +0000

In my last post I argued that agentic AI is following the same Build, Scale, Govern arc that cloud and microservices followed, just compressed. I ended it with a guess: that the governance layer for agents, the service mesh moment, would arrive faster than most roadmaps assume.

So this post is the follow-through. What actually makes governing autonomous agents different, what the most regulated industries are doing about it, what the platforms have started to solve, and what is still nobody’s solved problem.

— -

Why the old playbook doesn’t stretch

We already know how to govern software. I spent years of my career doing it: API gateways, IAM policies, distributed tracing, cost allocation tags. So the tempting assumption is that agents are just another workload that plugs into the same machinery.

They are not, and the reason is simple. Everything in that machinery assumes request-response systems with deterministic behavior. You send a request, the service runs fixed logic, you get a response. With an agent, you hand over a goal and the system works out the steps itself at runtime. That one difference breaks the playbook in four places.

Observability: Tracing an HTTP call tells you what happened. With an agent, you also need to know why. Why did it pick tool A over tool B? What context did it have? What did it ignore? Traditional APM (application performance monitoring — the tooling we use to watch services in production) has no concept of reasoning. A trace of an agent’s execution is not an explanation of its decision.

Cost: A buggy microservice retries and fails. A buggy agent loops, spawns sub-agents, and burns tokens with no natural ceiling. In my last post I told the story of the finance review nobody could answer: why did this month’s bill jump forty percent. With agents, that question can arrive after a single bad afternoon. Cost used to be an infrastructure line item. Now it is a runtime behavior that needs runtime controls.

Identity and access: An agent acts on behalf of a user but runs with its own identity. Classic IAM was never built to express “this agent may call this tool, with these parameters, only while acting for this user.” Agents also bring an attack surface services never had: prompt injection, where malicious content in the agent’s inputs becomes an instruction. Your agent’s context window is now part of your security perimeter.

Audit: When a deterministic system makes a decision, you point at the code. When an agent makes a decision, the honest answer involves a probability distribution. There is no shared definition yet of what an audit trail means for a system whose behavior is not fixed between reviews.

— -

What the regulators just told us

If anyone should have answers, it is the industries that have governed models for decades. What they are telling us right now is revealing, and a little uncomfortable.

Banking has had formal model risk management since 2011, when US regulators issued SR 11–7. An entire discipline grew around it: model inventories, independent validation teams, model risk committees. In April 2026, the Federal Reserve, OCC, and FDIC replaced it with SR 26–2 , the first major revision in about fifteen years. The revision explicitly scopes out generative and agentic AI, calling these technologies novel and rapidly evolving.

Read that carefully. The most mature model governance regime in the world just told banks that the systems they are actively deploying sit outside the framework. According to NVIDIA’s 2026 State of AI in Financial Services report, based on a survey of more than 800 industry professionals, 42% of financial firms are already using or assessing agentic AI, and 21% have already deployed agents. Agent-specific guidance is signaled but not yet issued. Until it arrives, the deployment curve is ahead of the governance curve.

The reason for the exclusion matters, because it confirms the argument above. Traditional validation assumes a model’s structure and behavior stay stable between review cycles. You validate it once, monitor it, document it. An agent that adapts its own steps at runtime breaks the assumption underneath all three pillars. The principles are still sound. The methods behind them no longer fit.

Pharma tells a similar story from a different angle. There is no single AI rulebook in life sciences. Validation teams piece together governance from a patchwork of frameworks: FDA’s Computer Software Assurance guidance, a 2025 draft guidance on AI credibility, ICH quality risk management, GAMP AI guidance, and voluntary standards like ISO 42001. The FDA and EMA jointly issued ten good AI practice principles in early 2026, and EU AI Act high-risk obligations are phasing in this year.

The FDA’s underlying position is the one worth holding onto: AI may inform the work, but it does not own the accountability. A human must remain responsible for the decision and be able to explain why it was appropriate. No dashboard satisfies that requirement on its own.

Two heavily regulated industries, one conclusion. Existing frameworks assume systems whose behavior can be pinned down at validation time. Agents cannot be pinned down that way. The frameworks are being rewritten in real time, and enterprises deploying agents today are writing their own interim rules. Having watched enterprises do exactly that during early cloud adoption, I can tell you how it tends to go: unevenly.

— -

Where the platforms are converging

While regulation catches up, the platform layer is moving. A common architectural pattern is emerging across vendors, one I think will outlast any single product: enforce governance outside the agent, at a boundary the agent cannot reason around.

AWS’s Amazon Bedrock AgentCore is the most complete concrete example of the pattern today (AgentCore from here on). Its policy capability, generally available since March 2026, sits at a gateway between the agent and its tools. Every tool call crosses that boundary and gets checked against rules written in Cedar, a purpose-built policy language. The rules define which tools an agent may call, what parameters it may pass, and under what conditions.

The design choice that matters most: the checks run at the gateway layer, outside the agent’s code. The agent never sees the policy, so it cannot work around it. AWS describes this in its own documentation as probabilistic detection with deterministic enforcement — the model’s judgment can be uncertain, but the final allow-or-deny decision is not. Whatever you think of the product, that separation is the clearest governance principle for agents I have come across.

The same boundary now also hosts guardrails. As of mid-2026, AgentCore can evaluate tool call inputs and agent outputs in real time for prompt injection, sensitive data, and harmful content, and block them before they reach downstream systems. Telemetry comes out in OpenTelemetry-compatible format, with per-agent logs that keep traces, prompts, and policy decisions together — what an auditor would actually ask for. Policies can also run in log-only mode before moving to active enforcement. Anyone who has rolled out service mesh policies will recognise the pattern: dry run first, enforce second.

AWS is not alone in this. The big clouds are converging on the same boundary. Azure API Management now acts as an AI gateway that exposes APIs as MCP servers (MCP — the Model Context Protocol — is an open standard for connecting agents to tools), governs agent-to-agent traffic, and applies content safety checks at the gateway. Google’s Apigee has added MCP support and a central catalog for governed tools. Open-source and AI-native gateways like LiteLLM and Portkey put token budgets, guardrails, and MCP-level controls at a proxy in front of the model. Different products, same architectural bet. AgentCore is the worked example in this post because it currently expresses the pattern most fully, not because the pattern is AWS’s alone.

— -

What nobody has solved

The platform primitives are arriving. The governance program is not. Five gaps, in the order I would worry about them.

The boundary only governs what crosses it. Gateway-based policy applies to tool calls routed through the gateway. If an agent calls an SDK directly from its own code, that call never touches the policy layer. Skip the gateway and you skip the governance, silently. Keeping every tool call on the governed path is an engineering discipline problem, not a product feature. Service mesh taught this lesson the hard way: coverage gaps are where incidents happen.

Execution audit is not reasoning audit. You can now trace every step an agent took. Explaining to a regulator why it took those steps is a different problem, and nobody has solved it. Telemetry is evidence, not an explanation. The FDA’s accountability standard — a human who can explain why the decision was appropriate — is still met by humans, not by dashboards.

Cost governance is mostly reporting, not enforcement. Dashboards show token spend per agent. Hard budget ceilings, per-agent spend limits, automatic circuit breakers on runaway loops: teams are still building these themselves, the same way we all built cloud cost controls by hand after the first unexpected bill.

Cross-platform governance does not exist. AgentCore governs AgentCore. Azure governs Azure. Real enterprises run agents across clouds, internal frameworks, and vendor products, and there is no single governed layer across all of them. If the framework fragmentation in the Build phase was frustrating, the governance fragmentation in the Govern phase will be worse, because the stakes are higher.

The platform gives primitives, not a program. Policy engines and audit logs are necessary. They are not sufficient. You still need your own identity model, tool schemas, audit retention, incident response plan, and a named human who is accountable when an agent does something no policy anticipated.

— -

Where this leaves us

In the cloud era, real governance tooling arrived roughly a decade after adoption started. In the microservices era, closer to five years. With agents, serious governance primitives are shipping within two to three years of the technology going mainstream. The compression I wrote about last time is real.

But notice what compressed and what did not. The tooling compressed. The hard problems did not. Explainability, cross-platform control, cost enforcement, and regulatory clarity are all still open — and open at the exact moment banks and pharma companies are deploying agents into frameworks that just admitted they do not fully cover them.

If you are running agents in production, or planning to, I am curious which of these gaps bites your team first. My bet is on the gateway bypass problem, because it is the one that looks solved right up until the day it is not.

— -

References

1. Federal Reserve, OCC, FDIC. Revised interagency guidance on model risk management (SR 26–2 / OCC Bulletin 2026–13), April 17, 2026. Analysis: Databricks, Model risk management in 2026: A banker’s guide to the revised interagency guidance

2. Kapron, Z. The Governance Gap That Could Break Financial Markets. _Forbes, April 22, 2026.

Sharma, K._ SR 11–7 in the Age of Agentic AI: Where the Framework Holds and Where It Strains. GARP Risk Intelligence, February 27, 2026.
4. Governing Generative AI Across Financial Institutions: An SR 26–2-Compatible Framework_. arXiv preprint.
AWS._ Amazon Bedrock AgentCore FAQs. 6. AWS. Amazon Bedrock AgentCore now supports Bedrock Guardrails in policy_. AWS What’s New, June 17, 2026.
AWS._ Observe your agent applications on Amazon Bedrock AgentCore Observability. _AWS Documentation.
IntuitionLabs._ Best Agentic AI Platform for Pharma GxP Validation (2026)_.
IntuitionLabs._ Pharma AI Validation Packages for FDA and EMA Compliance_.
Xevalics._ FDA’s Position on AI in Pharmaceutical Quality Systems (2026)_.
Microsoft._ AI gateway capabilities in Azure API Management. _Microsoft Learn.
NVIDIA._ State of AI in Financial Services: 2026 Trends. Survey of 800+ industry professionals, published January 22, 2026.

How Cloud Adoption and Microservices Teach Us Where Agentic AI Is Headed

Bhavik Shah — Sat, 25 Jul 2026 13:01:02 +0000

How Cloud Adoption and Microservices Teach Us Where Agentic AI Is Headed

I published a piece last week https://medium.com/@bhavik123/agent-architecture-vs-agentic-platform-architecture-why-your-second-agent-is-a-different-problem-6ffc22de4849 arguing that your second agent is a different problem than your first — that scaling from one agent to a fleet of them changes the architecture question entirely. A reader read that piece and said something that’s stuck with me since::

*We’ve seen this before.*

They meant that cloud adoption went through this same mess years ago. So did microservices. And now agentic AI is going through it too — just faster.

I think they’re right, and I don’t say that from theory. I’ve lived through both of the earlier shifts myself, first as an engineer and later as an architect, across several different companies and industries. I remember when “should we move this to the cloud” turned into a year-long argument about which tools to standardize on. And I remember a similar argument happening again a few years later, when “let’s break up this monolith into microservices” led to the exact same kind of fight — just with new names attached: Kubernetes versus Docker Swarm versus Mesos, instead of AWS versus Azure versus Google Cloud.

Different technology, different decade, same underlying fight: *how do we run a lot of small, independent pieces without everything falling apart?*

So when people ask me where agentic AI is headed — the exact question my last post was circling — I’m not really guessing. It feels familiar, because I’ve watched this exact sequence play out twice already.

The pattern, in three steps

Here’s the pattern I keep seeing, as simply as I can put it:

**Step 1 — Everyone builds their own version.**

A new technology becomes good enough to use in real, production systems — not just demos. But there’s no standard way to run it yet. So every company builds its own version of the plumbing: how to monitor it, secure it, and keep it running. It’s wasteful, and everyone knows it’s wasteful, but there’s nothing better to use yet.

**Step 2 — A few tools compete to become “the standard.”**

Once a lot of companies are running these systems, a new question takes over: *how do we manage dozens or hundreds of these at once, reliably?* This is when two or three competing tools show up, each trying to become the default answer. Betting on the wrong one is a real risk — teams that pick a tool that later loses out often have to redo a lot of work.

**Step 3 — The “boring” but essential stuff shows up.**

Eventually, one tool wins (or a couple survive side by side), and attention shifts to the less exciting work: who’s allowed to do what, how much it costs, how to track what happened when something breaks. This is governance — unglamorous, but it’s the thing that decides whether a system survives an audit or a bad night.

I’ve watched this play out with cloud infrastructure. I’ve watched it play out with microservices. And right now, I’m watching the very early stage of the same thing happening with AI agents.

Cloud, from where I sat

AWS put the technology on the table in 2006. Almost nobody I worked with in enterprise IT took it seriously for years after that. It was a startup and engineering-playground thing — useful if you were building something new and had nothing to lose, not something a large enterprise staked a real migration on. The gap between “the technology exists” and “enterprises actually trust it” was itself close to five years.

That changed around 2011 and 2012, once Azure and Google Cloud gave enterprises more than one credible vendor to choose from and the conversation shifted from “should we consider this” to “which workloads, and how fast.” But here’s the part that matters: enterprises didn’t just adopt what startups had been using. The moment large companies committed real workloads, they came with demands startups never had to make — real access control, real cost accountability, real audit trails, compliance sign-off. A lot of what became “cloud governance” wasn’t the providers getting ahead of the problem. It was enterprises pushing for it once they were actually in, and providers building it out in response.

The automation problem showed up first. Each provider answered it in a way that only worked on their own platform — AWS with CloudFormation, Azure with Resource Manager, Google with Deployment Manager. As an architect, this is the decision that actually keeps you up at night, because it isn’t a technology choice, it’s a lock-in choice. I sat in the room when a company committed to one provider’s tooling, built two years of automation around it, and then a merger or a pricing renegotiation meant we needed to run workloads on a second cloud. None of that automation traveled. We rebuilt it. Terraform arrived later and solved exactly this — write it once, run it on whichever cloud you’re actually using — but for the companies that adopted early, that fix came a few years too late to save the rework.

Governance was the part nobody put on the roadmap, and it cost us for exactly that reason. Access control, cost accountability, audit trails — early on they got the bare minimum: an IAM policy someone set up once and never revisited. That worked fine when ten people had cloud access. It stopped working the moment a few hundred did, spread across a dozen business units, each one able to spin up resources nobody else could see. The turning point wasn’t a technology maturing — it was the first time a finance review asked “why did this month’s bill jump forty percent” and nobody could answer in less than a week. After that, access control, cost tracking, and audit logging stopped being something we’d get to eventually and became a requirement before anything went to production.

That’s why I think the cloud curve is genuinely the longest of the three. It’s not just adoption to maturity — it’s invention, then a multi-year trust gap before enterprises moved at all, then the scramble to build the governance layer enterprises then demanded on their way in. Startups didn’t need most of that. Enterprises insisted on it, and the providers spent years catching up to the requirement.

Looking back, that whole arc — from AWS putting the idea on the table in 2006 to governance being table stakes around 2022 — comes out to about sixteen years, by my own count.

Microservices, closer to home

Spring Boot and its peers matured around 2010 to 2015, and every team I knew with an aging monolith started cutting it apart. I was one of them. And the early period was rough in a very specific way — service discovery, distributed tracing, trying to figure out why request 4 of 12 in a chain failed at 2am with nothing but a log line and a hunch. We were all learning this at the same time, badly, in production.

The orchestration fight is the part I watched most closely, because I was in the room for it more than once. Mesos, Docker Swarm, Cloud Foundry — all of them were credible for a stretch. Kubernetes showed up in 2014 and by 2017 or so it had basically won, not because it launched first but because it stayed open and portable while everyone else’s roadmap got tangled up in vendor politics. I moved more than one platform onto it, and the teams that had already built some operational muscle during the chaos years adjusted faster than the teams that sat on the sidelines waiting for a winner.

Then came service mesh — tools like Istio and Linkerd, or the cloud-managed flavors of the same idea if you’d rather not run it yourself: AWS pushes you toward VPC Lattice or ECS Service Connect now, GCP folded its offering into Cloud Service Mesh, and Azure ships a managed Istio add-on for AKS. Whichever version you use, what it actually does is sit between all your services and quietly handle the stuff that never shows up in a demo: routing traffic to healthy instances instead of broken ones, encrypting traffic between services automatically, retrying a request that failed the first time, and cutting off calls to a service that’s clearly having a bad day so its problems don’t cascade into everyone else’s. None of that is glamorous. It’s also the reason a system stays up when something breaks at 4pm on a Friday instead of taking the whole platform down with it.

That cycle took maybe ten to twelve years. Faster than cloud.

And now, agents

You can probably see where this goes.

Back in 2023 and 2024, LangGraph, CrewAI, AutoGen and the rest matured to the point where teams stopped experimenting and started actually shipping agents into production. True to form, everyone built their own version of orchestration and monitoring underneath it, because no standard existed yet. That was phase one, and it’s already behind us.

Right now, in 2026, we’re squarely in the middle of the orchestration fight. AWS has AgentCore, and — like Microsoft with Azure AI Foundry — has adopted Google’s A2A protocol rather than try to compete with it. Google has Vertex AI Agent Builder. Salesforce has Agentforce. There’s LangGraph Platform, CrewAI Enterprise, and no small number of teams still building it themselves the way we all did with cloud and with microservices. If you want the historical comparison, this is the Kubernetes-vs-Mesos moment — several credible options, no clear winner yet, and a real cost to guessing wrong.

The governance layer is just starting to take shape: policy engines, cost tracking, agent identity and access control, budget enforcement so an agent loop doesn’t quietly burn through your model spend. It’s the service mesh moment for agents. Having built the last one by hand, my guess is it arrives faster than most roadmaps currently assume.

Why it keeps getting faster

Line the three timelines up and it’s a little unsettling:

Cloud: 2006 → 2012 → 2017 → 2022  
Microservices: 2010 → 2015 → 2019 → 2024  
Agentic AI: 2023 → 2025 → 2027 → 2030?

Cloud took about sixteen years start to finish, if you count from when AWS made the technology available rather than from when enterprises actually trusted it. Microservices took ten to twelve. If the pattern holds, agents do it in six or seven, because we’re not starting from zero this time. Enough of us have already built an orchestration layer under duress, and already built a governance layer after the fact, that the whole industry has some muscle memory it didn’t have in 2010.

What I’d actually tell a team deciding right now

Here’s the practical version of all this, and it comes with a caveat: I’ve seen both sides of the “move now or wait” decision play out, and it’s genuinely not obvious which one is right.

In the cloud era, the teams that moved early built everything themselves and paid for it in complexity. The teams that waited got a much smoother ride once standards settled — but by then the early movers had years of scars that turned out to be useful. Cloud eventually stabilized either way, so the early-mover advantage was real but not decisive.

In the microservices era I watched this split happen inside the same organization, sometimes on adjacent teams. The team that built on Spring Boot early and rode the wave into Kubernetes ended up with people who understood failure modes nobody could have explained on a whiteboard. The team that waited got an easier technical path but showed up a few years behind on operational judgment. That gap mattered for a while — three or four years — before it leveled out.

Agents feel higher stakes to me than either of those, and here’s why: cloud and microservices were infrastructure. Agents are increasingly the business logic itself. The lessons you learn running them in production — how they drift, where they fail silently, what governance actually needs to catch — aren’t the kind of thing you can pick up from a case study later. You kind of have to have broken something to know it.

So if you’re building now, you’re buying operational experience while the standards are still unsettled, and that’s worth something regardless of which framework wins. If you’re waiting, you’ll get a cleaner path eventually, but you’re trading that for time your competitors are spending learning what actually breaks.

Neither choice is wrong. I just don’t think you get to pretend the pattern isn’t there.

A few guesses, on the record

If history repeats the way I think it will, by 2027 or 2028:

Some version of A2A, or whatever succeeds it, becomes the de facto way agents talk to each other — the HTTP-for-agents outcome. One or two orchestration platforms end up dominant, and most of today’s contenders quietly fall off. Governance stops being a differentiator and becomes a checklist item, the same way IAM did for cloud and service mesh did for microservices. And rolling your own orchestration becomes possible but rare, the way running your own Kubernetes cluster from scratch is possible but rare today.

I could easily be wrong about the specific winners. I’d put more money on the general shape, mostly because I’ve now watched that shape happen twice, from the inside, in different companies with different constraints, and it rhymed both times.

Where this leaves us

None of this is really a prediction in the crystal-ball sense. It’s just pattern matching, done slowly, over a career.

Frameworks, then orchestration, then governance — that’s cloud, that’s microservices, and that’s where agentic AI is heading, just compressed. The teams that notice this early don’t have to be caught flat-footed by the governance phase. They can build the observability and policy hooks in now, before the industry decides those are mandatory instead of optional.

That’s the thing I wish someone had told me before my first cloud migration instead of after it.

If governance is genuinely coming whether we like it or not, the real question is whether you build that layer yourself now, or design things flexibly enough to plug into whatever standard eventually wins. I don’t think there’s one right answer. I’d be curious where your team lands on it.

Agent Architecture vs. Agentic Platform Architecture: Why Your Second Agent Is a Different Problem

Bhavik Shah — Fri, 17 Jul 2026 21:31:02 +0000

Building one agent is the easy part. The real architecture questions show up when you try to run several.

Every few days a recruiter or hiring manager reaches out about an “AI Architect” role, and the goal is usually some version of the same idea:

“We want to build agents to automate different parts of the business — support triage, supply-chain tasks, code migrations, some finance ops…”

It’s a reasonable goal, and the enthusiasm is real. But when the conversation reaches the operational questions — how will agents authenticate to each other? who caps token spend when something misbehaves? how do events move between agents? where does a human approve the risky actions? — it usually becomes clear those parts haven’t been scoped yet.

That’s not a knock on anyone. The distinction at play is genuinely easy to miss — I’ve watched experienced teams miss it, and I’ve gotten it wrong myself. Agent architecture (how a single autonomous entity thinks and acts) and agentic platform architecture (the infrastructure that lets several of them run reliably) are different problems that happen to share a word. Build a handful of agents without the second one and things tend to work fine in the demo — then get fragile and expensive in production.

This post walks through both: what’s inside each, where each shines and hurts, real platform examples (AWS Bedrock AgentCore and its peers), the failure modes that show up in production, and the practices that help avoid them.

The core distinction: the car and the highway system

An agent architecture describes one autonomous worker from the inside: how it reasons, what it remembers, which tools it can reach, and what limits bind it. It is the design of a high-performance car.

An agentic platform architecture is everything that lets hundreds of those cars share the road safely: the road network, the traffic rules, the fueling stops, and the licensing office. It treats each agent as a small, replaceable service and carries — as shared infrastructure — everything the agents shouldn’t have to.

One answers “how does this specific agent accomplish its task?” The other answers “how does an organization run, watch, and govern a growing fleet of agents that different teams keep shipping?”

Figure 1 — The same building blocks, coupled inside one app vs. provided as shared, governed platform services.

Part 1: Inside a single agent

Strip away the hype and a production-grade agent is five parts working together:

1. The model — the reasoning core. The engine that decides what to do next, sequences the steps, and recovers when a step goes wrong. Tune it for task accuracy before you worry about cost or latency.

2. Instructions — the operating contract. Your team’s procedures translated into the prompt: a tightly scoped brief, worked examples, the edge cases spelled out, and what the agent should do when each thing fails.

3. The reasoning loop — the cognitive pattern. The agent doesn’t just answer; it plans, acts, observes the result, and reflects, looping until the task is done. This is where patterns like ReAct (Reason + Act), Plan-and-Solve, and self-correction live. A useful test for whether you’ve built an agent at all: step away from the keyboard. If the task keeps advancing without you, it’s an agent; if it stalls waiting for your next message, it’s a chatbot.

4. Memory — and it’s three things, not one. Short-term working memory holds the current task state and recent steps. Long-term / episodic memory persists past interactions, user preferences, and previously successful runs. Semantic memory is your enterprise knowledge, chunked into a vector store (pgvector, Qdrant, Pinecone) and retrieved via RAG. Skip the distinction and your agent is brilliant for one turn and amnesiac the next.

5. Tools — the hands. Capabilities exposed via strict schemas: a web-search tool, a code executor, database queries, an email/Slack API, a browser, a CRM updater. Increasingly these are wired through the Model Context Protocol (MCP), the emerging standard for describing tools so any agent can discover and call them. Alongside tools sit skills — packaged, reusable instructions the agent loads on demand, like a report-writer, a code-reviewer, or an invoice-parser.

You rarely build all of this from scratch. Agent frameworks — LangGraph (graph-based, stateful orchestration), CrewAI (role-based multi-agent teams), Microsoft’s AutoGen, AWS Strands Agents, LlamaIndex — give you the loop, tool bindings, and memory hooks as libraries. But note what they are: authoring tools. They help you write the agent. They do not run, govern, or scale it — that is the platform’s job, and conflating the two is where teams get burned.

Advantages of the single-agent approach

Speed to value. You can ship something genuinely useful in days, not quarters.
Full control. Every prompt, retry, and tool binding is yours to tune for one specific job.
Low ceremony. No message buses, no multi-tenant identity, no platform team required.
Cheap to run — at first. One process, one model bill, one deploy pipeline.

Disadvantages

Nothing is reusable. The loop, the logging, the tool wiring — all rebuilt for the next agent.
Observability is whatever you remembered to log. Debugging means grepping print statements and guessing what the model did.
Scaling is vertical. A traffic spike means a bigger box; one slow request stalls the rest.
Governance by good intentions. Keys in the codebase, access rules in the prompt, budgets in someone’s head.

Where a single agent is genuinely the right call

Prototypes and proofs-of-concept where the question is “does this work at all?”
A single, well-scoped internal task: triaging one inbox, summarizing one report type, migrating one codebase.
Teams of one or two engineers with no platform organization behind them.

Structural challenges at the agent level

Loop runaway. The most common way agents fail in production: a tool errors or returns something ambiguous, the agent retries, the retry fails the same way, and the cycle spins all night on your token bill. The runtime — not the prompt — must enforce a ceiling on steps per task and a hard spend limit per request.
Context-window decay. On long runs, accumulated reasoning traces crowd the window and answer quality drifts downward. You need compaction — summarize and prune as you go — to hold quality steady.
Brittle structured extraction. Relying on the LLM to format complex API arguments works until a model upgrade subtly changes its output habits. Validate every tool call against its schema; never trust, always parse.

The Monolithic Agent Trap

Here’s how the single-agent approach fails in slow motion. You build agent #2 and copy the loop. Agent #3, you copy it again — but tweak the retry logic, so now they’ve quietly drifted apart. By agent #10, infrastructure concerns — connection pools, secret keys, homegrown logging, a hard-pinned model endpoint, security rules — are baked into every agent’s code, each slightly differently.

Then the company switches model providers, or legal updates one compliance rule — and the change fans out into a hand-edit of every agent you own. Nothing stops the marketing team’s agent from calling an internal finance API — no matter what its prompt says — because access control was never a system property, only a prompt suggestion.

The trap in one sentence: infrastructure baked into agent code works for one agent, and fails combinatorially for twenty.

None of these are agent problems. They’re platform problems — and you’ve been solving them one agent at a time, which doesn’t compose.

Figure 2 — Layer by layer: the pain with one agent, and how a platform removes it.

Part 2: The agentic platform — infrastructure for a fleet

An agentic platform is the shared runtime an enterprise puts underneath all of its agents — hosting them, brokering their communication, auditing their actions, and enforcing its rules on every one. The agents become thin — mostly a prompt plus a policy — and everything hard moves down into shared services:

Unified front door & API gateway. Multi-tenant boundaries, functional routing, incoming webhooks and events. Departmental “front doors” (HR assistant, finance copilot, support agents) all land on the same platform.
Agent runtime / harness. One standardized, versioned reasoning scaffold — loop, retries, timeouts, context assembly, step caps — that every agent inherits instead of reinventing.
Orchestration & agent-to-agent communication. Managed handoffs between agents — supervisor, pipeline, or peer-to-peer — whether over an event bus (Kafka, Redis streams) or the emerging Agent-to-Agent (A2A) protocol, which standardizes how agents from different teams (or vendors) exchange structured work.
Data & knowledge layer. Centralized knowledge graphs, vector stores and semantic indexes, streaming data — shared grounding instead of per-agent silos.
Memory services. Short-term, long-term/episodic, and semantic memory offered as managed services with retention control.
Tool & API fabric. Tools registered once — increasingly via the Model Context Protocol (MCP) — permissioned centrally, reused by any agent that needs them.
Model gateway / inference router. Routes each sub-task to the right model (frontier model for hard reasoning, cheap fast model for parsing), with caching, rate-limiting, and automatic failover between providers. A provider outage becomes a config change, not a 2 a.m. refactor.
Governance, identity & guardrails. IAM/RBAC so agents have real identities and scoped permissions; centralized input/output guardrails; audit logs; enforced budgets.
Human-in-the-loop service. One approval engine for the whole fleet: when any agent reaches for a high-risk action — a refund, a production change, an outbound customer email — the platform freezes that run until a person approves, then resumes it.
Observability & evaluations. End-to-end tracing of every step, tool call, token and dollar (OpenTelemetry, LangSmith, Arize Phoenix), plus automated evals that catch quality regressions before they ship.
Lifecycle management. Provision, version, pause, and deprecate agents like any other production service, with CI/CD and staged rollouts.

A worked example: AWS Bedrock AgentCore

If the platform concept feels abstract, look at how AWS productized it. Amazon Bedrock AgentCore — generally available since October 2025 — is essentially the platform column of this article sold as composable managed services. Its building blocks map almost one-to-one:

Runtime — serverless, session-isolated execution with long-running windows (up to eight hours) and Agent-to-Agent (A2A) protocol support. Works with any framework (LangGraph, CrewAI, Strands, LlamaIndex) and any model, inside or outside Bedrock.
Gateway — turns APIs and Lambda functions into agent-compatible tools and connects to existing MCP servers, acting as one secure endpoint where agents discover and use tools.
Memory — managed session and long-term memory with pluggable extraction/consolidation strategies.
Identity — agents get real identities, OAuth/IAM-based authorization, and secure token vaults so they can act on behalf of users with scoped access.
Policy — centralized, natural-language or policy-as-code controls over what agents are permitted to do (reached GA in early 2026).
Observability & Evaluations — step-by-step execution tracing plus built-in evaluators for response quality, safety, task completion, and tool usage.
A managed harness — the newest piece (GA mid-2026): define an agent with one API call, invoke it with another, with the loop, tools, memory and tracing handled by the platform.

The strategic read: a hyperscaler looked at what every enterprise was hand-rolling — runtime, gateway, memory, identity, policy, observability — and shipped exactly that list. That’s strong independent confirmation of where the platform boundary sits.

Other platforms worth knowing

Google Vertex AI Agent Builder / Agent Engine — Google Cloud’s managed agent infrastructure, with native RAG and A2A support; natural fit for BigQuery/Gemini-centric shops.
Azure AI Foundry Agent Service — Microsoft’s developer-grade managed runtime with per-agent Entra identity, private networking, and multi-framework support (LangGraph, OpenAI Agents SDK, Claude Agent SDK).
LangGraph Platform — the managed deployment layer for LangGraph agents, with state persisted at every execution step; maximum control over the execution graph, portable across clouds.
Salesforce Agentforce — the CRM-native agentic platform: agents built directly on Salesforce data, workflows, and permissioning, aimed at sales and service automation for organizations already living in that ecosystem.
CrewAI Enterprise — the managed tier of the fastest route to multi-agent prototypes, when role-based collaboration is your primary pattern.
Custom platforms — asynchronous runtimes built on distributed task queues (Celery + Redis) over Kubernetes/EKS, for teams that need full control and accept full operational burden.

A pragmatic 2026 heuristic: if you’re committed to one cloud, use that hyperscaler’s platform for deployment — but keep your agent logic in portable framework code so the authoring layer isn’t locked in.

Frameworks vs. platforms, in one line: LangGraph and CrewAI help you write an agent; AgentCore, Vertex AI Agent Builder, and Agentforce help you run a fleet of them — with identity, guardrails, observability, and scale handled for you. Most production stacks use one of each.

Advantages of the platform approach

Write-once infrastructure. The loop, guardrails, tracing, and tool fabric are built once and inherited by every agent.
Governance as a system property. Access control, budgets, and approvals are enforced by the platform — not requested politely in a prompt.
Organizational leverage. Data scientists tune prompts and reasoning; the platform team owns scale, security, and cost. Different teams ship agents without re-solving infrastructure.
Resilience by default. Model failover, worker isolation, and staged rollouts mean one bad agent or one provider outage doesn’t take down the fleet.

Disadvantages

Upfront cost and ceremony. A platform is real engineering. Built too early, it’s expensive scaffolding around a product that doesn’t exist yet.
New failure surface. Message buses, identity systems, and gateways can themselves fail — and platform bugs affect every agent at once.
Potential lock-in. Managed platforms trade portability for convenience; choose where you accept that trade deliberately.
Organizational overhead. Someone has to own the platform — its roadmap, its SLAs, its upgrades.

Platform-level engineering challenges

Multi-tenant state management. Keeping every tenant’s data walled off from every other’s — while long-running agent sessions still reach their own state quickly.
Unsustainable scale costs. Token bills compound across a fleet. Survival requires caching (prompt and semantic) plus routing discipline: reserve frontier models for the reasoning that needs them, and push parsing and extraction to small, cheap ones.
Distributed auditing complexity. When a workflow fails three handoffs deep, per-agent logs won’t tell you where the payload went bad. You need traces that follow one request across every agent it touched.

The shield is deeper than it looks: four layers of defense

“We added guardrails” usually means one moderation call on the output. In production, protection works better as defense in depth — four layers, each guarding a different thing, with every request crossing all of them:

Inbound defense — screen what reaches the model. Request and file validation, injection and jailbreak screening, isolation of system instructions, rate and size limits.
Knowledge integrity — trust what the agent reasons over. Source vetting for anything RAG pulls in, grounding and freshness checks, PII redaction in retrieval, and memory hygiene: clear rules for what gets stored, recalled, and retained.
Action control — bound what the agent can do. Least-privilege tool scopes, sandboxed execution, transaction and write ceilings, and human sign-off on high-risk calls.
Runtime assurance — watch how the whole run behaves. Loop and anomaly detection, step caps and spend budgets, output policy validation, and an end-to-end audit trail. This layer never sleeps: it wraps every stage of every run.

In a single agent you hand-roll pieces of these layers and hope. On a platform all four live once, as shared, versioned policy every agent inherits — the difference between “we added some safety checks” and “safety is enforced by default, everywhere.”

Figure 3 — Defense in depth: a request crosses three gates to reach the agent core, inside an always-on assurance layer.

When agents team up: three topologies

Once a platform exists, agents start collaborating — and the shape of the collaboration is an architectural decision:

Supervisor (hierarchical). A lead agent breaks the goal apart, farms the pieces out to specialists, and assembles the final result. Suits complex, cross-functional work — a product-spec agent feeding a coding agent feeding a QA agent.
Pipeline (sequential). Work flows in one direction: each agent’s structured output becomes the next one’s input. Suits predictable, linear back-office flows — invoice intake, then audit, then entry.
Peer-to-peer (network). Autonomous peers coordinate directly, pulling each other in when they need help. Suits open-ended, dynamic problems like supply-chain negotiation.

Reach for multiple agents only when the work truly needs parallel effort or distinct specialisms. A single well-instrumented agent beats a poorly governed swarm every time.

Figure 4 — Supervisor, pipeline, and peer-to-peer: match the topology to the work.

Best practices — and the pitfalls they prevent

Best practiceThe pitfall it prevents Standardize the harness first. One versioned reasoning scaffold — loop, retries, context assembly — shared by all agents.Ten hand-rolled loops that quietly drift apart, each with different bugs. Hard step caps and per-request budgets, enforced by the runtime. Loop runaway — the #1 production failure — burning tokens against a failing tool all night. Trace everything from day one. Every step, tool call, token and dollar, with distributed tracing across agent handoffs.Debugging by grepping print statements; being unable to explain why Agent B received garbage from Agent A. Evals in CI, not vibes in prod. Golden test sets, behavioral assertions, expected tool sequences — run on every prompt or model change.Silent quality regressions shipped because “the demo still looked fine.” Route models through a gateway. Cheap models for parsing, frontier models for hard reasoning, automatic failover.Emergency refactors on a provider outage; a cost curve that kills the project at scale.Give agents real identities (IAM/RBAC), not prompt-based trust.The marketing agent calling the finance API because nothing but a sentence in a prompt said it couldn’t. Centralize human-in-the-loop as a service. One approval engine, risk thresholds defined per action class.High-risk mutations (refunds, prod writes, customer emails) executing with no human gate — or every team building its own inconsistent one.Register tools once (MCP), permission them centrally.Bespoke integrations and API keys duplicated across codebases. Design memory in three tiers (short-term, episodic, semantic) with retention rules.Amnesiac agents, unbounded context windows, and sensitive data retained forever. Start simple; extract the platform when you feel the pain twice. Build agent #1 as an app. When agent #2 needs the same plumbing, that’s the platform’s birthday.Both failure modes at once: premature platform ceremony, or the Monolithic Agent Trap.

So which do you need?

If you’re shipping your first agent, or a single well-scoped one — build the agent. A platform you don’t need yet is expensive ceremony.

But if your roadmap has ten agents on it, then building each as a standalone app means maintaining ten reasoning loops, ten logging schemes, ten copies of your security rules, and zero shared visibility. At that point the platform isn’t over-engineering. It’s the thing that keeps the fleet debuggable, affordable, and safe.

In an agent architecture, the cross-cutting concerns — the harness, observability, scalability, governance — are plumbing you hide inside the app. In a platform architecture, they are the product.

Build the agent first. Just know the second one is a different kind of problem — and design for it before it designs for you.

Are you building a multi-agent system right now? I’d love to hear how you’re handling agent runtime isolation, global guardrails, and cost control in your stack — drop a comment.

References & Further Reading

This article builds on established research and tooling from the AI engineering community. Key references:

Research & Patterns

“Reason+Act” (ReAct) — Yao et al., 2022. Foundational agent reasoning pattern. arxiv.org/abs/2210.03629
“Plan and Solve Prompting” — Wang et al., 2023. Multi-step agentic reasoning. arxiv.org/abs/2305.04091

Frameworks & Standards

LangGraph — LangChain’s stateful agent framework. langchain-ai.github.io/langgraph/
CrewAI — Role-based multi-agent orchestration. crewai.com
AutoGen — Microsoft’s multi-agent conversation framework. microsoft.github.io/autogen/
Model Context Protocol (MCP) — Anthropic’s standard for agent-tool integration. modelcontextprotocol.io

Platforms & Services

AWS Bedrock AgentCore — Amazon’s managed agent runtime. aws.amazon.com/bedrock/agentcore/
Google Vertex AI Agent Builder — Google Cloud’s agent development platform. cloud.google.com/vertex-ai/docs/agents
Salesforce Agentforce — CRM-native agentic platform. salesforce.com/products/agentforce/
Azure AI Foundry — Microsoft’s AI development environment. azure.microsoft.com/en-us/products/ai-foundry/

This article reflects current best practices in agentic AI architecture as of 2026. The field evolves rapidly; for the latest on frameworks, standards, and platforms, refer to their official documentation.