Every few years, software engineering invents a new noun and spends the next eighteen months arguing about whether it deserves to exist. Microservices were "just" applications with extra network hops, until they weren't. Serverless functions were "just" stateless code, until state management around them became its own discipline. Now it's agents.
The question is worth taking seriously, not because the answer is obvious, but because getting it wrong in either direction costs real engineering time. Build custom infrastructure for something that fits neatly into an existing model, and you've wasted months reinventing Kubernetes badly. Force an agent into a workload model that can't represent what it actually needs to do, and you'll spend those same months fighting the abstraction instead of the problem.
Comparing Agents to Known Workload Types
Start with the closest comparisons. A serverless function is short-lived, stateless, and triggered by an event. It runs, returns a result, and disappears. Most agents don't behave that way. An agent handling a customer support escalation might run for minutes, hold conversation state across multiple tool calls, and need to resume if interrupted. That's a mismatch serverless platforms weren't designed for, though plenty of teams try anyway because the tooling is familiar.
Kubernetes workloads, plain worker processes, handle long-running tasks well and support persistent state through volumes or external stores. What they don't handle natively is the agent's specific lifecycle: waiting on a model response, branching based on that response, calling a tool, waiting again, potentially looping back. A generic worker can be coded to do this, but it's not what the abstraction was built for, and you end up writing a mini state machine inside a system that has no native concept of one.
Actors, in the classic computer science sense, get closer. An actor holds private state, processes messages one at a time, and can spawn other actors. That maps reasonably well onto an agent that holds conversation context and delegates subtasks. The gap is that actor frameworks weren't designed with model calls, token costs, or non-deterministic outputs in mind, so a lot still has to be built on top.
Workflow engines are the closest fit for a specific kind of agent: one whose steps are largely known in advance, with the model filling in decisions at defined points. If your agent's behavior is "call the model, branch on the result, call a tool, repeat until done," a workflow engine with a model-call step type gets you most of the way there with a mature, battle-tested orchestration layer underneath.
So Where's the Actual Gap?
The honest answer is narrower than the hype suggests. Most of what looks like a new requirement for agents, state persistence, retries, observability, is a solved problem in existing infrastructure. The genuinely new piece is handling non-determinism as a first-class concern. Traditional workflow engines assume a step either succeeds or fails in a predictable way. An agent step can succeed, produce a plausible-looking output, and still be wrong in a way that only a human or a downstream check catches later. Existing orchestration tools don't have a native concept of "this succeeded technically but might be semantically wrong," and that's the piece worth building new tooling around, not the state management or the retries.
Where People Get This Wrong
A lot of teams reach for a brand-new "agent framework" the moment they start building, skipping past the question of whether a workflow engine with a model-call node would have gotten them there faster and with less risk. New frameworks in this space are, as of this writing, immature. Fewer production hours logged, fewer edge cases discovered, thinner documentation on failure modes. Meanwhile Kubernetes, Temporal, and similar mature systems have years of hardening around exactly the problems agents run into: crash recovery, retries, distributed state.
The reverse mistake happens too, usually at bigger, more risk-averse organizations. Forcing an agent into a rigid workflow engine designed for deterministic business processes, and then fighting the tool every time the agent needs to branch in a way the workflow definition didn't anticipate. Agents genuinely do need more flexibility in how they branch and retry than a typical BPM tool assumes. Picking the wrong side of this tradeoff either way costs months.
The Mumbai Perspective on Agent Infrastructure Decisions
This decision shows up constantly in client conversations for teams doing enterprise software development in Mumbai, particularly for BFSI and logistics clients where existing infrastructure investments are already substantial. A bank that has spent years hardening its Kubernetes deployment pipeline is understandably reluctant to bolt on an entirely new, unfamiliar agent runtime just because a vendor recommended the latest framework. The more defensible path, in most of these cases, is extending what already exists: a workflow engine layer with agent-specific step types, running on infrastructure the client's own engineering team already knows how to operate and debug.
This is where a software development company in Mumbai with real production experience across both traditional backend systems and newer agent tooling has a genuine advantage over a team that's only ever built on one side of that line. Toadster's approach, for instance, generally leans toward extending a client's existing infrastructure rather than replacing it wholesale, specifically because that reduces the operational risk of adopting agents in the first place. That's not the flashiest pitch to make in a sales call, but it tends to be the one that holds up eighteen months later when the client's ops team is the one on call at 2 a.m.
The Practical Test
Ask what the agent's lifecycle actually looks like before deciding on infrastructure. Does it run for seconds or hours? Does it need to survive a process restart mid-task? Does its branching logic look mostly predictable, or genuinely open-ended? An agent that's mostly a fixed sequence with model-driven decisions at a few points fits a workflow engine well. An agent that needs to dynamically decide its own next steps, spawn sub-agents, and hold long-lived context is closer to needing purpose-built agent infrastructure. Most real agents land somewhere in between, which is exactly why the answer to "is this a new primitive" is usually "partially, and only for the parts that are actually novel."
FAQ
Do I need a specialized agent framework to build production AI agents?
Not always. For agents with mostly predictable step sequences, a mature workflow engine extended with a model-call step type often works better than an unproven, purpose-built agent framework. Specialized frameworks earn their place when branching logic is genuinely dynamic and hard to represent as a fixed workflow.
What's the biggest infrastructure mistake teams make with agents
Treating agent infrastructure as entirely novel and rebuilding solved problems, state persistence, retries, crash recovery, from scratch instead of extending mature tooling that already handles those well.
How is an AI agent different from a serverless function?
Serverless functions are typically short-lived and stateless. Agents often need to hold context across multiple steps, resume after interruption, and branch based on non-deterministic model output, none of which fits the serverless model cleanly.
Should enterprises with existing Kubernetes infrastructure adopt new agent-specific platforms?
Usually the more practical route is extending existing infrastructure with agent-aware orchestration rather than migrating to an entirely separate platform, particularly when the existing team already has deep operational knowledge of the current stack.
Top comments (0)