DEV Community: MishabuildingAI

What is an agent runtime? Six questions that tell you if you need one

MishabuildingAI — Tue, 29 Sep 2026 03:08:45 +0000

An agent runtime is the layer that keeps an agent's work alive between turns: the session's state and event history, when the machine sleeps and wakes, pauses for approval, and the credentials the agent must never read.

That's the short version. The longer one matters because the four words people use for this stack (framework, harness, runtime, sandbox) mean different things depending on who is writing. LangChain, which coined a lot of this vocabulary, says in its own post that there is no clear definition of framework versus runtime versus harness.

Disclosure: I work on Gobare, a hosted agent runtime. We run the three layers as separate processes, so the lines below are ones we had to draw in code.

Four words, four jobs

Layer	What it is	What it owns	Examples
Framework	A library you write an agent with	Your control flow	LangChain, CrewAI, OpenAI Agents SDK
Harness	A finished agent loop around a model	Prompting, tool calls, context within a turn	pi, Claude Agent SDK, Codex
Runtime	The service that runs that loop over time	State, events, lifecycle, waits, credentials	Bedrock AgentCore Runtime, Google Agent Runtime, Gobare
Sandbox	An isolated machine	Where commands, files and browsers execute	E2B, Daytona, Modal

Two things make this messier in practice. OpenAI's Agents API and Claude Managed Agents bundle all of it behind one API, so comparing them with a sandbox provider goes nowhere. And some roundups file sandboxes under "runtimes", which is true only in the sense that code runs there.

One test that separates them

Ask what is still there when the agent's process ends: the turn finished, the machine was reclaimed, or something crashed.

A framework leaves nothing. It was a library in your process.
A harness leaves whatever it wrote to disk.
A sandbox leaves its filesystem until it is destroyed.
A runtime is the thing that decides the answer.

The hyperscalers answer it differently, and say so. AWS's AgentCore Runtime gives each session a microVM, stops it after 15 minutes idle by default or 8 hours of compute, and documents session state as ephemeral, with AgentCore Memory as the place for anything durable. Google's Agent Runtime, formerly Vertex AI Agent Engine, supports long-running operations of up to seven days. Both are coherent. The mistake is picking one without reading its answer.

What a runtime has to do

If you build one yourself, this is the list you end up writing:

Keep the session. Transcript and events stored outside the machine, replayable from the last event a client saw.
Sleep and wake the machine. And tell "idle" from "running a long build", which is where naive timers break.
Wait for people and for your code. A function only your backend can run, an approval, a question. Durable, so a restart doesn't lose the question.
Hold secrets the agent can't read. The model key should never enter the sandbox, which means something outside it makes the model call.
Notify your systems. Signed webhooks, retried, delivered at least once, so your backend doesn't hold a connection open for an hour.
Enforce limits. Concurrency, rate limits, a maximum lifetime per machine.
Let results outlive the machine. OpenAI's hosted sandboxes and Gobare both publish files under /workspace/outputs as artifacts, which suggests this is settling into a shared convention.

Do you need one? Six questions

Count how many are true for your agent:

[ ] A task takes longer than one HTTP request will wait.
[ ] It has to stop and wait for a person or your system.
[ ] Many sessions run at once, for many users.
[ ] A crash or a deploy must not lose the work.
[ ] It holds credentials the agent itself must not read.
[ ] Someone other than the person who started it reads the result later.

Zero or one: a model and a harness are enough. A chatbot that answers in one request doesn't need a runtime. Two or more: you are building one whether you call it that or not. It shows up as a queue, a state table, a reaper for idle machines, a webhook sender and a secret boundary.

Build it or use one

Build on a durable execution engine (Temporal, Inngest), a sandbox provider and a harness. You get every choice and own every failure mode. It is the right call when the runtime is your product.

Use one, and pick by what it binds you to:

AgentCore Runtime and Google Agent Runtime bind you to a cloud you may already be on.
OpenAI's Agents API and Claude Managed Agents bind you to a model provider.
Gobare runs the open-source pi harness on any model you connect with your own key. It runs on a single sandbox provider and you can't bring your own compute, and a sandbox is reclaimed two hours after it was created, paused time included. Those are real limits to weigh.

Which of the six questions was the one that pushed you to build a runtime, or away from it?

The full version, with a layer diagram and an FAQ on AgentCore and Google's Agent Runtime, is on gobare.dev.

Before you leave Claude Managed Agents, check these five things

MishabuildingAI — Tue, 29 Sep 2026 03:06:03 +0000

If you build only on Claude, Claude Managed Agents is the hosted harness to use. Here are the five cases where you need something else, checked against Anthropic's docs.

Disclosure: I work on Gobare, one of the alternatives below. I've quoted Anthropic's documentation rather than paraphrasing it, and there's a section on where we're weaker.

When to stay

Claude Managed Agents gives you a lot that is hard to build yourself:

Versioned agent configurations.
Sandboxes that run in "an Anthropic-managed cloud sandbox, or a self-hosted sandbox on your own infrastructure".
Scheduled deployments for cron-style runs, and per-session spend budgets.
Full checkpoints. When a session goes idle, "its sandbox is checkpointed, preserving the full sandbox state, including the filesystem, installed packages, and any files the agent created."
Published pricing: "$0.08 per session-hour", and "Runtime is measured to the millisecond and accrues only while the session's status is running."

If none of the checks below applies to you, stop reading and stay.

Check 1: does the model have to be something other than Claude?

Anthropic describes the product as the harness "for running Claude as an autonomous agent", and the setup guide says "Claude 4.5 and later models are supported."

If your evals pick DeepSeek for bulk turns, a customer mandates a provider, or you route by cost, the harness can't run it. This is the most common reason people search for an alternative, and the simplest to test: list the models your product needs in a year.

Check 2: do you need to read or change the agent loop?

You configure the system prompt, tools, MCP servers and skills. How tool calls are dispatched, how errors are recovered and how context is compacted runs on Anthropic's side.

Most teams never need to see that code. If you've ever had to step through a harness to understand why an agent looped, you'll know whether you're one of the teams that does.

Check 3: can your webhooks afford to drop an event?

This one surprised me. Anthropic's webhook docs say: "Anthropic makes up to three delivery attempts", and after the last one "the event is dropped: it isn't queued for later delivery and there's no signal that it was lost." They also recommend reconciling by listing or fetching the resource through the API.

That's a reasonable design, and it's documented honestly. It does mean that if a webhook triggers billing, a database write or a user notification, you need a polling reconciler next to it. Check whether you've built one.

Check 4: will someone need a shell in the running machine?

When a long agent run gets stuck on a broken native dependency or a hung background process, the event stream often isn't enough. I didn't find a documented way to get an interactive shell into an Anthropic-managed cloud sandbox. With a self-hosted sandbox the machine is yours, so you can get in, but then you run the compute.

Check 5: does compliance require Zero Data Retention?

Anthropic is direct: "Managed Agents is not currently eligible for Zero Data Retention or HIPAA Business Associate Agreement (BAA) coverage", because sessions store their history, sandbox state and outputs.

No hosted alternative I know of fixes this, including ours. If ZDR is a hard requirement, self-host.

If you hit one: three routes

	Who runs it	Models	Fixes
Stay on Claude Managed Agents	Anthropic	Claude 4.5+	None of the five, but it's the best at everything else
Self-host open source (OpenMA, open-managed-agents, AstraBox)	You	Any	Any of the five, because the code, the machines and the data are yours; you run the platform
Hosted runtime with your own key (Gobare)	Gobare	Nine providers plus any OpenAI- or Anthropic-compatible endpoint	1, 3 and 4; for 2 you can read the loop but not change it

For the self-hosted route, OpenMA (Apache-2.0) describes itself as "an open-source alternative to Claude Managed Agents and OpenAI Agents API" and also offers a hosted version. open-managed-agents (AGPL-3.0) adds org, team and RBAC governance.

On the hosted route, what Gobare does for checks 1 to 4: you connect a key from any supported provider and it never enters the sandbox. The harness is pi, which is open source and runs unmodified. Webhooks are retried six times over roughly nine hours, with dead-letter tracking. An engineer can ssh -p 2222 s-$SESSION@ssh.gobare.dev into the session's machine, after registering a key in the Console.

Where Gobare is weaker

Before you switch to us, know what you give up:

No self-hosted sandboxes, no scheduled runs, no per-session budgets, no multi-agent threads.
Saved agents aren't versioned.
Snapshots are best-effort: no .git, no node_modules, skipped above 300 MB, and running processes aren't kept.
A sandbox is reclaimed two hours after it was created, paused time included, and a running turn is interrupted. Sessions with a published address or an open SSH connection are exempt, and the next input starts a fresh sandbox from the snapshot.
10 session creations per minute per token; 25 concurrent sessions per organisation by default.
We also store the transcript, so we're not ZDR either.
Pricing isn't published yet. It's free while in alpha, with no margin on tokens.

The short version

Run the five checks. If none applies, stay on Claude Managed Agents. If check 5 applies, self-host. If only checks 1, 3 or 4 apply and you'd rather not run a platform, a hosted runtime on your own key is the middle route.

The full comparison, with a concept map from Managed Agents primitives to another runtime and side-by-side requests, is on gobare.dev.

Which of the five made you look for an alternative, or is there a sixth I've missed?

Before you pick a hosted agent runtime, check what happens at idle

MishabuildingAI — Sun, 27 Sep 2026 14:05:52 +0000

Disclosure: I work on Gobare, one of the three runtimes below. To keep that from mattering, I quote the other two instead of describing them, and there is a section on where each of us loses, Gobare included. All quotes are from public docs, checked 27 September 2026. Misha.

I read the sandbox docs of three hosted agent runtimes side by side, ours included. In a 30-second demo they look the same. The agent writes code, runs it, shows a result.

The difference is one question the demo never asks:

After the agent stops typing, does its machine still exist?

Perplexity, OpenAI and Gobare give three different answers. None of them is wrong. They are building different things. But if you pick one without checking, you find out an hour into a job, at the step where the work was supposed to land.

This post is the check I wish I had done first.

First, be clear which OpenAI product you mean

OpenAI ships two things with similar names. They are not layers of each other.

The Agents SDK is open source and runs in your process. You own the loop. Other models work through LiteLLM (Python) or the AI SDK adapter (TypeScript).
The Agents API is hosted. OpenAI runs the loop, and it is built around OpenAI models.

So "can I use Claude with OpenAI's agents?" has two answers. In the SDK, yes. In the hosted Agents API, the one that gives you a sandbox, non-OpenAI models are on a second-class path.

Everything below is about hosted runtimes: the ones that give you a machine.

Three answers, in their own words

Perplexity, on their sandbox tool:

"Multiple executions in a single response share the same container: files written and packages installed by an earlier step are still there for a later one."

Between responses, nothing is promised. That is the right design for an answer engine that computes. If you want an answer, paying to keep a machine alive afterwards is waste.

OpenAI, on hosted sandboxes:

"If activity and keep-alives stop for an hour, the sandbox can be deleted."

Files you write to /workspace/outputs become immutable artifacts that "remain downloadable after the sandbox expires." The rest of the workspace does not survive. Clean contract: the sandbox is a place to do work, not to keep it.

Gobare, on limits:

"A workspace is also paused after five minutes with nothing happening, and woken by the next thing you send."

Before the machine is reclaimed, the workspace files are snapshotted. It is best-effort. Git history, installed dependencies and running processes are not in the snapshot, and a workspace over 300MB is skipped.

Side by side:

	Perplexity	OpenAI hosted	Gobare
Machine lives for	One response	Until an hour with no activity or keep-alives	Paused after five idle minutes, woken by your next message
What survives	Nothing promised between responses	Files in `/workspace/outputs`, as artifacts	Workspace files, best-effort snapshot
What doesn't	The container	The rest of the workspace	Git history, installed deps, running processes; workspaces over 300MB

Three workloads where the answer decides things

These are not failures of a bad runtime. They are where a correct runtime, picked for the wrong shape of work, gives you a system you cannot finish.

A job longer than the idle window. Say a 40-minute refactor across several turns, with gaps while your dispatcher does something else. On a per-request container each response starts clean, so this can't be expressed. On a session sandbox it works, if you know exactly what "idle" means to your provider.
A task that stops to ask a human. Approval mid-run means the machine waits, maybe for hours. Whether that wait is free, billed or impossible depends on how the runtime treats an idle machine.
An agent that serves something. If the agent starts a preview server, the machine has to exist when the visitor arrives. That can be long after the agent stopped writing code. If the sandbox goes when execution ends, the preview URL goes with it.

The container has a public price

Perplexity publishes a price for the container itself, on their pricing page:

"$0.03 per session (≤20-min billing window)"

That is roughly $0.09 an hour for a managed container. It is the clearest public anchor I found for what hosted agent compute costs.

It also explains the product. Perplexity's money is in the search index. For them, the runtime does not need to be profitable, and that sets the floor for everyone else.

Where each one loses

Perplexity: no session state across responses. If work has to survive a response boundary, you build that yourself on previous_response_id.
OpenAI hosted: an hour of quiet and the working tree is gone. Only what you published to /workspace/outputs stays. Non-OpenAI models are second-class on the hosted path.
OpenAI self-hosted: you own the lifecycle, so you also own its outages. With Daytona as the provider you must set auto_stop_interval=0, or the sandbox stops mid-agent.
Gobare:
- Single sandbox provider (E2B). No bring-your-own-compute. That is real lock-in, and the fairest criticism of us here.
- A workspace is reclaimed two hours of active time after it starts. Paused time doesn't count, but a turn still running at that point fails.
- No model fallback chain. Perplexity tries up to five models; we have nothing equivalent.
- Saved agents are not versioned.
- No search index.

Questions to ask any runtime's docs

Take this list to whichever runtime you are evaluating. Every item comes from a place where these three differ.

Lifetime

Within one response or turn, do executions share a container?
Is anything promised between responses?
How long can the machine sit idle before it is paused or deleted?
What counts as activity? On self-hosted compute, does the provider's idle tracking see the agent's connection, or do I have to disable auto-stop?
Is there a hard cap on session lifetime, and what happens to a turn that is running when it hits?

State

What survives the machine: nothing, an outputs folder, or the whole workspace?
What is excluded? Git history, installed dependencies, running processes, anything over a size limit?

Workload fit

Can the machine wait hours for a human? Is that wait free, billed or impossible?
Can something the agent serves stay reachable after it stops generating?

Control

Who runs the agent loop, me or the vendor?
Which models are first-class on the hosted path?
Can I bring my own compute?
Is there a model fallback when a provider goes down?
Are saved agent configurations versioned?
Is there a published price for the container itself?

The one question that sorts the rest

Not "which runtime is best." Ask:

After my agent stops typing, do I still need the machine?

If no, a per-request container is the honest choice, and you shouldn't pay for more. Perplexity's model is the clean version of that.

If yes, you are choosing between owning the compute yourself and having someone manage its state for you. Both are defensible. Not knowing which one you picked is not.

Full version with every source link: gobare.dev/blog/agent-runtimes-disagree-on-what-a-session-is

If you have run agents in production: which of these questions did you only learn to ask after something broke?