Sibonelo N.

Posted on Aug 16

What Is AgentCore Runtime, Really? Breaking Down the Thing I Already Benchmarked

#aws #bedrock #serverless #architecture

In the first article I measured AgentCore Runtime six different ways: four languages, three frameworks, PUBLIC vs VPC-mode, in-VPC vs external callers. Lots of numbers. What I skipped over is the more basic question: what is Runtime actually doing to produce those numbers?

This article fills that gap, using the same benchmark repo as the worked example instead of starting from a blank slate. It also covers something the first article couldn't have: two days after that benchmark ran, AWS shipped a second compute type for Runtime entirely. More on that below.

What AgentCore Runtime Is

AWS describes it plainly: Runtime is "a secure, serverless runtime environment purpose-built for deploying and scaling dynamic AI agents and tools," offering fast cold starts, session isolation, built-in identity, and support for multi-modal, multi-agent workloads. It doesn't care which framework you brought (CrewAI, LangGraph, LlamaIndex, Google ADK, OpenAI Agents SDK, Strands, or nothing at all) or which model you call (Bedrock, OpenAI, Gemini, anything). It speaks two agent-to-agent/tool protocols out of the box: MCP and A2A.

That framework-agnostic, model-agnostic design is exactly why Finding 1 and Finding 5 in the benchmark article landed the way they did — container language and framework choice added zero measurable overhead, because Runtime doesn't touch either. It just runs your container and enforces a contract.

What's Actually Inside a microVM

I used the word "microVM" in the first article without stopping to explain it, so here it is properly: a microVM is a minimal virtual machine, built on the open-source Firecracker virtualization technology that also powers AWS Lambda. It gives you real VM-level isolation — its own kernel, its own memory space, no shared host kernel with any other tenant — but strips away everything a general-purpose VM boots that an agent session doesn't need, so it can start in tens of milliseconds instead of the seconds a traditional EC2 instance takes.

That's the specific trade AgentCore Runtime is making by default: stronger isolation than a container (which shares the host kernel), at close to container-speed startup, rather than the isolation of a full EC2 instance at EC2 boot times. Every session gets one, and it's why the isolation guarantee reads the way it does in the docs: "no shared state, no shared filesystem" between sessions, even concurrent ones for the same runtime.

Here's what that actually looks like on a warm call — the case that produced the 80-160ms in-VPC numbers from article 1:

And here's the cold path — a brand-new sessionId with no existing microVM to route to:

The only structural difference between the two is one extra step: provisioning a microVM and waiting for /ping to report Healthy before the first /invocations call can go through. Everything downstream of that is identical. That single step is also the entire explanation for why cold start is seconds and warm is milliseconds — it's not a slower code path, it's a whole extra phase that warm calls skip.

Here's a warm call for real, run through the console's own test panel against one of the benchmark repo's echo runtimes:

request_processed_ms: 37. That's not from a load-testing script, it's a single live click. The rest of the response — container_id, uptime_seconds, language — comes straight from the runtime's own code, which is worth looking at directly next.

The Four Building Blocks

Peel back "Runtime" and there are four concrete objects, each with its own lifecycle:

Runtime — the top-level resource. Has a unique identity and is versioned. This is the thing you CreateAgentRuntime once and then update repeatedly.

Version — an immutable snapshot. Every time you change the container image, protocol settings, or network settings, a new version is cut. Version 1 is created automatically the first time you create the Runtime.

Endpoint — an addressable ARN pointing at a specific version. A DEFAULT endpoint is created automatically and always tracks the latest version; you can also create named endpoints (dev/test/prod) that pin to a specific version deliberately, so you're not forced onto whatever was last deployed. Endpoints move through CREATING → READY (or CREATE_FAILED), and UPDATING → READY (or UPDATE_FAILED) without downtime to callers.

That's the theory. Here's what it looks like on an actual runtime — goHttpLatencyTest, compute type microVMs, one version, two endpoints:

And drilling into one of those endpoints:

Session — the actual unit of execution. Identified by a runtimeSessionId (yours, or generated for you on first call), a session runs in a dedicated microVM with fully isolated CPU, memory, and filesystem. It persists for up to 8 hours, moves between Active and Idle, and gets Terminated after 15 minutes of inactivity, at the 8-hour cap, or if deemed unhealthy. Termination destroys the entire microVM and sanitizes memory — reusing the same runtimeSessionId afterward just gets you a fresh environment, not the old one back.

This is the mechanism behind the repo's session-affinity test: embedding a random container ID at startup and checking whether it stays constant across calls is really just checking "am I still inside the same microVM, or did my session get torn down and rebuilt."

Three Protocols, One Contract Each

Whatever's inside the container, Runtime expects one of three fixed contracts, chosen when you create the Runtime:

Protocol	Port	Main path	Health check	Shape
HTTP	8080	`POST /invocations`	`GET /ping`	REST, optional SSE/WebSocket
MCP	8000	`POST /mcp`	`GET /ping`	JSON-RPC 2.0, Streamable-HTTP
A2A	9000	`POST /`	`GET /.well-known/agent-card.json`	JSON-RPC 2.0

Every echo runtime in the benchmark repo — Go, Node.js, Java, Python — implements exactly the HTTP contract: bind 0.0.0.0, answer /ping with a health status, answer /invocations with JSON. Nothing about the contract is language-specific, which is the whole reason a 22 MB Go binary and a 494 MB Java/Corretto image land within 10ms of each other once warm: they're both just satisfying the same thin HTTP contract, and the platform's own routing floor dwarfs whatever the container itself does with the request.

What This Looks Like in Code

It's worth seeing how little code actually satisfies that contract. Here's the Go runtime's entire structure — no subdirectories, no dependencies, one source file:

source/agentcore-go-runtime/
├── Dockerfile
├── deploy.sh
├── go.mod
└── main.go

The Dockerfile is a standard two-stage build: compile a static binary, then copy just the binary into a bare alpine image. Nothing else goes in:

# Multi-stage build for minimal container size
FROM golang:1.22-alpine AS builder
WORKDIR /app

# Copy module file (no external deps)
COPY go.mod ./
COPY main.go .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-s -w" -o /runtime main.go

# Final image: minimal alpine
FROM alpine:3.20
RUN apk --no-cache add ca-certificates
COPY --from=builder /runtime /runtime

# AgentCore HTTP protocol requires port 8080
EXPOSE 8080

RUN adduser -D -u 1001 agentcore
USER agentcore

ENTRYPOINT ["/runtime"]

GOARCH=arm64 isn't a style choice — AgentCore's Firecracker microVMs are ARM64, so anything built for the wrong architecture fails at deploy time, not at runtime. And main.go itself is just two HTTP handlers wired to the two paths from the contract table:

func ping(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(PingResponse{
        Status:           "Healthy",
        TimeOfLastUpdate: time.Now().Unix(),
    })
}

func invocations(w http.ResponseWriter, r *http.Request) {
    // ...decode the request, run whatever "action" it asked for...
    result["_meta"] = Meta{
        ContainerID:        containerID,
        UptimeSeconds:      time.Since(startupTime).Seconds(),
        RequestProcessedMs: requestProcessedMs,
        Language:           "go",
    }
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(result)
}

func main() {
    http.HandleFunc("/ping", ping)
    http.HandleFunc("/invocations", invocations)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Those exact field names — container_id, uptime_seconds, request_processed_ms, language — are what showed up in the test-panel response a few sections back. There's no gap between the code and the observed behavior; the console screenshot is this main.go running.

The console will even generate a starter invocation snippet for any runtime you create, in Python, TypeScript, or JavaScript:

Why PUBLIC vs VPC-Mode Didn't Move the Numbers

Network mode changes where the container's network interface lives — platform-managed infrastructure for PUBLIC, an ENI in your own subnet for VPC-mode — not how sessions, versions, or the request path work. Once a session's microVM is warm, the request still goes through the same routing and lands in the same kind of isolated execution environment either way. That's why Finding 3 showed identical warm latency between the two: the difference is cold-start ENI attachment time, not steady-state architecture.

The New Alternative: Runtime Instances and Capacity Providers

Everything above describes the microVM compute type — the only one that existed when I ran the first article's benchmarks. On 06 Aug 2026, two days after that article published, AWS announced a second compute type for Runtime: Instances.

Where microVMs are fully AWS-managed, ephemeral, and capped at 8 hours per session, Instances runs your agent on Amazon EC2 managed instances inside your own AWS account. AgentCore still provisions, patches, scales, and tears them down for you — you don't do standard EC2 lifecycle operations on them yourself — but the compute lives in your account, so your existing Savings Plans, Reserved Instances, and On-Demand Capacity Reservations apply, and the instances (and any attached EBS volumes) show up in your own EC2 console.

The infrastructure itself is defined by a new resource called a capacity provider: a reusable template specifying the OS, allowed instance types, VPC/subnets, storage volumes, and IAM roles. You attach a capacity provider to a Runtime at creation time via capacityProviderConfiguration, and you can't change compute type after that — it's a decision made once, per Runtime.

What Instances buys you that microVMs structurally can't:

Characteristic	microVMs	Instances
Best suited for	Lightweight, fast-scaling API agents	Long-running, stateful, or collaborative workloads
Maximum session duration	Up to 8 hours	Up to 14 days
Operating systems	Linux containers (arm64 only)	Linux (x86_64 and arm64)
Networking	PUBLIC or VPC	VPC only
Agents per session	One runtime hosts one agent (1:1)	One session can host multiple agents (1:N)
GPU access	Not supported	Supported (g4dn/g5/g6/g6e/gr6/g6f/gr6f/g7e, inf2)
Pricing	Consumption-based, billed by AgentCore	Runs in your account; your EC2 pricing agreements apply

Two consequences worth calling out for anyone coming from the first article's mental model:

Sessions can now survive stop/resume, not just stay idle. On microVMs, termination is final — a reused runtimeSessionId after termination gets a brand-new environment. On Instances, when a session hits its lifetime or is stopped, AgentCore tears down the EC2 instance but keeps the attached EBS volume. Invoking the same runtimeSessionId again provisions a fresh instance, re-attaches that volume, and your agent's filesystem picks up where it left off — potentially on a freshly patched machine image.

Multiple agents can now share one execution environment. The microVM model is strictly 1:1 — one Runtime, one agent, one microVM per session. On Instances, if two separate Runtimes are attached to the same capacity provider and invoked with the same runtimeSessionId, both agents land on the same EC2 instance and share its filesystem. That's a materially different collaboration primitive than anything the microVM model offers.

None of the numbers in the first article measured Instances — that compute type didn't exist yet, and every benchmark there was run against microVMs specifically. That's a legitimate follow-up benchmark on its own: does a GPU-backed Instances session behave the same way under the "does language matter" and "does network mode matter" questions, or does the EC2-backed model change the shape of the answer? Worth its own repo, not a retrofit onto this one.

Seeing It for Real

Everything above is easier to believe with an actual DescribeAgentRuntime response in front of you instead of a docs paraphrase. Here's the real output for one of the benchmark repo's own runtimes, the Go echo container in PUBLIC mode:

{
    "agentRuntimeArn": "arn:aws:bedrock-agentcore:us-east-1:<account-id>:runtime/goHttpLatencyTest-5CKthu5tD0",
    "agentRuntimeName": "goHttpLatencyTest",
    "agentRuntimeVersion": "1",
    "networkConfiguration": {
        "networkMode": "PUBLIC"
    },
    "status": "READY",
    "lifecycleConfiguration": {
        "idleRuntimeSessionTimeout": 900,
        "maxLifetime": 28800
    },
    "agentRuntimeArtifact": {
        "containerConfiguration": {
            "containerUri": "<account-id>.dkr.ecr.us-east-1.amazonaws.com/agentcore-go-runtime:http-v1"
        }
    },
    "protocolConfiguration": {
        "serverProtocol": "HTTP"
    }
}

Two numbers to notice: idleRuntimeSessionTimeout: 900 and maxLifetime: 28800 are exactly the 15-minute idle timeout and 8-hour max lifetime from the session lifecycle diagram above, in seconds. That's not a coincidence or a rounded approximation — it's the literal same value the docs describe, straight from the API.

For the VPC-mode Python runtime, the only structural difference is the networkConfiguration block growing to include securityGroups and subnets — everything else (protocol, lifecycle, artifact shape) is identical. That's the API-level version of Finding 3: PUBLIC and VPC-mode differ in exactly one config block and nowhere else.

DEV Community