Bedrock AgentCore Runtime Latency: What Actually Matters (Multi-Language Benchmarking)

This article presents comprehensive latency benchmarking results for AWS Bedrock AgentCore Runtime across four programming languages (Go, Node.js, Java, Python), three framework options (raw boto3, Strands SDK, LangChain), multiple network configurations, and caller locations. The goal is to provide data-driven guidance on what design choices affect latency and what does not.

Background

AgentCore Runtime is the compute layer for hosting agents and tools in Bedrock AgentCore. Customers frequently ask:

• Which language should I write my runtime container in?
• Does PUBLIC vs VPC network mode affect performance?
• Should I use Strands SDK, LangChain, or raw SDK calls?
• Does the client SDK language matter?
• Where should my caller run for best latency?

This article answers each question with measured data from a controlled test environment.

Test Environment

• Region: us-east-1
• Protocol: HTTP (port 8080)
• Model (for real agent tests): Claude Haiku 4.5 via cross-region inference profile
• VPC: Private subnets with com.amazonaws.us-east-1.bedrock-agentcore interface endpoint
• Caller locations tested: In-VPC Lambda (private subnet) and external (boto3 with connection reuse)
• All container images built for linux/arm64 (AgentCore Firecracker VMs are ARM64)

Finding 1: Container language does not affect warm latency

Four echo runtimes were deployed, each implementing the same trivial HTTP server (receive JSON, return it with metadata). All use the HTTP protocol on port 8080.

Language	Image Size	In-VPC Median	External Mean
Go	22 MB	85ms	968ms
Node.js	229 MB	80ms	950ms
Java (Corretto 21)	494 MB	77ms	945ms
Python (3.12 + gunicorn)	52 MB	84ms	937ms

All four are within 10ms of each other on warm in-VPC calls. The platform routing floor dominates. Language choice should be driven by team expertise and ecosystem, not latency.

Finding 2: Caller location is the single biggest factor

The same runtime invoked from three different locations:

Caller Location	Mean Latency	What Adds Cost
In-VPC Lambda (VPC endpoint)	80-90ms	Platform routing only
External (boto3, connection reuse)	937-968ms	Network RTT to region
External (AWS CLI, no reuse)	1,750-1,830ms	Network RTT + TLS + SDK init

Placing callers in the same VPC with a bedrock-agentcore interface endpoint provides 5-10x latency improvement. This is the single most impactful optimization.

Finding 3: PUBLIC vs VPC-MODE runtime has no latency impact

VPC-MODE places the container's ENI in your private subnet. PUBLIC mode runs on platform-managed infrastructure.

Language	PUBLIC Median	VPC-MODE Median
Go	87ms	85ms
Node.js	80ms	84ms
Java	77ms	85ms
Python	84ms	87ms

Identical once warm. VPC-MODE has higher cold-start spikes (ENI attachment takes 500-2500ms on first invocation) and takes longer to provision (~2.5 minutes vs 5 seconds for PUBLIC). Use VPC-MODE for security and compliance (network isolation, access to private resources), not for performance.

The 3-way comparison (External vs In-VPC PUBLIC vs In-VPC VPC-MODE) shows the full picture:

Finding 4: Client SDK language does not affect warm latency

Three Lambda functions were deployed, each written in its native language, calling its corresponding VPC-mode runtime:

Lambda Language	SDK	Warm Median	Cold Start
Go	aws-sdk-go-v2	98ms	904ms
Node.js	@aws-sdk/client-bedrock-agentcore	115ms	873ms
Python	boto3	96ms	2,631ms

Warm latency is identical. Cold start differs: Go and Node.js initialize in under 1 second, Python (boto3) takes 2.6 seconds due to interpreter and credential chain resolution overhead.

Finding 5: Framework choice adds zero measurable overhead

Three variants of the same SageMaker SME agent were deployed, all using Claude Haiku 4.5 with the same system prompt and question:

Framework	Framework Overhead	Image Size
Raw boto3 (invoke_model)	0.2ms	180 MB
Strands Agents SDK	0.2ms	347 MB
LangChain (ChatBedrock)	0.1ms	438 MB

Framework processing overhead is under 0.3ms for all three. The model inference call (5-26 seconds depending on output length) accounts for over 95% of end-to-end latency. Choose frameworks based on developer productivity:

• Raw boto3: Lightest image, full control, manual orchestration
• Strands SDK: Built-in tool registration, memory management, conversation handling
• LangChain: Rich ecosystem of chains, RAG integrations, prompt templates

Finding 6: Connection pool size matters only at high concurrency

Connection pool stress tests with boto3 (urllib3 backend) at varying pool sizes:

Concurrency	Pool=2 Mean	Pool=10 Mean (default)	Pool=50 Mean
1 thread	84ms	81ms	81ms
10 threads	153ms	139ms	132ms
50 threads	204ms	155ms	113ms
100 threads	254ms	138ms	142ms

No errors at any concurrency level. The default pool size (10) is adequate for most workloads.

Finding 7: Model inference dominates real agent latency

For a real SageMaker SME agent answering technical questions:

Component	Latency	Percentage of E2E
Container processing (framework + parsing)	less than 1ms	less than 0.01%
Platform routing (SigV4, DDB, Firecracker)	95-375ms	3-5%
Model inference (Claude Haiku 4.5)	5,700-26,000ms	95%+

The model call scales with output token count (approximately 5ms per output token for Haiku 4.5).

What actually matters (ranked)

Rank	Factor	Impact
1	Caller location (in-VPC vs external)	5-10x
2	Model choice (for real agents)	Determines total time
3	SDK cold start (Go/Node vs Python)	3x on first call only
4	Connection pool size	Mild at high concurrency
5	Container language	Zero impact
6	Runtime network mode (PUBLIC vs VPC)	Zero impact
7	Client SDK language (warm)	Zero impact

Optimization Recommendations

1. Place callers in-VPC with a bedrock-agentcore interface endpoint for 5-10x latency reduction.

2. Reuse SDK clients across invocations. Never create a new client per call. In Lambda, initialize at module scope outside the handler.

3. Do not optimize container language for latency. Pick what your team knows. The platform routing floor makes it invisible.

4. Use VPC-MODE for security, not performance. Identical once warm.

5. Consolidate multi-tool agents into fewer runtimes. Each sequential InvokeAgentRuntime call adds platform routing overhead.

6. Use streaming for better perceived performance. Model inference is the bottleneck.

Code Configuration Support

Language	Code Config (zip to S3)	Container Config (Docker)
Python	Yes (3.10 through 3.14)	Yes
Node.js	Yes (NODE_22)	Yes
Go	No	Yes
Java	No	Yes

Architecture

Caller (Lambda / ECS / App)
  -> [VPC Endpoint or Public Endpoint]
    -> AgentCore Platform (SigV4 validation, session lookup, routing)
      -> Container (PUBLIC: platform VM | VPC: ENI in your subnet)
        -> [Optional] Bedrock InvokeModel (model inference)

Source Code

All source code, raw data, Lambda functions, and charts are available here:

Neloh / agentcore-latency-benchmarks

AgentCore Runtime latency benchmarking: Go, Node.js, Java, Python across PUBLIC/VPC modes

AgentCore Runtime Latency Benchmarks

Comprehensive latency benchmarking of AWS Bedrock AgentCore Runtime across 4 programming languages (Go, Node.js, Java, Python), multiple network configurations, and caller locations.

Key Findings

Factor	Impact on Warm Latency
Caller location (in-VPC vs external)	5-10x - the #1 factor
Model choice (for real agents)	Determines total time (95%+)
SDK cold start (Go/Node vs Python)	3x on first call only
Connection pool size	Mild at high concurrency
Container language	Zero impact
Runtime network mode (PUBLIC vs VPC)	Zero impact
Client SDK language (warm)	Zero impact

---

Test Results

1. Four-Language Echo Comparison (In-VPC)

All languages perform identically once warm. The platform routing floor (~80ms in-VPC) dominates.

---

2. In-VPC vs External Caller

Calling from within the VPC (via bedrock-agentcore interface endpoint) is 5-10x faster than calling over the public internet.

---

3. Full Comparison: Echo vs Real Agent (SME + Claude Haiku 4.5)

Model inference (Bedrock) accounts for 95%+ of…

View on GitHub

References

• Bedrock AgentCore Runtime documentation
• Strands Agents SDK
• LangChain AWS integration