DEV Community: Sibonelo N.

Bedrock AgentCore Runtime Latency: What Actually Matters (Multi-Language Benchmarking)

Sibonelo N. — Tue, 04 Aug 2026 20:24:28 +0000

This article presents comprehensive latency benchmarking results for AWS Bedrock AgentCore Runtime across four programming languages (Go, Node.js, Java, Python), three framework options (raw boto3, Strands SDK, LangChain), multiple network configurations, and caller locations. The goal is to provide data-driven guidance on what design choices affect latency and what does not.

Background

AgentCore Runtime is the compute layer for hosting agents and tools in Bedrock AgentCore. Customers frequently ask:

Which language should I write my runtime container in?
Does PUBLIC vs VPC network mode affect performance?
Should I use Strands SDK, LangChain, or raw SDK calls?
Does the client SDK language matter?
Where should my caller run for best latency?

This article answers each question with measured data from a controlled test environment.

Test Environment

Region: us-east-1
Protocol: HTTP (port 8080)
Model (for real agent tests): Claude Haiku 4.5 via cross-region inference profile
VPC: Private subnets with com.amazonaws.us-east-1.bedrock-agentcore interface endpoint
Caller locations tested: In-VPC Lambda (private subnet) and external (boto3 with connection reuse)
All container images built for linux/arm64 (AgentCore Firecracker VMs are ARM64)

Finding 1: Container language does not affect warm latency

Four echo runtimes were deployed, each implementing the same trivial HTTP server (receive JSON, return it with metadata). All use the HTTP protocol on port 8080.

Language	Image Size	In-VPC Median	External Mean
Go	22 MB	85ms	968ms
Node.js	229 MB	80ms	950ms
Java (Corretto 21)	494 MB	77ms	945ms
Python (3.12 + gunicorn)	52 MB	84ms	937ms

All four are within 10ms of each other on warm in-VPC calls. The platform routing floor dominates. Language choice should be driven by team expertise and ecosystem, not latency.

Finding 2: Caller location is the single biggest factor

The same runtime invoked from three different locations:

Caller Location	Mean Latency	What Adds Cost
In-VPC Lambda (VPC endpoint)	80-90ms	Platform routing only
External (boto3, connection reuse)	937-968ms	Network RTT to region
External (AWS CLI, no reuse)	1,750-1,830ms	Network RTT + TLS + SDK init

Placing callers in the same VPC with a bedrock-agentcore interface endpoint provides 5-10x latency improvement. This is the single most impactful optimization.

Finding 3: PUBLIC vs VPC-MODE runtime has no latency impact

VPC-MODE places the container's ENI in your private subnet. PUBLIC mode runs on platform-managed infrastructure.

Language	PUBLIC Median	VPC-MODE Median
Go	87ms	85ms
Node.js	80ms	84ms
Java	77ms	85ms
Python	84ms	87ms

Identical once warm. VPC-MODE has higher cold-start spikes (ENI attachment takes 500-2500ms on first invocation) and takes longer to provision (~2.5 minutes vs 5 seconds for PUBLIC). Use VPC-MODE for security and compliance (network isolation, access to private resources), not for performance.

The 3-way comparison (External vs In-VPC PUBLIC vs In-VPC VPC-MODE) shows the full picture:

Finding 4: Client SDK language does not affect warm latency

Three Lambda functions were deployed, each written in its native language, calling its corresponding VPC-mode runtime:

Lambda Language	SDK	Warm Median	Cold Start
Go	aws-sdk-go-v2	98ms	904ms
Node.js	@aws-sdk/client-bedrock-agentcore	115ms	873ms
Python	boto3	96ms	2,631ms

Warm latency is identical. Cold start differs: Go and Node.js initialize in under 1 second, Python (boto3) takes 2.6 seconds due to interpreter and credential chain resolution overhead.

Finding 5: Framework choice adds zero measurable overhead

Three variants of the same SageMaker SME agent were deployed, all using Claude Haiku 4.5 with the same system prompt and question:

Framework	Framework Overhead	Image Size
Raw boto3 (invoke_model)	0.2ms	180 MB
Strands Agents SDK	0.2ms	347 MB
LangChain (ChatBedrock)	0.1ms	438 MB

Framework processing overhead is under 0.3ms for all three. The model inference call (5-26 seconds depending on output length) accounts for over 95% of end-to-end latency. Choose frameworks based on developer productivity:

Raw boto3: Lightest image, full control, manual orchestration
Strands SDK: Built-in tool registration, memory management, conversation handling
LangChain: Rich ecosystem of chains, RAG integrations, prompt templates

Finding 6: Connection pool size matters only at high concurrency

Connection pool stress tests with boto3 (urllib3 backend) at varying pool sizes:

Concurrency	Pool=2 Mean	Pool=10 Mean (default)	Pool=50 Mean
1 thread	84ms	81ms	81ms
10 threads	153ms	139ms	132ms
50 threads	204ms	155ms	113ms
100 threads	254ms	138ms	142ms

No errors at any concurrency level. The default pool size (10) is adequate for most workloads.

Finding 7: Model inference dominates real agent latency

For a real SageMaker SME agent answering technical questions:

Component	Latency	Percentage of E2E
Container processing (framework + parsing)	less than 1ms	less than 0.01%
Platform routing (SigV4, DDB, Firecracker)	95-375ms	3-5%
Model inference (Claude Haiku 4.5)	5,700-26,000ms	95%+

The model call scales with output token count (approximately 5ms per output token for Haiku 4.5).

What actually matters (ranked)

Rank	Factor	Impact
1	Caller location (in-VPC vs external)	5-10x
2	Model choice (for real agents)	Determines total time
3	SDK cold start (Go/Node vs Python)	3x on first call only
4	Connection pool size	Mild at high concurrency
5	Container language	Zero impact
6	Runtime network mode (PUBLIC vs VPC)	Zero impact
7	Client SDK language (warm)	Zero impact

Optimization Recommendations

Place callers in-VPC with a bedrock-agentcore interface endpoint for 5-10x latency reduction.
Reuse SDK clients across invocations. Never create a new client per call. In Lambda, initialize at module scope outside the handler.
Do not optimize container language for latency. Pick what your team knows. The platform routing floor makes it invisible.
Use VPC-MODE for security, not performance. Identical once warm.
Consolidate multi-tool agents into fewer runtimes. Each sequential InvokeAgentRuntime call adds platform routing overhead.
Use streaming for better perceived performance. Model inference is the bottleneck.

Code Configuration Support

Language	Code Config (zip to S3)	Container Config (Docker)
Python	Yes (3.10 through 3.14)	Yes
Node.js	Yes (NODE_22)	Yes
Go	No	Yes
Java	No	Yes

Architecture

Caller (Lambda / ECS / App)
  -> [VPC Endpoint or Public Endpoint]
    -> AgentCore Platform (SigV4 validation, session lookup, routing)
      -> Container (PUBLIC: platform VM | VPC: ENI in your subnet)
        -> [Optional] Bedrock InvokeModel (model inference)

Source Code

All source code, raw data, Lambda functions, and charts are available here:

Neloh / agentcore-latency-benchmarks

AgentCore Runtime latency benchmarking: Go, Node.js, Java, Python across PUBLIC/VPC modes

AgentCore Runtime Latency Benchmarks

Comprehensive latency benchmarking of AWS Bedrock AgentCore Runtime across 4 programming languages (Go, Node.js, Java, Python), multiple network configurations, and caller locations.

Key Findings

Factor	Impact on Warm Latency
Caller location (in-VPC vs external)	5-10x - the #1 factor
Model choice (for real agents)	Determines total time (95%+)
SDK cold start (Go/Node vs Python)	3x on first call only
Connection pool size	Mild at high concurrency
Container language	Zero impact
Runtime network mode (PUBLIC vs VPC)	Zero impact
Client SDK language (warm)	Zero impact

Test Results

1. Four-Language Echo Comparison (In-VPC)

All languages perform identically once warm. The platform routing floor (~80ms in-VPC) dominates.

2. In-VPC vs External Caller

Calling from within the VPC (via bedrock-agentcore interface endpoint) is 5-10x faster than calling over the public internet.

3. Full Comparison: Echo vs Real Agent (SME + Claude Haiku 4.5)

Model inference (Bedrock) accounts for 95%+ of…

View on GitHub

References

Chainlink on Docker: Part 1

Sibonelo N. — Thu, 29 Jun 2023 06:34:50 +0000

Adding off-chain data to a Blockchain network is not a straightforward process. For a Blockchain network to achieve Byzantine Fault tolerance, all the nodes or peer nodes in the network must unanimously agree on the state of the blockchain, also known as the world state. This means that the ledger, which contains the complete history of transactions, must always be consistent and verifiable by all nodes. No node can add a transaction to the ledger without the agreement of other peer nodes. To ensure resilience, every peer node maintains a copy of the ledger. Therefore, even if a particular node becomes unavailable due to network issues or malicious attacks, other nodes can continue to operate within the blockchain network and handle requests. If a compromised node tries to rejoin the network, it may need to be in-line with the blockchain rules (or policies) or be removed from the network through voting, especially if it continues to introduce conflicting transactions that deviate from the overall world state. In Blockchain, the principle of quorum rules applies, where the majority consensus is crucial, and mechanisms are in place to prevent a central entity from validating every transaction of each node.

In my own opinion, Chainlink seems to be the leading oracle that allows integration of data that is considered off-chain to the Blockchain network. With the consistent trend of Web3 and Generative AI it is important to know what Chainlink is. What is the role of Chainlink in various use cases such as Decentralized Finance, Non-Fungible Tokens (NFTs), Music and Gaming. In a overly simplified way, basically, an oracle or mainly a "decentralised oracle network" (DON) allows off-chain data to be integrated to a Blockchain network such as Ethereum Mainnet or the Testnets such as Sepolia.

The aim of this article is to cover the steps that one would follow when configuring a Chainlink node locally via Docker. Please use the link for the hardware requirements if you aim to set this up within your own machine. The steps are as follows for this article:

PostgreSQL database setup
Details on setting up an Alchemy or Infura free account for Sepolia, getting the API KEY.
Configure an Testnet environment via docker for Sepolia as well.
Attempt to fulfil requests with the Chainlink node through a Smart contract written in Solidity.
Validation of transactions via Etherscan.

Setting up PostgreSQL

You can pull an existing docker for PostgreSQL and setup the following commands to create a database user, password and the database where Chainlink will create the relational databases of what store Chainlink events, transactions and events.

docker run --name cl-postgres -v $HOME/.chainlink-sepolia/db:/var/lib/postgresql/data -e POSTGRES_PASSWORD=myPostgresPW -d -p 5432:5432 postgres:11.12

Once the above has been ran, you need to create the USER and database.

sudo docker exec -it cl-postgres psql -U postgres -c "CREATE USER $USERNAME WITH PASSWORD '$PASSWORD';"

sudo docker exec -it cl-postgres psql -U postgres -c "CREATE DATABASE "chainlink_sepoliadb";"

sudo docker exec -it cl-postgres psql -U postgres -c "GRANT ALL PRIVILEGES ON DATABASE "chainlink_sepoliadb" TO $USERNAME;"

The above is crucial as these PostgreSQL will be the database where the tables for Chainlink node are stored.

You can use the following ways to test if the PostgreSQL database is up and running

if you have psql CLI within your local machine, you can run the command psql postgresql://$USER:$PASSWORD@localhost:5432/chainlink_sepoliadb?sslmode=disable to see if you get logged in to the PostgreSQL shell where the database is the chainlink_sepolia as mentioned within the command. For an example, see this command below:

~/.chainlink-sepolia$ psql postgresql://$USER:$PASSWORD@localhost:5432/chainlink_sepoliadb?sslmode=disable
psql (14.8 (Ubuntu 14.8-0ubuntu0.22.04.1))
Type "help" for help.

chainlink_sepoliadb=>

Or you can use pgAdmin in order to connect to your localhost on port 5432 as it was exposed via Docker commands. You can do administrator related tasks on the database via this tool.

Signup for Alchemy Sepolia Testnet.

You need to get the Websocket URL of your Alchemy application. The .env file for the Chainlink node will need the Websocket URL one can always use HTTPS URL to interact with blockchain network however for the Chainlink setup, you need Websocket URL.

Setting up the Testnet client.

This is up to you. However, in this tutorial we are using the Sepolia Geth client. Using the free faucets to fund this type of testnet for fake currency seems to be currently supported as Goerli is deprecated. You can do the configuration for Goerli but the tutorial on official documentation places Sepolia network forward and regards Goerli deprecated and might have potential issues when setting up. One suggestion, is to run the commands that produce constant logs via the Unix screen command so that they execute in the background of each terminal.

As we did for PostgreSQL, you can configure the Geth client for Sepolia using the following steps:

docker pull ethereum/client-go:latest
mkdir ~/.geth-sepolia
docker run --name eth -p 8546:8546 -v ~/.geth-sepolia:/geth -it \ ethereum/client-go --sepolia --ws --ipcdisable \ --ws.addr 0.0.0.0 --ws.origins="*" --datadir /geth

If the container was stopped for any reason, you can restart it with the following command

sudo docker start -i eth

You can detach from the running client by using the commands Ctrl + P, Ctrl + Q as we do not want to stop it. Chainlink requires this container on running status in order to connect to it.

Setting up the Chainlink node

Once you have the docker up and running for PostgreSQL, Sepolia testnet and you have setup the hidden files .api file API KEY for Infura/Alchemy, .password file for your Chainlink UI and a .env file within a hidden directory .chainlink-sepolia. The .env will contain all your underlying network configurations such as the ChainID for the Testnet and the LINK address to use for your chainlink node. It might look like the following:

LOG_LEVEL=debug
ROOT=/chainlink
ETH_CHAIN_ID=11155111
CHAINLINK_TLS_PORT=0
SECURE_COOKIES=false
CHAINLINK_DEV=true
LINK_CONTRACT_ADDRESS=0x779877A7B0D9E8603169DdbD7836e478b4624789
ALLOW_ORIGINS=*
MIN_INCOMING_CONFIRMATIONS=1
ETH_URL=wss://eth-sepolia.g.alchemy.com/v2/RTDG...
DATABASE_URL=postgresql://$USER:$PASSWORD@localhost:5432/chainlink_sepoliadb?sslmode=disable

You need to make sure that you do not commit the above file to Github or any version control service. You start the chainlink node by running the following command.

sudo docker run --name chainlink_sepolia \ 
     --network host -p 6688:6688 \
     -v $HOME/.chainlink-sepolia:/chainlink \
     -it --env-file=$HOME/.chainlink-sepolia/.env \ 
      smartcontract/chainlink:1.1.0 local n -p \
      /chainlink/.password -a /chainlink/.api

This generates logs and push to a local folder in the same hidden directoty called, log.jsonl.

Once the docker containers are up and running, you can visit the URL http://localhost:6688 and you should be able to see the Chainlink UI tool. This is where you should locate the Chainlink key, along with the Job UI console where you would define your HTTP request syntax using TOML.

Setup a Smart contract within Solidity for a basic HTTP request (will be included in Part 2):

This section will be in part 2 of this article. I will discuss the underlying smart contract that is responsible for reaching out an world-wide web service (via HTTPs) to fetch off-chain. Further I aim to discuss how Chainlink works with the on-chain smart contracts to make sure the blockchain network remains deterministic. This part would also involve setting up a frontend for the project and cover security best practices.

Please note that the suggestions made here are purely from my understanding of the Chainlink software and can be corrected where misleading or false. The purpose is to learn and grow in the space of Web3, so comments are welcome for improvements.