Imagine your users getting instant AI‑generated code suggestions without ever hitting your origin servers. By moving LLM calls to the edge you cut latency and offload traffic, while still keeping the power of Claude behind Bedrock.
Why Edge AI Matters
When a developer types a line of code and clicks “suggest”, they expect the answer in a blink. If the request has to travel from the user’s browser → your CDN → a central API server → the AI model and back, each hop adds milliseconds. Those milliseconds pile up into a noticeable lag that feels “slow”.
Think of the internet as a delivery network. A package shipped across the country takes longer than a package dropped off at the nearest local post office. Edge AI puts the “post office” right where the user is, so the “package” (the AI response) travels a much shorter distance.
Why this matters:
- User experience – sub‑second replies feel responsive, increasing adoption.
- Cost – fewer bytes cross the backbone, reducing data‑transfer bills.
- Scalability – the central backend sees less load, letting you serve more concurrent users.
In plain English: Moving the AI call to the edge is like moving a coffee shop from downtown to every neighborhood block—people get their coffee faster, and the downtown shop isn’t overwhelmed.
Creating a CloudFront Distribution with a Lambda Web Function Origin
CloudFront is Amazon’s global content‑delivery network (CDN). It caches static assets at edge locations and can also route dynamic requests to custom origins. A Lambda Web Function (LWF) is a lightweight Lambda that runs at an edge location and can serve as a origin for CloudFront, meaning CloudFront forwards the request to the LWF instead of a traditional web server.
The new “origin” type is handy because you no longer need a separate API gateway or EC2 instance just to forward a request to Bedrock. The LWF runs on the same edge node that serves the static assets, keeping everything together.
Step‑by‑step creation with the AWS SDK
Below is a minimal script that uses the @aws-sdk/client-cloudfront package to:
- Create an S3 bucket for static assets (optional, but often paired with a CDN).
- Deploy a Lambda Web Function (code is uploaded separately; we only reference its ARN here).
- Build a CloudFront distribution that points to the LWF as its origin.
// import the CloudFront client from the AWS SDK v3
import {
CloudFrontClient,
CreateDistributionCommand,
DistributionConfig,
} from "@aws-sdk/client-cloudfront";
// The ARN of the Lambda Web Function you deployed earlier.
// Example: arn:aws:lambda:us-east-1:123456789012:function:my-edge-lwf
const LWF_ARN = process.env.LWF_ARN!;
// Basic CloudFront config that uses the LWF as its origin
const distributionConfig: DistributionConfig = {
// A friendly name for the distribution
Comment: "Edge AI for Claude suggestions",
// Turn on the default cache behavior – it will forward all requests to the LWF
DefaultCacheBehavior: {
TargetOriginId: "LWFOrigin",
ViewerProtocolPolicy: "redirect-to-https",
// Allow all HTTP methods because our LWF expects POST /prompt
AllowedMethods: {
Quantity: 2,
Items: ["GET", "HEAD", "POST"],
// Cache only GET/HEAD; POST will be passed through
CachedMethods: { Quantity: 2, Items: ["GET", "HEAD"] },
},
// Forward the request body to the LWF (important for POST)
ForwardedValues: {
QueryString: true,
Cookies: { Forward: "all" },
// Enable body forwarding (requires Lambda@Edge or LWF)
// Note: This is automatically handled for LWF origins.
},
// Reduce TTL so that dynamic responses aren’t cached for too long
MinTTL: 0,
DefaultTTL: 0,
MaxTTL: 0,
},
// Define the LWF as an origin
Origins: {
Quantity: 1,
Items: [
{
Id: "LWFOrigin",
// The special domain name that points to the LWF
DomainName: `${LWF_ARN.split(":")[5]}.lambda-url.us-east-1.on.aws`,
// LWF origins don’t need an origin path
OriginPath: "",
CustomHeaders: { Quantity: 0 },
// The LWF runs over HTTPS; we don’t need an S3 or custom origin config
},
],
},
// Enable the distribution (true = live)
Enabled: true,
};
// Create the CloudFront client (region is ignored for global service)
const cfClient = new CloudFrontClient({ region: "us-east-1" });
async function createDistribution() {
try {
const cmd = new CreateDistributionCommand({ DistributionConfig: distributionConfig });
const response = await cfClient.send(cmd);
console.log("Distribution created with ID:", response.Distribution?.Id);
} catch (err) {
console.error("Failed to create distribution:", err);
process.exit(1);
}
}
// Run the function
createDistribution();
Key points in the code
-
TargetOriginIdlinks the cache behavior to the origin we defined. -
AllowedMethodsincludesPOSTbecause our LWF expects a JSON payload. -
MinTTL,DefaultTTL, andMaxTTLare set to0so that dynamic AI responses are never cached by CloudFront (you’ll see a dedicated caching strategy later).
Tip: CloudFront Functions (the ultra‑lightweight scripting option) cannot make network calls, so they’re not suitable for AI calls. Use a Lambda Web Function instead.
Gotchas you may hit
| Service | Gotcha | Why it matters |
|---|---|---|
| CloudFront Functions | 2 MB code limit, no network access | You can’t call Bedrock from a Function; you need LWF. |
| Lambda@Edge | 1 MB response limit, 128 MB memory max | Large streamed AI responses can be truncated; LWF lifts those limits. |
| Cache invalidation | Takes up to 60 seconds | Expect a short delay when you manually purge objects. |
| Origin Access Identity (OAI) | Deprecated but still allowed | Some teams keep OAI on old configs, leading to “Access Denied” errors. |
Calling Claude via Bedrock from the Edge with fetch
Bedrock is Amazon’s managed platform for large language models (LLMs) like Claude, Jurassic, etc. The service offers an HTTPS endpoint called invokeModel that you call with a JSON payload containing the prompt.
A Lambda Web Function runs in a sandbox that does not automatically load AWS credentials from the usual SDK chain (environment variables, EC2 metadata, etc.). If you try to use the standard Bedrock client inside the LWF, you’ll receive a 403 “Forbidden” error. The fix is to sign the HTTP request yourself using SigV4 (Signature Version 4) – the same signing process the SDK performs under the hood.
The @aws-sdk/signature-v4 package lets you create a signed fetch request in a few lines.
Full handler code
// Lambda Web Function entry point – Node.js 22 runtime
import { SignatureV4 } from "@aws-sdk/signature-v4";
import { Sha256 } from "@aws-crypto/sha256-browser";
import { defaultProvider } from "@aws-sdk/credential-provider-node";
// Bedrock model identifier – Claude‑instant‑1 in the us-east-1 region
const BEDROCK_MODEL_ID = "anthropic.claude-instant-v1";
// Bedrock endpoint – region‑specific
const BEDROCK_ENDPOINT = "https://bedrock-runtime.us-east-1.amazonaws.com";
// Helper: create a signed fetch request for Bedrock invokeModel
async function signedFetch(url, body) {
// Grab credentials from the default provider chain (access key, secret, token)
const credentials = await defaultProvider()();
// Build a SignatureV4 signer for the "bedrock" service
const signer = new SignatureV4({
service: "bedrock",
region: "us-east-1",
credentials,
sha256: Sha256,
});
// Create a raw HTTP request object that fetch understands
const request = {
method: "POST",
hostname: "bedrock-runtime.us-east-1.amazonaws.com",
path: url,
headers: {
"content-type": "application/json",
},
body: JSON.stringify(body),
};
// Sign the request – this adds the Authorization header and x‑amz‑date, etc.
const signed = await signer.sign(request);
// Perform the fetch with the signed headers
return fetch(`${BEDROCK_ENDPOINT}${url}`, {
method: "POST",
headers: signed.headers,
body: signed.body,
});
}
// Simple in‑memory cache for repeated prompts (see next section for more details)
const promptCache = new Map(); // key: prompt string, value: { response, expiresAt }
// Lambda Web Function handler – receives POST /prompt
export async function handler(event) {
try {
// Parse the incoming JSON body
const body = JSON.parse(event.body);
const userPrompt = body.prompt?.trim();
if (!userPrompt) {
return {
statusCode: 400,
body: JSON.stringify({ error: "Missing 'prompt' in request body" }),
};
}
// --------- Prompt caching (optional) ----------
const cached = promptCache.get(userPrompt);
if (cached && cached.expiresAt > Date.now()) {
// Return cached response quickly
return {
statusCode: 200,
body: JSON.stringify({ answer: cached.response, cached: true }),
};
}
// --------- Build Bedrock request ----------
const bedrockPayload = {
modelId: BEDROCK_MODEL_ID,
// Claude expects a "messages" array; we send a single user message
messages: [{ role: "user", content: userPrompt }],
// Request streaming output so we can forward chunks as they arrive
stream: true,
};
// Call Bedrock with a signed fetch
const response = await signedFetch("/model/invoke", bedrockPayload);
// Handle common error case – quota exceeded
if (response.status === 429) {
const errBody = await response.text();
console.error("Rate limit hit:", errBody);
return {
statusCode: 429,
body: JSON.stringify({ error: "ServiceQuotaExceededException: Too many requests" }),
};
}
// --------- Stream Bedrock's SSE back to the client ----------
// Bedrock returns Server‑Sent Events (SSE) when stream:true.
// We'll pipe those events directly through the edge.
const reader = response.body.getReader();
const encoder = new TextEncoder();
// Prepare a streaming response for CloudFront
const stream = new ReadableStream({
async start(controller) {
// Helper to parse SSE lines like "data: {...}"
const parseSSE = (chunk) => {
const text = new TextDecoder().decode(chunk);
const lines = text.split("\n");
for (const line of lines) {
if (line.startsWith("data:")) {
const json = line.slice(5).trim();
if (json !== "[DONE]") {
const data = JSON.parse(json);
// Claude returns the generated text in `completion`
const answer = data.completion;
controller.enqueue(encoder.encode(answer));
// Store in cache for next identical prompt (TTL 30 seconds)
promptCache.set(userPrompt, {
response: answer,
expiresAt: Date.now() + 30_000,
});
} else {
controller.close();
}
}
}
};
// Read chunks from Bedrock and feed them to the parser
while (true) {
const { done, value } = await reader.read();
if (done) break;
parseSSE(value);
}
},
});
// Return the streamed response to the browser
return {
statusCode: 200,
headers: { "content-type": "text/plain; charset=utf-8" },
body: stream,
};
} catch (err) {
console.error("Unexpected error:", err);
return {
statusCode: 500,
body: JSON.stringify({ error: "Internal server error" }),
};
}
}
Walk‑through of the most important lines
-
Credential fetching –
defaultProvider()reads the IAM role attached to the LWF. This works even though the LWF cannot use the normal SDK client automatically. -
Signature creation –
SignatureV4builds theAuthorizationheader required by Bedrock. Without it you’d get a 403. -
Streaming – Bedrock returns Server‑Sent Events (SSE). We read the raw bytes, split on newline, and forward only the
data:lines, which contain the generated text. This keeps the latency low because the client sees the answer as it’s produced. -
Error handling for quota – When Bedrock returns 429 we surface a clear message (
ServiceQuotaExceededException) so callers know they’ve hit the per‑minute token limit.
Key takeaway: The edge cannot rely on the SDK’s hidden signing; you must sign the request yourself. Think of the signature as a sealed envelope that proves “this request really comes from my AWS account”.
Gotchas specific to Bedrock
| Gotcha | What it looks like | How to avoid it |
|---|---|---|
| Token limits per minute | 429 responses even though each request only uses a few tokens | Batch identical prompts, use caching, monitor usage |
| SSE parsing | Missing the “data:” prefix leads to empty output | Always split on newline and ignore lines without data:
|
| Knowledge Base sync delay | New documents not searchable for up to several minutes | Warm‑up the cache or pre‑load prompts after updates |
| Model selection | Some models don’t support streaming, causing 400 errors | Verify stream:true is supported for the chosen model |
| Cross‑region latency | Calls to a Bedrock region far from the edge location can be >200 ms | Deploy the LWF in the same region as the Bedrock endpoint (e.g., us‑east‑1) |
Prompt Caching and Token Reuse at the Edge
Even with edge‑level latency, calling an LLM for every identical prompt is wasteful. Each call consumes tokens (the unit Bedrock counts for billing) and contributes to the per‑minute quota. A tiny cache that lives inside the Lambda Web Function’s memory can dramatically reduce repeat calls.
Simple in‑memory cache design
Because a Lambda Web Function may be reused for multiple requests (the “warm” container), an ordinary JavaScript Map is enough for short‑lived caching. We add an expiration timestamp so stale entries are purged automatically.
// Cache entry shape
// {
// response: string, // The text Claude generated
// expiresAt: number // Epoch ms when entry should be discarded
// }
// Global map – lives as long as the Lambda container stays warm
const promptCache = new Map();
/**
* Store a prompt/response pair for a limited time.
* @param {string} prompt - The exact user prompt string.
* @param {string} answer - Claude's generated answer.
* @param {number} ttlMs - Time‑to‑live in milliseconds (default 30 s).
*/
function cachePrompt(prompt, answer, ttlMs = 30_000) {
promptCache.set(prompt, {
response: answer,
expiresAt: Date.now() + ttlMs,
});
}
/**
* Retrieve a cached answer if it exists and is still fresh.
* @param {string} prompt
* @returns {string|undefined}
*/
function getCachedAnswer(prompt) {
const entry = promptCache.get(prompt);
if (entry && entry.expiresAt > Date.now()) {
return entry.response;
}
// Remove stale entry
if (entry) promptCache.delete(prompt);
return undefined;
}
Why cache at the edge?
If a user on a page repeatedly asks “How do I reverse a string in JavaScript?” the first request hits Bedrock, but the next three can be satisfied locally, saving both time and money.
Tip: Keep the TTL short (e.g., 30 seconds) to stay fresh with model updates while still catching rapid repeats.
Integrating the cache into the handler
In the handler shown earlier, we call getCachedAnswer right after parsing the request. If a fresh entry exists, we immediately return it with a small JSON wrapper indicating it came from the cache. When streaming, we store each chunk’s answer in the cache after the first full message is assembled.
Observability and Debugging with CloudWatch Logs
When your code runs at the edge, traditional local debugging isn’t possible. CloudWatch Logs is the primary way to see what’s happening inside a Lambda Web Function.
Sending logs from the edge
The LWF runtime provides a global console object that automatically writes to CloudWatch. Adding console.log statements at strategic points gives you visibility into:
- Incoming request payloads (sanitized for PII)
- Bedrock response status codes
- Cache hits vs. misses
- Errors such as
ServiceQuotaExceededException
export async function handler(event) {
console.log("Received event:", JSON.stringify(event)); // log request metadata
// ... after parsing prompt
console.log("User prompt:", userPrompt);
// Cache check
const cached = getCachedAnswer(userPrompt);
if (cached) {
console.log("Cache hit for prompt");
return {
statusCode: 200,
body: JSON.stringify({ answer: cached, cached: true }),
};
}
console.log("Cache miss – invoking Bedrock");
// After Bedrock call
console.log("Bedrock HTTP status:", response.status);
if (response.status === 429) {
console.warn("Quota exceeded for Bedrock");
}
// Inside SSE parsing loop
console.log("Received chunk from Bedrock:", chunk.length, "bytes");
// ... rest of handler
}
Monitoring latency and errors
Create a CloudWatch metric filter that looks for Bedrock HTTP status: 5.. or 429 in the logs. Then set up an alarm that notifies you (e.g., via SNS) when the error rate spikes. This helps you catch sudden quota exhaustion or network hiccups before users notice.
Debugging tip for the 403 “Forbidden” error
If you see a 403 in the logs, double‑check two things:
- The request is signed (
Authorizationheader present). You can printsigned.headersbefore thefetchcall. - The IAM role attached to the Lambda Web Function has
bedrock-runtime:InvokeModelpermission for the specific model ARN.
In plain English: A 403 means “AWS doesn’t think this request belongs to you”. Make sure the signature is there and the role is allowed to call Claude.
The Takeaway
- Edge AI moves the expensive LLM call from a central server to the CDN, delivering sub‑second answers and reducing backend load.
- A Lambda Web Function can act as a CloudFront origin, but it cannot rely on the SDK’s automatic credential chain. Use @aws-sdk/signature-v4 to sign a raw
fetchrequest to Bedrock. - Stream Bedrock’s Server‑Sent Events directly back to the browser so users see the answer as it’s generated.
- A lightweight in‑memory cache at the edge saves tokens, respects per‑minute limits, and speeds up repeated prompts.
- CloudWatch logs and metric filters give you the only window into what’s happening on the edge; watch for 403 (missing signature) and 429 (quota) responses.
With these pieces in place, you can build interactive developer tools, code assistants, or any real‑time AI feature that feels local to the user, while still leveraging the strong language capabilities of Claude on AWS Bedrock. Happy coding at the edge!
Transparency notice
This article was written with the help of an AI system — Groq (GPT OSS 120B).
Published: 2026-10-13 · Primary focus: CloudFront
All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.Find an error? Drop a comment — corrections are always welcome.
Top comments (0)