Bedrock prompt caching: cut Claude costs with TypeScript

Bedrock prompt caching is, for me, the simplest lever to pull when an AI application built on Claude gets too expensive or too slow. Most assistants send the same material with every request: a long system prompt, rules, tool definitions, sometimes an entire handbook. The model reads that block again each time, and you pay for it in full each time. Yet often not a single character of it changes for weeks.

In this article we build a small help assistant for a fictional bike rental shop. It runs as a Lambda function deployed with the Serverless Framework, talks to Claude through the Converse API and keeps its static part in the cache. We also work out when caching pays off and measure the hit rate in CloudWatch. Spoiler: the most common reason for an empty cache is a single line containing the current date.

In short

Prompt caching stores the unchanged beginning of a prompt for a few minutes, so Bedrock doesn’t have to process it again on the next call. Cached tokens that are read cost only a fraction, writing costs a bit more than normal, which is why it pays off from the very first hit. The biggest pitfall is a prompt prefix that quietly changes with every request.

What you will learn in this article

• How prompt caching on Bedrock works and what a cache point is
• How to calculate the cost with and without the cache yourself
• How to arrange tools, system prompt and messages for good cache hits
• How to deploy a Lambda with a cache point using the Serverless Framework
• How to measure hit rate and savings in CloudWatch Logs Insights

What is Bedrock prompt caching?

A language model processes the entire prompt from start to finish on every request. With prompt caching, Bedrock remembers the intermediate result for part of it, and that part is always a prefix, meaning the beginning of the prompt up to a certain position. If the next request arrives with exactly the same beginning, the model picks up from there instead of computing everything again. That saves compute time and therefore money.

You mark the position up to which content should be cached with a cache point (in the Converse API it is called cachePoint). Everything before it is a candidate for the cache, everything after it is processed normally. In addition, every entry has a TTL (time to live). For Claude models the default is five minutes, and newer models optionally support one hour. Every hit resets the clock. An assistant that gets asked regularly therefore keeps its cache warm on its own.

Implicit or explicit caching?

Bedrock offers two variants. With implicit caching, the service tries to reuse matching prefixes by itself, without you configuring anything. That is convenient, but explicitly “best effort”: nothing is guaranteed, and you have no influence on what gets cached. With explicit caching, you set the cache points yourself. That doesn’t guarantee a hit either, but you decide which part is eligible in the first place. Bedrock supports both for Claude, and for an application whose costs you want to plan, I pick the explicit variant.

Another handy feature is simplified cache management for Claude: you place a single cache point at the end of your static content, and from there Bedrock looks back about 20 content blocks for the longest matching prefix. For most assistants, this one point is all you need.

What does prompt caching cost, and when does it pay off?

With caching there are three kinds of input tokens, and each has its own price. Normal tokens cost the usual input price. Tokens written to the cache cost a bit more with Claude. Tokens read from the cache cost much less. The following factors come from the Anthropic documentation and refer to the normal input price of the respective model. You’ll find the actual amounts for Bedrock on the Bedrock pricing page, and you should look them up for your model and your Region.

Token type	Factor of the input price	When does it apply?
Normal input	1	Everything after the cache point, and on a miss
Cache write, 5-minute TTL	1.25	First call with a new prefix
Cache write, 1-hour TTL	2	First call with a new prefix and ttl: "1h"
Cache read	0.1 (lower on some new models)	Every hit within the TTL

Price factors for Claude according to the Anthropic documentation, as of October 2026. Check per-model exceptions.

The break-even calculation

With these factors, the math is manageable. When writing with a five-minute TTL, you pay 0.25 more per token than without the cache. Every hit saves you 0.9 per token. So you earn back the extra cost with the very first hit. With the one-hour TTL, writing costs 1.0 extra, so you need two hits before it pays off.

An example: your assistant sends 1,000 input tokens, 960 of which are static context and 40 the actual question. On the first call this costs 40 + 960 × 1.25 = 1,240 units instead of 1,000. From the second call on it is 40 + 960 × 0.1 = 136 units. With 100 requests inside the cache window, you end up at just under 15 percent of the original input cost. Output tokens, the model’s answer, stay just as expensive, by the way. Caching only saves on input.

Besides money, there are two more effects. The answer arrives faster because the long beginning isn’t processed again. And according to the Bedrock documentation, cache hits with Claude are not counted against your rate limit. If you regularly run into throttling, that gives you extra headroom.

How to structure your prompt for Bedrock prompt caching

Bedrock processes the parts of a Converse request in a fixed order: first tools, then system, then messages. For the cache, this is a chain. If something further up changes, everything after it becomes invalid. A changed tool description therefore also invalidates the cached system prompt and the conversation history.

Tools and system prompt form the prefix; the cache point separates it from the question.

This leads to a simple rule: stable content to the front, changing content to the back. Tools and rules belong at the beginning, then comes the cache point, and only after that the question and the conversation history. Because the cache point after the system prompt covers the whole prefix, the tools are automatically included.

Minimum length and limits

A cache point only takes effect if the prefix before it is long enough. The minimum length depends on the model and counts cumulatively across tools, system prompt and messages. If the prefix is too short, the request still succeeds, just without the cache, and you get no error message. For example, the Bedrock documentation lists:

Model	Minimum tokens before the cache point	TTL
Claude Haiku 5.5, Sonnet 5.5, Opus 5.5	512	5 minutes or 1 hour
Claude Sonnet 4.6, Sonnet 4.5	1,024	5 minutes or 1 hour
Claude Haiku 4.5, Opus 4.6	4,096	5 minutes or 1 hour
Claude 3.7 Sonnet	1,024	5 minutes only

Excerpt from the Bedrock documentation, as of October 2026. The full list is linked under “Sources”.

Claude allows at most four cache points per request. You place the first one after the static part; you only need more when parts change at different rates, for instance a fixed handbook and a daily price list. One important detail: the one-hour TTL must come before the five-minute TTL, otherwise Bedrock rejects the request.

Prompt caching with Bedrock in TypeScript, step by step

Now it gets practical. Our example is the bike rental shop “Spokeworks”, which doesn’t exist. Its help assistant answers questions about prices, deposits and cancellations based on a fixed set of rules, and it can use a tool to check how many bikes are free on a given day. The rules and the tool definition are the static part; the customer’s question is the dynamic part.

Prerequisites

• Node.js 22 and npm
• An AWS account with credentials on your machine (for example via aws configure or AWS SSO)
• Serverless Framework v4 (npm i -g serverless; you need a free account to log in on first start)
• Access to Claude Haiku 5.5 in Amazon Bedrock in your Region

Step 1: Create the project

mkdir spokeworks-help && cd spokeworks-help
npm init -y
npm i @aws-sdk/client-bedrock-runtime
npm i -D typescript tsx vitest @types/node @types/aws-lambda

The package.json gets "type": "module" and three scripts for type checking, tests and the demo:

{
  "name": "spokeworks-help",
  "version": "1.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "typecheck": "tsc --noEmit",
    "test": "vitest run",
    "demo": "tsx scripts/cache-demo.ts"
  },
  "dependencies": {
    "@aws-sdk/client-bedrock-runtime": "^3.1156.0"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.164",
    "@types/node": "^22.0.0",
    "tsx": "^4.23.15",
    "typescript": "^7.0.2",
    "vitest": "^5.0.5"
  }
}

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "noEmit": true,
    "types": ["node"]
  },
  "include": ["src", "test", "scripts"]
}

Step 2: Write the static context

The rules live in a separate file. Make sure nothing in it changes per request. A version label like “as of 2026-10” is fine, a timestamp with the time of day is not. With roughly 2,500 characters plus the tool definition, we end up above the 512-token minimum.

// src/handbook.ts
// Static context: rarely changes, so it goes at the front of the prompt.
export const HANDBOOK_VERSION = "2026-10";

export const handbook = `
You are the help assistant of the bike rental shop "Spokeworks".
Answer in a friendly, concise way and in English. Never invent rules.
If a question is not answered by the rules below, refer the customer
to the team in the shop.

Rental rules (as of ${HANDBOOK_VERSION}):
1. Renting is possible from age 16; under 18 only with the consent
   of a parent or legal guardian.
2. At pickup, the customer shows a photo ID.
3. City bikes cost 14 euros per day, e-bikes 32 euros, cargo bikes 39 euros.
4. Rentals of seven days or more get a 15 percent discount on the total price.
5. The deposit is 50 euros for city bikes and 200 euros for e-bikes
   and cargo bikes. It is refunded immediately if the bike is returned undamaged.
6. Reservations can be cancelled free of charge up to 24 hours before pickup.
   After that, we charge one rental day.
7. Bikes can be returned until 7 pm on the agreed day. Every started
   hour of delay costs 5 euros, at most one rental day.
8. Helmets are free, child seats cost 4 euros per day, panniers
   3 euros per day. Accessories must be listed with the reservation.
9. In case of a breakdown, the customer calls the breakdown service.
   Within the city limits we swap the bike free of charge
   within 90 minutes.
10. Theft must be reported to the police immediately and to us with the
    case number. Without a police report, the current value of the bike is due.
11. E-bike batteries are fully charged at pickup. A spare battery costs
    6 euros per day and is only available with a reservation.
12. Cargo bikes may carry at most 80 kilograms or two children
    wearing seat belts.
13. Trips outside Germany are only allowed after written agreement.
14. Groups of six bikes or more register at least three days in advance.
    Groups get a free 15-minute introduction.
15. Payment is by card at pickup. We only accept cash for the deposit
    on city bikes.
16. In case of rain or storm, the reservation can be moved by up to three
    days free of charge on the pickup day if the weather service has issued
    an official warning for the city.

Answer format:
- Start with the direct answer in one sentence.
- Then name the matching rule number in parentheses, for example (rule 6).
- Use the tool "check_availability" when asked about free bikes on a
  date. Never guess stock levels.
`.trim();

Step 3: Build the request with a cache point

The core is small. The system prompt consists of two blocks: the text and, directly after it, the cachePoint. I only set the TTL when one hour is explicitly requested. For models that only support five minutes, the field would otherwise cause a ValidationException.

// src/prompt.ts
import type {
  ConverseCommandInput,
  Message,
  SystemContentBlock,
  Tool,
} from "@aws-sdk/client-bedrock-runtime";
import { handbook } from "./handbook.js";

export type CacheTtl = "5m" | "1h";

// Rough rule of thumb: about four characters per token. Good enough for a
// warning; for billing, only the values in the response count.
export function estimateTokens(text: string): number {
  return Math.ceil(text.length / 4);
}

export const tools: Tool[] = [
  {
    toolSpec: {
      name: "check_availability",
      description:
        "Checks how many bikes of a type are free on a date. " +
        "Use the tool only for specific dates in the format YYYY-MM-DD.",
      inputSchema: {
        json: {
          type: "object",
          properties: {
            date: { type: "string", description: "Date in the format YYYY-MM-DD" },
            bikeType: { type: "string", enum: ["city", "e-bike", "cargo"] },
          },
          required: ["date", "bikeType"],
        },
      },
    },
  },
];

export function buildRequest(
  modelId: string,
  history: Message[],
  ttl: CacheTtl = "5m",
): ConverseCommandInput {
  // Cache order: tools -> system -> messages.
  // The cache point after the system prompt covers the tools AND the handbook.
  const system: SystemContentBlock[] = [
    { text: handbook },
    { cachePoint: ttl === "1h" ? { type: "default", ttl: "1h" } : { type: "default" } },
  ];

  return {
    modelId,
    system,
    toolConfig: { tools },
    messages: history,
    inferenceConfig: { maxTokens: 400, temperature: 0.2 },
  };
}

Step 4: Evaluate the cost from the response

In the usage field, the Converse API also returns cacheReadInputTokens and cacheWriteInputTokens. There is a trap in inputTokens: with caching enabled, the field only counts the tokens that were neither read nor written. The total is the sum of all three fields. This small function turns them into hit rate and relative cost:

// src/usage.ts
import type { TokenUsage } from "@aws-sdk/client-bedrock-runtime";

// Factors relative to the normal input price. Check them for your model
// on the Bedrock pricing page; they differ between models.
export interface PriceFactors {
  write: number; // e.g. 1.25 with a 5-minute TTL
  read: number; // e.g. 0.1
}

export interface CacheReport {
  totalInput: number;
  fromCache: number;
  writtenToCache: number;
  hitRate: number; // share of input tokens read from cache, 0 to 1
  relativeCost: number; // input cost relative to "no cache", 1 = same price
}

export function evaluate(usage: Partial<TokenUsage>, factors: PriceFactors): CacheReport {
  // With caching enabled, inputTokens only contains the NON-cached tokens.
  const normal = usage.inputTokens ?? 0;
  const read = usage.cacheReadInputTokens ?? 0;
  const written = usage.cacheWriteInputTokens ?? 0;
  const total = normal + read + written;

  if (total === 0) {
    return { totalInput: 0, fromCache: 0, writtenToCache: 0, hitRate: 0, relativeCost: 1 };
  }

  const costWithCache = normal + written * factors.write + read * factors.read;

  return {
    totalInput: total,
    fromCache: read,
    writtenToCache: written,
    hitRate: read / total,
    relativeCost: costWithCache / total,
  };
}

// After how many hits does a cache entry pay off?
// Extra cost when writing: (write - 1). Saving per hit: (1 - read).
export function hitsToBreakEven(factors: PriceFactors): number {
  return Math.ceil((factors.write - 1) / (1 - factors.read));
}

Step 5: The Lambda handler with a tool loop

The handler accepts a question, calls Claude and runs the tool when needed. This is where caching shows its value particularly well: if the model asks for free bikes, a second call with the same prefix follows, and it already reads from the cache. Every round writes a log line in JSON format that we analyze later.

// src/handler.ts
import {
  BedrockRuntimeClient,
  ConverseCommand,
  type ContentBlock,
  type Message,
} from "@aws-sdk/client-bedrock-runtime";
import type { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from "aws-lambda";
import { buildRequest, estimateTokens, type CacheTtl } from "./prompt.js";
import { handbook } from "./handbook.js";
import { evaluate } from "./usage.js";

const client = new BedrockRuntimeClient({});
const MODEL_ID = process.env.MODEL_ID ?? "global.anthropic.claude-haiku-5-5";
const CACHE_TTL = (process.env.CACHE_TTL ?? "5m") as CacheTtl;
const MIN_TOKENS = Number(process.env.CACHE_MIN_TOKENS ?? "512");
const FACTORS = { write: CACHE_TTL === "1h" ? 2 : 1.25, read: 0.1 };

// Check once per cold start whether the handbook is long enough at all.
if (estimateTokens(handbook) < MIN_TOKENS) {
  console.warn(JSON.stringify({ note: "Prompt probably too short for the cache", minTokens: MIN_TOKENS }));
}

// Example tool: in real life you would query your database here.
function checkAvailability(input: { date: string; bikeType: string }) {
  const stock: Record<string, number> = { city: 12, "e-bike": 4, cargo: 1 };
  return { date: input.date, bikeType: input.bikeType, free: stock[input.bikeType] ?? 0 };
}

export async function handler(event: APIGatewayProxyEventV2): Promise<APIGatewayProxyResultV2> {
  const question = (JSON.parse(event.body ?? "{}") as { question?: string }).question?.trim();
  if (!question) {
    return { statusCode: 400, body: JSON.stringify({ error: "Field 'question' is missing" }) };
  }

  const history: Message[] = [{ role: "user", content: [{ text: question }] }];

  // Small agent loop: at most three rounds with tool calls.
  for (let round = 1; round <= 3; round++) {
    const response = await client.send(new ConverseCommand(buildRequest(MODEL_ID, history, CACHE_TTL)));
    const usage = response.usage ?? {};
    console.log(JSON.stringify({ type: "cache-usage", round, modelId: MODEL_ID, ttl: CACHE_TTL, ...evaluate(usage, FACTORS) }));

    const message = response.output?.message;
    if (!message) break;
    history.push(message);

    if (response.stopReason !== "tool_use") {
      const text = (message.content ?? []).map((b) => b.text ?? "").join("").trim();
      return { statusCode: 200, body: JSON.stringify({ answer: text }) };
    }

    const results: ContentBlock[] = [];
    for (const block of message.content ?? []) {
      if (block.toolUse?.name === "check_availability") {
        const data = checkAvailability(block.toolUse.input as { date: string; bikeType: string });
        results.push({ toolResult: { toolUseId: block.toolUse.toolUseId, content: [{ json: data }] } });
      }
    }
    history.push({ role: "user", content: results });
  }

  return { statusCode: 502, body: JSON.stringify({ error: "No answer after three rounds" }) };
}

🎬 Two calls, one prefix – this animated diagram only runs on my site: watch it on sascha-ms.de

Step 6: Deploy with the Serverless Framework

For the infrastructure I use the Serverless Framework this time. Version 4 bundles TypeScript with esbuild on its own, so you don’t need a plugin. The IAM role deserves the most attention: for a global inference profile, the function needs permissions on the profile and on the model, both in your own Region and without a Region for the other destination Regions. The condition makes sure the model can only be invoked through this profile.

# serverless.yml
service: spokeworks-help
frameworkVersion: "4"

provider:
  name: aws
  runtime: nodejs22.x
  region: eu-central-1
  architecture: arm64
  memorySize: 512
  timeout: 30
  logRetentionInDays: 14
  environment:
    MODEL_ID: global.anthropic.claude-haiku-5-5
    CACHE_TTL: 5m
    CACHE_MIN_TOKENS: "512"
  iam:
    role:
      statements:
        # 1. The global inference profile itself
        - Effect: Allow
          Action: bedrock:InvokeModel
          Resource: arn:aws:bedrock:${aws:region}:${aws:accountId}:inference-profile/global.anthropic.claude-haiku-5-5
        # 2. The model in your own Region, only via this profile
        - Effect: Allow
          Action: bedrock:InvokeModel
          Resource: arn:aws:bedrock:${aws:region}::foundation-model/anthropic.claude-haiku-5-5
          Condition:
            StringEquals:
              bedrock:InferenceProfileArn: arn:aws:bedrock:${aws:region}:${aws:accountId}:inference-profile/global.anthropic.claude-haiku-5-5
        # 3. The model in all destination Regions of global inference
        - Effect: Allow
          Action: bedrock:InvokeModel
          Resource: arn:aws:bedrock:::foundation-model/anthropic.claude-haiku-5-5
          Condition:
            StringEquals:
              bedrock:InferenceProfileArn: arn:aws:bedrock:${aws:region}:${aws:accountId}:inference-profile/global.anthropic.claude-haiku-5-5

functions:
  help:
    handler: src/handler.handler
    events:
      - httpApi:
          path: /help
          method: post

npm run typecheck
npm test
serverless deploy

After the deployment, the framework prints the URL of the HTTP API. Use it to test the assistant:

curl -s -X POST https://YOUR-API-ID.execute-api.eu-central-1.amazonaws.com/help \
  -H "Content-Type: application/json" \
  -d '{"question":"What does an e-bike cost for eight days?"}'

Step 7: Check whether the cache works

The quickest way to see the effect is locally. The demo script sends two different questions right after each other, both with the same prefix:

// scripts/cache-demo.ts
// Sends two questions right after each other and prints the cache fields.
import { BedrockRuntimeClient, ConverseCommand } from "@aws-sdk/client-bedrock-runtime";
import { buildRequest } from "../src/prompt.js";

const client = new BedrockRuntimeClient({ region: process.env.AWS_REGION ?? "eu-central-1" });
const modelId = process.env.MODEL_ID ?? "global.anthropic.claude-haiku-5-5";

for (const question of ["What does a cargo bike cost for a week?", "Until when can I cancel for free?"]) {
  const start = Date.now();
  const r = await client.send(
    new ConverseCommand(buildRequest(modelId, [{ role: "user", content: [{ text: question }] }])),
  );
  const u = r.usage;
  console.log(
    `${question}\n  input=${u?.inputTokens} cacheWrite=${u?.cacheWriteInputTokens ?? 0} ` +
      `cacheRead=${u?.cacheReadInputTokens ?? 0} duration=${Date.now() - start} ms`,
  );
}

With npm run demo you should see a value for cacheWrite on the first call and the same value for cacheRead on the second. The exact numbers depend on the model’s tokenizer. If both lines show zero, the prefix is too short or the model doesn’t support explicit caching. The tests, by the way, check without AWS access that the cache point sits in the right place and the math is correct:

// test/prompt.test.ts
import { describe, expect, it } from "vitest";
import { buildRequest, estimateTokens } from "../src/prompt.js";
import { handbook } from "../src/handbook.js";

describe("buildRequest", () => {
  const history = [{ role: "user" as const, content: [{ text: "What does an e-bike cost?" }] }];

  it("puts the cache point right after the handbook", () => {
    const r = buildRequest("model", history);
    expect(r.system?.[0]).toEqual({ text: handbook });
    expect(r.system?.[1]).toEqual({ cachePoint: { type: "default" } });
  });

  it("only sets the TTL when one hour is requested", () => {
    const r = buildRequest("model", history, "1h");
    expect(r.system?.[1]).toEqual({ cachePoint: { type: "default", ttl: "1h" } });
  });

  it("keeps the messages out of the static part", () => {
    const r = buildRequest("model", history);
    expect(JSON.stringify(r.system)).not.toContain("e-bike cost?");
  });

  it("is long enough for the 512-token minimum", () => {
    expect(estimateTokens(handbook)).toBeGreaterThanOrEqual(512);
  });
});

// test/usage.test.ts
import { describe, expect, it } from "vitest";
import { hitsToBreakEven, evaluate } from "../src/usage.js";

describe("evaluate", () => {
  it("calculates a cache hit correctly", () => {
    const r = evaluate({ inputTokens: 40, cacheReadInputTokens: 960, cacheWriteInputTokens: 0, outputTokens: 80, totalTokens: 1080 }, { write: 1.25, read: 0.1 });
    expect(r.totalInput).toBe(1000);
    expect(r.hitRate).toBeCloseTo(0.96);
    expect(r.relativeCost).toBeCloseTo(0.136);
  });

  it("makes the first call more expensive because of the write", () => {
    const r = evaluate({ inputTokens: 40, cacheReadInputTokens: 0, cacheWriteInputTokens: 960, outputTokens: 80, totalTokens: 1080 }, { write: 1.25, read: 0.1 });
    expect(r.relativeCost).toBeCloseTo(1.24);
  });

  it("copes without usage fields", () => {
    expect(evaluate({}, { write: 1.25, read: 0.1 }).relativeCost).toBe(1);
  });
});

describe("hitsToBreakEven", () => {
  it("pays off from the first hit with a 5-minute TTL", () => {
    expect(hitsToBreakEven({ write: 1.25, read: 0.1 })).toBe(1);
  });
  it("needs two hits with a 1-hour TTL", () => {
    expect(hitsToBreakEven({ write: 2, read: 0.1 })).toBe(2);
  });
});

Measure the hit rate in CloudWatch Logs Insights

Because the handler writes one JSON line per round, CloudWatch Logs Insights detects the fields automatically. This query shows, per hour, how much input came from the cache and what you paid compared to “no cache”:

fields @timestamp, hitRate, relativeCost
| filter type = "cache-usage"
| stats avg(hitRate) as rate, avg(relativeCost) as cost, count(*) as calls by bin(1h)
| sort @timestamp desc

A hit rate around 0.9 under steady traffic is a good sign. If it suddenly drops, someone almost always changed the static part, or more than five minutes passed between two requests. If you have already set up alarms and dashboards for Lambda, you can add these values there. I show how to do that with CDK in Monitoring Lambda: Alarms, Dashboard and Budget with CDK.

Common prompt caching mistakes

A timestamp in the system prompt

“Today is Monday, October 26, 2:03 pm” at the very top of the system prompt is well meant and reliably destroys the cache. Every request then has a different beginning. If the date belongs in the context, put it in the user message after the cache point. The same applies to user names, session IDs and randomly sorted lists.

Tools that change per request

Some agents assemble the tool list depending on the user’s role. Because the tools sit at the very front, every combination gets its own cache entry, or there is no hit at all. A fixed list whose order never changes works better. Check what a user isn’t allowed to do inside the tool itself. According to the Anthropic documentation, a changed tool_choice and added or removed images also invalidate the cache for the messages.

Parallel requests right after startup

A cache entry only becomes available once the first response starts streaming. If you fire off ten requests at the same time after a deployment, in the worst case all ten write to the cache and pay the surcharge. For batch jobs it is therefore worth waiting for one request first and then continuing in parallel. With cross-Region inference, additional writes can also happen because requests are distributed across different Regions.

Misreading inputTokens

If you have calculated your costs from inputTokens so far, you will suddenly see tiny values after enabling the cache and celebrate too early. The field now only contains the non-cached remainder. For honest numbers you always need the sum of all three fields, as in evaluate above.

Choosing the one-hour TTL out of habit

An hour sounds like more hits, but it costs twice as much to write. If your assistant is asked several times within five minutes anyway, the short cache refreshes for free on every hit. The hour pays off when the gaps between requests are typically longer than five but shorter than 60 minutes, for example in a chat where someone goes to grab a coffee in between.

Keeping cost and security in view

The example only costs money when you call it: Lambda and the HTTP API bill per request, Bedrock per token. Still, delete the stack with serverless remove after trying it out, because the HTTP API is publicly reachable and every call costs tokens. For real use, put authentication in front of it, for example a JWT authorizer, plus a budget alarm in AWS Budgets. The cache itself is tied to your account. Even so, the usual rule applies: no personal data in the static part, because it is shared by all requests.

My take

Prompt caching is not a feature that needs a big celebration, and that is exactly why I like it. It is one cache point, a bit of discipline in how you structure the prompt and a log line to measure it. For every assistant with a long system prompt, it should be the default in my view, not the optimization for later.

Where I disagree is the idea that caching is a free pass for ever longer prompts. A handbook with 30,000 tokens gets cheaper through the cache, but not better. The model still has to take the content into account, and long contexts don’t automatically make answers more accurate. I would clean up the prompt first and cache it afterwards, not the other way around. And if you build agents with many tools, you benefit twice from clean, stable tool descriptions; I wrote about that in How to write tool descriptions for AI agents.

FAQ

Is prompt caching on Bedrock enabled by default?

Bedrock attempts implicit caching automatically for supported models, without a guarantee. For predictable hits, set a cachePoint after the static part yourself in the Converse API.

How long does a prompt stay in the cache?

For Claude, five minutes by default, and one hour on request with newer models. Every hit restarts the timer, so a regularly used cache stays warm.

Does prompt caching work with batch inference?

No. According to the Bedrock documentation, prompt caching is only available for on-demand inference, not for the batch API.

How do I recognize a cache hit?

By the cacheReadInputTokens field in the response. If it is greater than zero, part of the prompt was read from the cache. If instead there is a value in cacheWriteInputTokens, the prefix was just stored.

Your next step

Take your existing Bedrock call, look at which part of it is identical on every request, and place a single cachePoint after it. Then log the three usage fields for a week. If you need structured answers, you can combine this directly with Bedrock structured outputs with Claude and TypeScript. I’m curious: how long is the static part of your prompts, and have you ever measured what it costs you per month?

Sources

• Amazon Bedrock User Guide: Prompt caching for faster model inference
• Claude documentation: Prompt caching (price factors, cache invalidation, concurrent requests)
• Amazon Bedrock API Reference: TokenUsage
• Amazon Bedrock Pricing
• Serverless Framework v4 documentation

This post was created with AI assistance.

---

👉 Next step: Working with AWS CDK? Grab my printable AWS CDK cheat sheet with commands, concepts and a stack skeleton.

Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en