Claim check pattern: large messages with S3 and SQS

The claim check pattern solves a problem that usually shows up in event-driven systems at the worst possible moment: a message suddenly becomes too large. In testing, the data was small. In production, someone uploads a six-hour bike ride with one GPS point per second, and the queue rejects it. Or SQS accepts it, but the next step in Step Functions fails because its limit is 256 KiB.

I care about this topic because the limits have grown over the last few years. SQS, EventBridge and asynchronous Lambda invocations now handle around one megabyte. That makes it tempting to consider the problem solved. In practice, though, a message chain is only as large as its smallest link, and large messages cost more in SQS than small ones. So in this article I will show you how to move large payloads to S3 cleanly and send only a small claim ticket through the queue.

In short

With the claim check pattern, you store large data in S3 and send only a reference with a checksum. That keeps you below every size limit in your chain, and SQS no longer bills you for each extra 64 KB chunk. The most common pitfall is cleanup: if you delete the objects too early, the payload is gone when a retry or a redrive from the dead-letter queue arrives.

What you will learn in this article

• How the claim check pattern works and when it still pays off despite 1 MiB limits
• A small TypeScript library for checkIn and checkOut with a SHA-256 checksum
• A complete Serverless Framework project with S3, SQS, DLQ and a lifecycle rule
• Why the lifecycle period must outlast the message lifetime in the DLQ

What is the claim check pattern?

Originally, the name comes from the book “Enterprise Integration Patterns” by Gregor Hohpe and Bobby Woolf. You know the idea from the airport: you drop off your heavy suitcase at the counter and get a small baggage tag. With the tag you travel light, and at your destination you pick the suitcase up again.

Applied to Serverless, this means the sender writes the actual payload (the business data) to storage, which on AWS is almost always an S3 bucket. To the queue or event bus it sends only a small envelope with the bucket, the object key and a checksum. Then the receiver reads the envelope, fetches the data from S3 and carries on. For every service in between, the message is tiny.

Three ingredients

To make this reliable, you need three ingredients. First, a threshold above which the payload is offloaded. Second, a checksum, so the receiver notices when the object no longer matches the ticket. Third, a cleanup rule that deletes old objects, but only once nobody needs them anymore.

🎬 Flow of the claim check pattern with S3 and SQS – this animated diagram only runs on my site: watch it on sascha-ms.de

When does the claim check pattern still pay off?

For a long time the answer was simple: anything above 256 KB did not fit into SQS, SNS or EventBridge. Today the picture is different. SQS allows messages up to 1 MiB, EventBridge accepts just under 1 MB per PutEvents call, and asynchronous Lambda invocations also take 1 MB. For SNS, the default per topic is still 256 KiB; you can raise it to up to 1 MiB, but then only SQS, Lambda and Firehose are allowed as subscribers. Step Functions stays at 256 KiB for inputs and outputs.

The smallest limit in the chain decides: Step Functions and SNS (default) are at 256 KiB.

That leaves four good reasons for the pattern, even when your data stays below one megabyte.

The chain is only as large as its smallest link

For instance, a message that only passes through SQS today may end up in a Step Functions execution or be distributed through an SNS topic tomorrow. If you offload from the start, you do not need to rebuild anything later. That is why the threshold in my example is 200 KiB, deliberately below the smallest limit in the chain.

Large messages cost more in SQS

SQS bills every started 64 KB chunk of a message as a separate request. So when you send a 1 MiB message, it counts as 16 requests. For large payloads, an S3 object plus a small claim ticket is often cheaper. Do the math for your volumes with the current SQS and S3 prices, because the break-even point depends on size and volume.

More than 1 MiB

For example, PDFs, images, exports or long series of measurements break even the new limits. At that point there is no way around offloading.

Less data in logs and dead-letter queues

If you log messages or inspect them in a DLQ, a claim ticket shows only references. Meanwhile, the actual data stays in the bucket, where you control access and encryption in one place. I find that especially helpful with personal data.

The example: ride logs for a bike rental

For this article I am making up a bike rental. In this scenario, every rental bike records its ride as a list of GPS points and uploads it after the bike is returned. An upload Lambda behind an HTTP API accepts the tour, responds immediately with 202 and hands it over to a processing Lambda through SQS. Afterwards, that function calculates the distance ridden. Short tours fit directly into the message, while long tours with thousands of points move to S3.

If the basic pattern with 202 and a queue is new to you, read the first part of the series about the asynchronous API with Lambda first. Here I focus on offloading.

Claim check pattern step by step with the Serverless Framework

Prerequisites

• Node.js 22 or newer and npm
• Serverless Framework v4 (npm install -g serverless), signed in with serverless login
• An AWS account with credentials for the eu-central-1 region
• Basic knowledge of Lambda and SQS; I explain the terms along the way

Step 1: Create the project

First, create an empty folder and install the dependencies. The AWS SDK v3 is included in the Lambda runtime, but I install it anyway so that types and tests work locally.

mkdir bike-rental-tours && cd bike-rental-tours
npm init -y
npm install @aws-sdk/client-s3 @aws-sdk/client-sqs
npm install -D typescript tsx @types/node @types/aws-lambda
mkdir -p src/handlers tests

The package.json gets "type": "module" and three scripts. The tsconfig.json allows imports with the .ts extension, so Node.js can run the tests directly through tsx.

{
  "name": "claim-check-s3-sqs",
  "version": "1.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "typecheck": "tsc --noEmit",
    "test": "node --test --import tsx tests/*.test.ts",
    "deploy": "serverless deploy"
  },
  "dependencies": {
    "@aws-sdk/client-s3": "^3.1164.0",
    "@aws-sdk/client-sqs": "^3.1164.0"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.164",
    "@types/node": "^22.10.0",
    "tsx": "^4.23.0",
    "typescript": "^5.9.3"
  }
}

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "allowImportingTsExtensions": true,
    "noEmit": true,
    "types": ["node"]
  },
  "include": ["src", "tests"]
}

Step 2: The envelope and the checkIn and checkOut functions

At its core, the solution is a small file without any AWS dependency. checkIn measures the payload in bytes, not in characters. That matters, because an “ä” takes two bytes in UTF-8. If the size is below the threshold, the payload stays inside the envelope. Otherwise the function computes a SHA-256 checksum, stores the object under exactly this hash and returns a claim ticket.

Using the hash as the key has a nice side effect: if a client sends the same tour twice, it ends up in the same object. The upload is therefore idempotent, which means you can repeat it as often as you like without changing the result.

// src/claim-check.ts
import { createHash } from 'node:crypto';

// Storage for large payloads (S3 in Lambda, an in-memory map in tests)
export interface PayloadStore {
  put(key: string, body: string): Promise<void>;
  get(key: string): Promise<string>;
}

export type Envelope =
  | { v: 1; kind: 'inline'; body: string }
  | { v: 1; kind: 'claim'; bucket: string; key: string; bytes: number; sha256: string };

export interface CheckInOptions {
  store: PayloadStore;
  bucket: string;
  thresholdBytes?: number;
  prefix?: string;
}

// Deliberately below 256 KiB: the smallest limit on the way (SNS default, Step Functions) wins
export const DEFAULT_THRESHOLD_BYTES = 200 * 1024;

const sha256Hex = (text: string): string =>
  createHash('sha256').update(text, 'utf8').digest('hex');

export async function checkIn(payload: string, options: CheckInOptions): Promise<Envelope> {
  const bytes = Buffer.byteLength(payload, 'utf8');
  const threshold = options.thresholdBytes ?? DEFAULT_THRESHOLD_BYTES;

  if (bytes <= threshold) {
    return { v: 1, kind: 'inline', body: payload };
  }

  // Content as key: the same payload always ends up in the same object (idempotent)
  const sha256 = sha256Hex(payload);
  const key = `${options.prefix ?? 'claims/'}${sha256}.json`;
  await options.store.put(key, payload);

  return { v: 1, kind: 'claim', bucket: options.bucket, key, bytes, sha256 };
}

export async function checkOut(envelope: Envelope, store: PayloadStore): Promise<string> {
  if (envelope.kind === 'inline') {
    return envelope.body;
  }

  const body = await store.get(envelope.key);
  if (sha256Hex(body) !== envelope.sha256) {
    throw new Error(`Checksum does not match object ${envelope.key}`);
  }
  return body;
}

export function parseEnvelope(raw: string): Envelope {
  let data: unknown;
  try {
    data = JSON.parse(raw);
  } catch {
    throw new Error('Message is not valid JSON');
  }

  const e = data as Record<string, unknown>;
  if (e?.v !== 1) {
    throw new Error('Unknown envelope version');
  }
  if (e.kind === 'inline' && typeof e.body === 'string') {
    return { v: 1, kind: 'inline', body: e.body };
  }
  if (
    e.kind === 'claim' &&
    typeof e.bucket === 'string' &&
    typeof e.key === 'string' &&
    typeof e.bytes === 'number' &&
    typeof e.sha256 === 'string' &&
    /^[a-f0-9]{64}$/.test(e.sha256)
  ) {
    return { v: 1, kind: 'claim', bucket: e.bucket, key: e.key, bytes: e.bytes, sha256: e.sha256 };
  }
  throw new Error('Envelope has an unknown format');
}

Moreover, the storage layer sits behind the PayloadStore interface. In Lambda that is S3, in the tests a simple map. This way you can test all of the logic without an AWS account.

Step 3: The S3 store

// src/s3-payload-store.ts
import { GetObjectCommand, PutObjectCommand, S3Client } from '@aws-sdk/client-s3';
import type { PayloadStore } from './claim-check.ts';

export class S3PayloadStore implements PayloadStore {
  constructor(
    private readonly bucket: string,
    private readonly s3: S3Client = new S3Client({}),
  ) {}

  async put(key: string, body: string): Promise<void> {
    await this.s3.send(
      new PutObjectCommand({
        Bucket: this.bucket,
        Key: key,
        Body: body,
        ContentType: 'application/json',
      }),
    );
  }

  async get(key: string): Promise<string> {
    const result = await this.s3.send(new GetObjectCommand({ Bucket: this.bucket, Key: key }));
    if (!result.Body) {
      throw new Error(`Objekt ${key} is empty`);
    }
    return result.Body.transformToString('utf-8');
  }
}

There is deliberately no delete command here. I explain why further down in the section on common mistakes.

Step 4: The tour model

By contrast, the business logic is intentionally simple: validate the tour and calculate the distance with the haversine formula, which gives the distance between two points on a sphere.

// src/tour.ts
export interface TrackPoint {
  lat: number;
  lon: number;
  t: string;
}

export interface Tour {
  tourId: string;
  bikeId: string;
  points: TrackPoint[];
}

export function parseTour(raw: string): Tour {
  const data = JSON.parse(raw) as Partial<Tour>;
  if (typeof data.tourId !== 'string' || typeof data.bikeId !== 'string') {
    throw new Error('tourId and bikeId are required');
  }
  if (!Array.isArray(data.points) || data.points.length < 2) {
    throw new Error('A tour needs at least two points');
  }
  return data as Tour;
}

// Distance between two coordinates in kilometers (haversine formula)
function haversineKm(a: TrackPoint, b: TrackPoint): number {
  const rad = (deg: number) => (deg * Math.PI) / 180;
  const dLat = rad(b.lat - a.lat);
  const dLon = rad(b.lon - a.lon);
  const h =
    Math.sin(dLat / 2) ** 2 +
    Math.cos(rad(a.lat)) * Math.cos(rad(b.lat)) * Math.sin(dLon / 2) ** 2;
  return 2 * 6371 * Math.asin(Math.sqrt(h));
}

export function distanceKm(tour: Tour): number {
  let sum = 0;
  for (let i = 1; i < tour.points.length; i++) {
    sum += haversineKm(tour.points[i - 1], tour.points[i]);
  }
  return Math.round(sum * 100) / 100;
}

Step 5: The upload Lambda

First, the upload function validates the tour. It then calls checkIn and sends the envelope to the queue. The makeUploadTour factory receives the store and the send function as parameters. That makes the handler testable without mocking the SDK.

// src/handlers/upload-tour.ts
import { SendMessageCommand, SQSClient } from '@aws-sdk/client-sqs';
import type { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from 'aws-lambda';
import { checkIn, type PayloadStore } from '../claim-check.ts';
import { S3PayloadStore } from '../s3-payload-store.ts';
import { parseTour } from '../tour.ts';

const BUCKET = process.env.PAYLOAD_BUCKET ?? '';
const QUEUE_URL = process.env.TOUR_QUEUE_URL ?? '';

type Send = (body: string) => Promise<void>;

export function makeUploadTour(store: PayloadStore, send: Send) {
  return async (event: APIGatewayProxyEventV2): Promise<APIGatewayProxyResultV2> => {
    const raw = event.isBase64Encoded
      ? Buffer.from(event.body ?? '', 'base64').toString('utf8')
      : (event.body ?? '');

    let tourId: string;
    try {
      tourId = parseTour(raw).tourId;
    } catch (err) {
      return { statusCode: 400, body: JSON.stringify({ message: (err as Error).message }) };
    }

    const envelope = await checkIn(raw, { store, bucket: BUCKET });
    await send(JSON.stringify(envelope));

    return {
      statusCode: 202,
      body: JSON.stringify({ tourId, stored: envelope.kind }),
    };
  };
}

const sqs = new SQSClient({});
export const handler = makeUploadTour(new S3PayloadStore(BUCKET), async (body) => {
  await sqs.send(new SendMessageCommand({ QueueUrl: QUEUE_URL, MessageBody: body }));
});

Step 6: The processing Lambda

Next, the second function reads messages from SQS. It unpacks the envelope, fetches the payload from S3 when needed, verifies the checksum and calculates the distance. If a single message fails, it reports only that one through partial batch response. The rest of the batch counts as done.

// src/handlers/process-tour.ts
import type { SQSBatchResponse, SQSEvent } from 'aws-lambda';
import { checkOut, parseEnvelope, type PayloadStore } from '../claim-check.ts';
import { S3PayloadStore } from '../s3-payload-store.ts';
import { distanceKm, parseTour } from '../tour.ts';

export function makeProcessTour(store: PayloadStore) {
  return async (event: SQSEvent): Promise<SQSBatchResponse> => {
    const batchItemFailures: SQSBatchResponse['batchItemFailures'] = [];

    for (const record of event.Records) {
      try {
        const envelope = parseEnvelope(record.body);
        const tour = parseTour(await checkOut(envelope, store));
        console.log(
          JSON.stringify({
            msg: 'Tour processed',
            tourId: tour.tourId,
            bikeId: tour.bikeId,
            points: tour.points.length,
            km: distanceKm(tour),
            via: envelope.kind,
          }),
        );
      } catch (err) {
        console.error(JSON.stringify({ msg: 'Tour failed', id: record.messageId, error: (err as Error).message }));
        // Only this message goes back to the queue, the rest of the batch counts as done
        batchItemFailures.push({ itemIdentifier: record.messageId });
      }
    }

    return { batchItemFailures };
  };
}

export const handler = makeProcessTour(new S3PayloadStore(process.env.PAYLOAD_BUCKET ?? ''));

Step 7: Infrastructure in serverless.yml

The serverless.yml creates both functions, the bucket, the queue and a dead-letter queue. Serverless Framework v4 builds TypeScript with esbuild automatically, so you do not need a plugin. Watch out for three details: the IAM permissions apply only to the claims/ prefix, a bucket policy denies unencrypted connections, and the lifecycle rule deletes objects after 15 days.

# serverless.yml
service: bike-rental-tours
frameworkVersion: '4'

provider:
  name: aws
  runtime: nodejs24.x
  region: eu-central-1
  architecture: arm64
  memorySize: 512
  logRetentionInDays: 14
  environment:
    PAYLOAD_BUCKET: !Ref PayloadBucket
    TOUR_QUEUE_URL: !Ref TourQueue
  iam:
    role:
      statements:
        - Effect: Allow
          Action: s3:PutObject
          Resource: !Join ['', [!GetAtt PayloadBucket.Arn, '/claims/*']]
        - Effect: Allow
          Action: s3:GetObject
          Resource: !Join ['', [!GetAtt PayloadBucket.Arn, '/claims/*']]
        - Effect: Allow
          Action: sqs:SendMessage
          Resource: !GetAtt TourQueue.Arn

functions:
  uploadTour:
    handler: src/handlers/upload-tour.handler
    timeout: 10
    events:
      - httpApi:
          method: POST
          path: /tours
  processTour:
    handler: src/handlers/process-tour.handler
    timeout: 30
    events:
      - sqs:
          arn: !GetAtt TourQueue.Arn
          batchSize: 10
          functionResponseType: ReportBatchItemFailures

resources:
  Resources:
    PayloadBucket:
      Type: AWS::S3::Bucket
      Properties:
        PublicAccessBlockConfiguration:
          BlockPublicAcls: true
          BlockPublicPolicy: true
          IgnorePublicAcls: true
          RestrictPublicBuckets: true
        BucketEncryption:
          ServerSideEncryptionConfiguration:
            - ServerSideEncryptionByDefault:
                SSEAlgorithm: AES256
        LifecycleConfiguration:
          Rules:
            # Longer than the DLQ retention (14 days), otherwise the payload is missing on redrive
            - Id: claims-cleanup
              Status: Enabled
              Prefix: claims/
              ExpirationInDays: 15

    PayloadBucketPolicy:
      Type: AWS::S3::BucketPolicy
      Properties:
        Bucket: !Ref PayloadBucket
        PolicyDocument:
          Statement:
            - Sid: TlsOnly
              Effect: Deny
              Principal: '*'
              Action: 's3:*'
              Resource:
                - !GetAtt PayloadBucket.Arn
                - !Join ['', [!GetAtt PayloadBucket.Arn, '/*']]
              Condition:
                Bool:
                  aws:SecureTransport: 'false'

    TourDlq:
      Type: AWS::SQS::Queue
      Properties:
        MessageRetentionPeriod: 1209600 # 14 days

    TourQueue:
      Type: AWS::SQS::Queue
      Properties:
        VisibilityTimeout: 180 # 6 × function timeout
        RedrivePolicy:
          deadLetterTargetArn: !GetAtt TourDlq.Arn
          maxReceiveCount: 5

The queue’s visibility timeout is six times the function timeout. AWS recommends this for Lambda with SQS, so that a message is not processed twice while the first invocation is still running.

Step 8: Test locally

For the tests, I use the built-in Node.js test runner. Instead of S3, an in-memory store does the job:

// tests/memory-store.ts
import type { PayloadStore } from '../src/claim-check.ts';

export class MemoryStore implements PayloadStore {
  readonly objects = new Map<string, string>();

  async put(key: string, body: string): Promise<void> {
    this.objects.set(key, body);
  }

  async get(key: string): Promise<string> {
    const body = this.objects.get(key);
    if (body === undefined) throw new Error(`NoSuchKey: ${key}`);
    return body;
  }
}

// tests/claim-check.test.ts
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { checkIn, checkOut, parseEnvelope } from '../src/claim-check.ts';
import { MemoryStore } from './memory-store.ts';

const options = (store: MemoryStore) => ({ store, bucket: 'test-bucket', thresholdBytes: 1024 });

test('small message stays inside the envelope', async () => {
  const store = new MemoryStore();
  const envelope = await checkIn('{"hello":"world"}', options(store));
  assert.equal(envelope.kind, 'inline');
  assert.equal(store.objects.size, 0);
});

test('large message moves to S3 and comes back unchanged', async () => {
  const store = new MemoryStore();
  const payload = JSON.stringify({ text: 'x'.repeat(5000) });
  const envelope = await checkIn(payload, options(store));
  assert.equal(envelope.kind, 'claim');
  assert.equal(store.objects.size, 1);
  assert.equal(await checkOut(parseEnvelope(JSON.stringify(envelope)), store), payload);
});

test('same content yields the same key', async () => {
  const store = new MemoryStore();
  const payload = 'y'.repeat(2000);
  const a = await checkIn(payload, options(store));
  const b = await checkIn(payload, options(store));
  assert.deepEqual(a, b);
  assert.equal(store.objects.size, 1);
});

test('umlauts count as two bytes', async () => {
  const store = new MemoryStore();
  // 600 characters, but 1200 bytes in UTF-8
  const envelope = await checkIn('ä'.repeat(600), options(store));
  assert.equal(envelope.kind, 'claim');
  if (envelope.kind === 'claim') assert.equal(envelope.bytes, 1200);
});

test('modified content in S3 is detected', async () => {
  const store = new MemoryStore();
  const envelope = await checkIn('z'.repeat(2000), options(store));
  if (envelope.kind !== 'claim') throw new Error('claim expected');
  store.objects.set(envelope.key, 'tampered');
  await assert.rejects(checkOut(envelope, store), /Checksum does not match/);
});

test('broken envelopes are rejected', () => {
  assert.throws(() => parseEnvelope('not json'), /not valid JSON/);
  assert.throws(() => parseEnvelope('{"v":2}'), /Unknown envelope version/);
  assert.throws(() => parseEnvelope('{"v":1,"kind":"claim","key":"a"}'), /unknown format/);
});

You can start the type check and the tests like this:

npm run typecheck
npm test

Finally, the expected result is # pass 6 and # fail 0 at the end of the output.

Step 9: Deploy and try it out

serverless deploy

# Create a long tour with 4,000 points (about 240 KB)
node -e "const p=Array.from({length:4000},(_,i)=>({lat:52.52+i*0.0001,lon:13.405,t:new Date(Date.UTC(2026,10,8,10,0,i)).toISOString()}));require('fs').writeFileSync('tour.json',JSON.stringify({tourId:'tour-4711',bikeId:'bike-17',points:p}))"

curl -s -X POST "https://<your-api-id>.execute-api.eu-central-1.amazonaws.com/tours" \
  -H "Content-Type: application/json" --data-binary @tour.json

serverless logs -f processTour --tail

The API responds with {"tourId":"tour-4711","stored":"claim"}. Shortly afterwards, a line with "via":"claim" and about 44 kilometers appears in the processing log. If you send a short tour with three points, the response says "stored":"inline", and no object is created in the bucket.

Common mistakes when offloading

Deleting the object right after processing

At first, this looks tidy, but it is risky. SQS delivers messages at least once, occasionally twice. If a retry runs after the object has been deleted, it fails. It gets worse with the dead-letter queue: when you redrive stranded messages from there, they point to nothing. That is why I leave cleanup to a lifecycle rule.

Choosing a lifecycle period that is too short

Therefore, the period must be longer than the maximum lifetime of a message. With standard queues, a message keeps its original timestamp when it moves to the DLQ, so it lives at most as long as the DLQ retention period, which is 14 days at most. With FIFO queues, the clock restarts in the DLQ. In the worst case, a message there can reach the retention of the source queue plus that of the DLQ, so up to 28 days. With 15 days you are on the safe side for the standard example, while FIFO needs more.

Counting characters instead of bytes

payload.length counts UTF-16 code units. The AWS limits, however, apply to bytes. With umlauts, emojis or Chinese place names, you end up just above the limit and wonder why. Buffer.byteLength(payload, 'utf8') measures correctly.

Putting presigned URLs into the message

At first glance, sending a presigned URL instead of bucket and key sounds convenient. Such URLs expire, though, and they work for anyone who finds them in a log. Permissions belong to the receiver’s role, not inside the message.

Tying the threshold to the wrong limit

If you set the threshold to 1 MiB because SQS can handle it, you run into trouble as soon as the same envelope passes through Step Functions or an SNS topic with the default limit. Also keep message attributes in mind, because SQS counts them towards the size.

Costs and security

For costs, requests matter most. Every offloaded message creates one PUT and at least one GET in S3, plus storage for at most 15 days. On the other hand, small messages stay inline and cost nothing extra. For large messages, however, you save the additional 64 KB chunks in SQS. Whether this pays off for you depends on the ratio of size and volume; you can find the current prices on the SQS and S3 pricing pages.

For security, three points matter to me. The bucket is fully private and encrypted with S3-managed keys. If you need your own KMS keys, add kms:Decrypt for the receiver. In addition, the roles may only read and write under claims/. And the checksum protects against swapped or modified objects. It does not replace permissions, but it makes errors visible immediately instead of later in processing.

What about the Extended Client Libraries?

AWS offers ready-made Extended Client Libraries for SQS and SNS that implement exactly this pattern and move payloads of up to 2 GB through S3. However, they only exist for Java and Python, and according to the documentation only for synchronous clients. For TypeScript in Lambda, I prefer to write those few lines myself. That way I decide on the envelope format, key and checksum, and I can send the envelope unchanged through SQS, EventBridge or Step Functions. By the way, the format in this article is not compatible with the Java and Python libraries. If a Java service using the library is waiting on the other side, you are better off with its format.

My take

I consider the claim check pattern one of the underrated patterns in Serverless architectures. It is unspectacular, quick to build and saves you from a whole class of errors that only show up under load. Admittedly, the larger limits in SQS and EventBridge are progress, but they only move the boundary. That is why today I would send every message whose size depends on user input through an envelope like this one.

Where I disagree: the pattern is no excuse for stuffing events with everything that happens to be at hand. After all, good events stay lean and describe what happened. Consequently, large data only belongs in them when the receiver really needs it. In many cases, a business ID is enough, and the receiver loads whatever it needs on its own.

FAQ

How large can an SQS message be?

Since 2025, up to 1 MiB (1,048,576 bytes), including message attributes. Every started 64 KB chunk is billed as a separate request.

What is the difference between claim check and the Extended Client Library?

The Extended Client Library is a ready-made implementation of the claim check pattern for Java and Python. However, the pattern itself is language-independent, and you can build it in TypeScript with a few lines of code.

When should I delete the objects in S3?

Ideally, you use a lifecycle rule whose period is longer than the maximum lifetime of the message, including the dead-letter queue. Deleting right after processing breaks retries and redrives.

Does the pattern also work with EventBridge and Step Functions?

Yes. Because the envelope is plain JSON, it fits into the detail of an event or the input of a Step Functions execution. Especially with Step Functions and its 256 KiB limit, this is the usual approach.

Your next step

To start, look for the messages in your system whose size depends on users: uploads, exports, free text, lists without an upper bound. Then measure the largest message of the past few weeks and compare it with the smallest limit in its chain. If it is above half of that limit, I would add the envelope now rather than after the first failure. Which message in your system last failed because of a size limit?

Continue the series: Go to the “Serverless patterns in practice” overview · Previous part: Human in the loop: approving AI agents with Step Functions · Next part: SSRF protection for webhooks: call user URLs safely

Sources

• Amazon SQS Developer Guide: message quotas (1 MiB)
• Amazon SQS Developer Guide: managing large messages with Amazon S3
• Amazon SQS Developer Guide: dead-letter queues and retention
• Amazon SQS pricing (billing in 64 KB chunks)
• Amazon EventBridge User Guide: PutEvents entry size
• Amazon SNS Developer Guide: large messages and MaximumMessageSize
• AWS Lambda Developer Guide: quotas (synchronous and asynchronous payload)
• AWS Step Functions Developer Guide: quotas (256 KiB)
• Enterprise Integration Patterns: Claim Check

This post was created with AI assistance.

---

👉 Next step: Need a schedule in EventBridge? My EventBridge cron helper builds and explains the expression for you.

📬 Stay updated: New articles on serverless, AI and leading tech teams via RSS, browser push or the weekly newsletter (in German).

Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en