The claim check pattern solves a problem that usually shows up in event-driven systems at the worst possible moment: a message suddenly becomes too large. In testing, the data was small. In production, someone uploads a six-hour bike ride with one GPS point per second, and the queue rejects it. Or SQS accepts it, but the next step in Step Functions fails because its limit is 256 KiB.

I care about this topic because the limits have grown over the last few years. SQS, EventBridge and asynchronous Lambda invocations now handle around one megabyte. That makes it tempting to consider the problem solved. In practice, though, a message chain is only as large as its smallest link, and large messages cost more in SQS than small ones. So in this article I will show you how to move large payloads to S3 cleanly and send only a small claim ticket through the queue.

In short

With the claim check pattern, you store large data in S3 and send only a reference with a checksum. That keeps you below every size limit in your chain, and SQS no longer bills you for each extra 64 KB chunk. The most common pitfall is cleanup: if you delete the objects too early, the payload is gone when a retry or a redrive from the dead-letter queue arrives.

What you will learn in this article

  • How the claim check pattern works and when it still pays off despite 1 MiB limits
  • A small TypeScript library for checkIn and checkOut with a SHA-256 checksum
  • A complete Serverless Framework project with S3, SQS, DLQ and a lifecycle rule
  • Why the lifecycle period must outlast the message lifetime in the DLQ

What is the claim check pattern?

Originally, the name comes from the book “Enterprise Integration Patterns” by Gregor Hohpe and Bobby Woolf. You know the idea from the airport: you drop off your heavy suitcase at the counter and get a small baggage tag. With the tag you travel light, and at your destination you pick the suitcase up again.

Applied to Serverless, this means the sender writes the actual payload (the business data) to storage, which on AWS is almost always an S3 bucket. To the queue or event bus it sends only a small envelope with the bucket, the object key and a checksum. Then the receiver reads the envelope, fetches the data from S3 and carries on. For every service in between, the message is tiny.

Three ingredients

To make this reliable, you need three ingredients. First, a threshold above which the payload is offloaded. Second, a checksum, so the receiver notices when the object no longer matches the ticket. Third, a cleanup rule that deletes old objects, but only once nobody needs them anymore.

🎬 Flow of the claim check pattern with S3 and SQS – this animated diagram only runs on my site: watch it on sascha-ms.de

When does the claim check pattern still pay off?

For a long time the answer was simple: anything above 256 KB did not fit into SQS, SNS or EventBridge. Today the picture is different. SQS allows messages up to 1 MiB, EventBridge accepts just under 1 MB per PutEvents call, and asynchronous Lambda invocations also take 1 MB. For SNS, the default per topic is still 256 KiB; you can raise it to up to 1 MiB, but then only SQS, Lambda and Firehose are allowed as subscribers. Step Functions stays at 256 KiB for inputs and outputs.

Bar chart: Step Functions and SNS 256 KiB, SQS 1 MiB, EventBridge and asynchronous Lambda 1 MB, synchronous Lambda 6 MB

The smallest limit in the chain decides: Step Functions and SNS (default) are at 256 KiB.

That leaves four good reasons for the pattern, even when your data stays below one megabyte.

The chain is only as large as its smallest link

For instance, a message that only passes through SQS today may end up in a Step Functions execution or be distributed through an SNS topic tomorrow. If you offload from the start, you do not need to rebuild anything later. That is why the threshold in my example is 200 KiB, deliberately below the smallest limit in the chain.

Large messages cost more in SQS

SQS bills every started 64 KB chunk of a message as a separate request. So when you send a 1 MiB message, it counts as 16 requests. For large payloads, an S3 object plus a small claim ticket is often cheaper. Do the math for your volumes with the current SQS and S3 prices, because the break-even point depends on size and volume.

More than 1 MiB

For example, PDFs, images, exports or long series of measurements break even the new limits. At that point there is no way around offloading.

Less data in logs and dead-letter queues

If you log messages or inspect them in a DLQ, a claim ticket shows only references. Meanwhile, the actual data stays in the bucket, where you control access and encryption in one place. I find that especially helpful with personal data.

The example: ride logs for a bike rental

For this article I am making up a bike rental. In this scenario, every rental bike records its ride as a list of GPS points and uploads it after the bike is returned. An upload Lambda behind an HTTP API accepts the tour, responds immediately with 202 and hands it over to a processing Lambda through SQS. Afterwards, that function calculates the distance ridden. Short tours fit directly into the message, while long tours with thousands of points move to S3.

If the basic pattern with 202 and a queue is new to you, read the first part of the series about the asynchronous API with Lambda first. Here I focus on offloading.

Claim check pattern step by step with the Serverless Framework

Prerequisites

  • Node.js 22 or newer and npm
  • Serverless Framework v4 (npm install -g serverless), signed in with serverless login
  • An AWS account with credentials for the eu-central-1 region
  • Basic knowledge of Lambda and SQS; I explain the terms along the way

Step 1: Create the project

First, create an empty folder and install the dependencies. The AWS SDK v3 is included in the Lambda runtime, but I install it anyway so that types and tests work locally.

mkdir bike-rental-tours && cd bike-rental-tours npm init -y npm install @aws-sdk/client-s3 @aws-sdk/client-sqs npm install -D typescript tsx @types/node @types/aws-lambda mkdir -p src/handlers tests

The package.json gets "type": "module" and three scripts. The tsconfig.json allows imports with the .ts extension, so Node.js can run the tests directly through tsx.

{ "name": "claim-check-s3-sqs", "version": "1.0.0", "private": true, "type": "module", "scripts": { "typecheck": "tsc --noEmit", "test": "node --test --import tsx tests/*.test.ts", "deploy": "serverless deploy" }, "dependencies": { "@aws-sdk/client-s3": "^3.1164.0", "@aws-sdk/client-sqs": "^3.1164.0" }, "devDependencies": { "@types/aws-lambda": "^8.10.164", "@types/node": "^22.10.0", "tsx": "^4.23.0", "typescript": "^5.9.3" } } 
{ "compilerOptions": { "target": "ES2022", "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, "esModuleInterop": true, "skipLibCheck": true, "allowImportingTsExtensions": true, "noEmit": true, "types": ["node"] }, "include": ["src", "tests"] }

Step 2: The envelope and the checkIn and checkOut functions

At its core, the solution is a small file without any AWS dependency. checkIn measures the payload in bytes, not in characters. That matters, because an “ä” takes two bytes in UTF-8. If the size is below the threshold, the payload stays inside the envelope. Otherwise the function computes a SHA-256 checksum, stores the object under exactly this hash and returns a claim ticket.

Using the hash as the key has a nice side effect: if a client sends the same tour twice, it ends up in the same object. The upload is therefore idempotent, which means you can repeat it as often as you like without changing the result.

// src/claim-check.ts import { createHash } from 'node:crypto'; // Storage for large payloads (S3 in Lambda, an in-memory map in tests) export interface PayloadStore { put(key: string, body: string): Promise<void>; get(key: string): Promise<string>; } export type Envelope = | { v: 1; kind: 'inline'; body: string } | { v: 1; kind: 'claim'; bucket: string; key: string; bytes: number; sha256: string }; export interface CheckInOptions { store: PayloadStore; bucket: string; thresholdBytes?: number; prefix?: string; } // Deliberately below 256 KiB: the smallest limit on the way (SNS default, Step Functions) wins export const DEFAULT_THRESHOLD_BYTES = 200 * 1024; const sha256Hex = (text: string): string => createHash('sha256').update(text, 'utf8').digest('hex'); export async function checkIn(payload: string, options: CheckInOptions): Promise<Envelope> { const bytes = Buffer.byteLength(payload, 'utf8'); const threshold = options.thresholdBytes ?? DEFAULT_THRESHOLD_BYTES; if (bytes <= threshold) { return { v: 1, kind: 'inline', body: payload }; } // Content as key: the same payload always ends up in the same object (idempotent) const sha256 = sha256Hex(payload); const key = `${options.prefix ?? 'claims/'}${sha256}.json`; await options.store.put(key, payload); return { v: 1, kind: 'claim', bucket: options.bucket, key, bytes, sha256 }; } export async function checkOut(envelope: Envelope, store: PayloadStore): Promise<string> { if (envelope.kind === 'inline') { return envelope.body; } const body = await store.get(envelope.key); if (sha256Hex(body) !== envelope.sha256) { throw new Error(`Checksum does not match object ${envelope.key}`); } return body; } export function parseEnvelope(raw: string): Envelope { let data: unknown; try { data = JSON.parse(raw); } catch { throw new Error('Message is not valid JSON'); } const e = data as Record<string, unknown>; if (e?.v !== 1) { throw new Error('Unknown envelope version'); } if (e.kind === 'inline' && typeof e.body === 'string') { return { v: 1, kind: 'inline', body: e.body }; } if ( e.kind === 'claim' && typeof e.bucket === 'string' && typeof e.key === 'string' && typeof e.bytes === 'number' && typeof e.sha256 === 'string' && /^[a-f0-9]{64}$/.test(e.sha256) ) { return { v: 1, kind: 'claim', bucket: e.bucket, key: e.key, bytes: e.bytes, sha256: e.sha256 }; } throw new Error('Envelope has an unknown format'); }

Moreover, the storage layer sits behind the PayloadStore interface. In Lambda that is S3, in the tests a simple map. This way you can test all of the logic without an AWS account.

Step 3: The S3 store 

// src/s3-payload-store.ts import { GetObjectCommand, PutObjectCommand, S3Client } from '@aws-sdk/client-s3'; import type { PayloadStore } from './claim-check.ts'; export class S3PayloadStore implements PayloadStore { constructor( private readonly bucket: string, private readonly s3: S3Client = new S3Client({}), ) {} async put(key: string, body: string): Promise<void> { await this.s3.send( new PutObjectCommand({ Bucket: this.bucket, Key: key, Body: body, ContentType: 'application/json', }), ); } async get(key: string): Promise<string> { const result = await this.s3.send(new GetObjectCommand({ Bucket: this.bucket, Key: key })); if (!result.Body) { throw new Error(`Objekt ${key} is empty`); } return result.Body.transformToString('utf-8'); } }

There is deliberately no delete command here. I explain why further down in the section on common mistakes.

Step 4: The tour model

By contrast, the business logic is intentionally simple: validate the tour and calculate the distance with the haversine formula, which gives the distance between two points on a sphere.

// src/tour.ts export interface TrackPoint { lat: number; lon: number; t: string; } export interface Tour { tourId: string; bikeId: string; points: TrackPoint[]; } export function parseTour(raw: string): Tour { const data = JSON.parse(raw) as Partial<Tour>; if (typeof data.tourId !== 'string' || typeof data.bikeId !== 'string') { throw new Error('tourId and bikeId are required'); } if (!Array.isArray(data.points) || data.points.length < 2) { throw new Error('A tour needs at least two points'); } return data as Tour; } // Distance between two coordinates in kilometers (haversine formula) function haversineKm(a: TrackPoint, b: TrackPoint): number { const rad = (deg: number) => (deg * Math.PI) / 180; const dLat = rad(b.lat - a.lat); const dLon = rad(b.lon - a.lon); const h = Math.sin(dLat / 2) ** 2 + Math.cos(rad(a.lat)) * Math.cos(rad(b.lat)) * Math.sin(dLon / 2) ** 2; return 2 * 6371 * Math.asin(Math.sqrt(h)); } export function distanceKm(tour: Tour): number { let sum = 0; for (let i = 1; i < tour.points.length; i++) { sum += haversineKm(tour.points[i - 1], tour.points[i]); } return Math.round(sum * 100) / 100; }

Step 5: The upload Lambda

First, the upload function validates the tour. It then calls checkIn and sends the envelope to the queue. The makeUploadTour factory receives the store and the send function as parameters. That makes the handler testable without mocking the SDK.

// src/handlers/upload-tour.ts import { SendMessageCommand, SQSClient } from '@aws-sdk/client-sqs'; import type { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from 'aws-lambda'; import { checkIn, type PayloadStore } from '../claim-check.ts'; import { S3PayloadStore } from '../s3-payload-store.ts'; import { parseTour } from '../tour.ts'; const BUCKET = process.env.PAYLOAD_BUCKET ?? ''; const QUEUE_URL = process.env.TOUR_QUEUE_URL ?? ''; type Send = (body: string) => Promise<void>; export function makeUploadTour(store: PayloadStore, send: Send) { return async (event: APIGatewayProxyEventV2): Promise<APIGatewayProxyResultV2> => { const raw = event.isBase64Encoded ? Buffer.from(event.body ?? '', 'base64').toString('utf8') : (event.body ?? ''); let tourId: string; try { tourId = parseTour(raw).tourId; } catch (err) { return { statusCode: 400, body: JSON.stringify({ message: (err as Error).message }) }; } const envelope = await checkIn(raw, { store, bucket: BUCKET }); await send(JSON.stringify(envelope)); return { statusCode: 202, body: JSON.stringify({ tourId, stored: envelope.kind }), }; }; } const sqs = new SQSClient({}); export const handler = makeUploadTour(new S3PayloadStore(BUCKET), async (body) => { await sqs.send(new SendMessageCommand({ QueueUrl: QUEUE_URL, MessageBody: body })); });

Step 6: The processing Lambda

Next, the second function reads messages from SQS. It unpacks the envelope, fetches the payload from S3 when needed, verifies the checksum and calculates the distance. If ...