DynamoDB Bulk Put Items: Load S3 Data into Existing Tables

With DynamoDB bulk put items you load large amounts of data from S3 straight into a table that already exists and is serving production traffic. Until now that was a gap: import from S3 always created a new table, and for everything else you built your own pipeline with Lambda, Step Functions or Glue, raised capacity and hoped the application would not get throttled in the meantime.

Since November 5, 2026, DynamoDB handles this part itself. What interests me most is not the speed but a small detail: every record in the input file can carry its own condition. That makes a nightly sync safe to repeat without overwriting newer data written by the application. In this article you build the whole thing once, end to end, with the Serverless Framework, TypeScript and a report that you actually evaluate at the end.

In short

Bulk put items writes data from S3 into an existing DynamoDB table without consuming the table’s read or write capacity. The key ingredient is a condition per record, which makes an import safe to repeat. The biggest pitfall: cancelled runs are not rolled back, and every written record also shows up in your streams.

What you will learn in this article

• When bulk put items fits better than BatchWriteItem or importing into a new table
• How to generate input files with a version guard per record in TypeScript
• How to start and monitor the import via CLI, SDK and a nightly Lambda function
• How to read the error report and manifest, and what the whole thing costs

What is DynamoDB bulk put items?

Bulk put items is a managed DynamoDB feature. You point it at a folder (more precisely, a prefix) in an S3 bucket, name the target table and a second S3 location for the report. The service does the rest: it reads the files, writes every item, retries failed writes and drops a report at the end.

Two properties make the difference. First, the writes take a separate path and consume no read or write capacity of the table. Your application does not notice, even on a table in provisioned mode. Second, new items are readable right away while the import is still running. There is no switch-over moment at the end.

The API consists of four calls: StartBulkPutItems starts an import, DescribeBulkOperation returns its progress, ListBulkOperations shows the runs of the last 90 days and CancelBulkOperation stops a run. The start call returns an ARN right away, a unique identifier for the operation. After that the actual import runs asynchronously.

DynamoDB accepts two input formats: DynamoDB JSON as JSON Lines (one JSON object per line) or Amazon Ion. Both can be uncompressed, gzip-compressed or Zstandard-compressed. All files under one prefix must share the same format and compression.

Bulk put items, BatchWriteItem or import from S3?

DynamoDB now offers three ways to get many items into a table. They compete less than it seems at first glance. The table below shows which one I would pick for what.

For a few hundred records from a Lambda function, BatchWriteItem stays the right choice. AWS says so itself, and it matches my view: an asynchronous job with a report in S3 is simply too much ceremony for small volumes. Bulk put items shines for syncs, backfills and migrations. In other words, everywhere you would previously have built your own pipeline.

If you are still deciding which service fits your use case, the AWS service finder may help you sort it out.

How an import runs

🎬 Flow of a DynamoDB bulk put items import – this animated diagram only runs on my site: watch it on sascha-ms.de

The diagram shows the example we are about to build. A script converts the supplier file into the input format and stores it in S3. Then DynamoDB takes over. Every successful write also creates a record in DynamoDB Streams if the table has streams enabled. That matters for everything attached to your streams, more on that later.

The example: a spare parts catalog for a bike rental

Picture a bike rental with its own workshop. A wholesaler delivers a CSV file every night with all spare parts: part number, name, price in cents and stock level. The workshop app reads from that table and also writes to it, for example when someone corrects a price by hand and bumps the catalog version.

This is where it gets interesting. The nightly import should create new parts and update older entries. It must not overwrite anything that already has a newer version, though. And if the run starts twice for whatever reason, nothing should break. Without a condition per record you would have to solve that in your own pipeline. With bulk put items it is one line in the input file.

Step by step: DynamoDB bulk put items with a version guard

Prerequisites

• An AWS account and a Region with bulk put items (the Standard tier is available in all commercial Regions)
• Node.js 22 or later and the Serverless Framework v4
• An up-to-date AWS CLI v2 (older versions do not know the new commands yet)
• @aws-sdk/client-dynamodb 3.1162.0 or later, I tested with 3.1163.0

Create a new project and install the dependencies:

mkdir bike-parts-catalog && cd bike-parts-catalog npm init -y npm pkg set type=module npm install @aws-sdk/client-dynamodb @aws-sdk/client-s3 @aws-sdk/util-dynamodb npm install -D typescript tsx @types/node @types/aws-lambda 

// tsconfig.json { "compilerOptions": { "target": "ES2022", "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, "esModuleInterop": true, "skipLibCheck": true, "noEmit": true, "types": ["node"] }, "include": ["src", "test"] } 

Step 1: Table and buckets with the Serverless Framework

The infrastructure consists of a table with the part number sku as partition key, a bucket for the input files and a bucket for the reports. On top comes a Lambda function that starts the import every night. The IAM permissions are deliberately tight: dynamodb:StartBulkOperation applies account-wide, while dynamodb:BulkPutItems is scoped to the table.

# serverless.yml service: bike-parts-catalog build: esbuild: # Bundle the SDK: the bulk commands need @aws-sdk/client-dynamodb 3.1162.0 or later exclude: [] provider: name: aws runtime: nodejs24.x region: eu-central-1 architecture: arm64 environment: TABLE_NAME: !Ref PartsTable INPUT_BUCKET: !Ref InputBucket REPORT_BUCKET: !Ref ReportBucket iam: role: statements: - Effect: Allow Action: - dynamodb:StartBulkOperation - dynamodb:DescribeBulkOperation - dynamodb:ListBulkOperations Resource: "*" - Effect: Allow Action: - dynamodb:BulkPutItems Resource: !GetAtt PartsTable.Arn - Effect: Allow Action: - s3:ListBucket Resource: !GetAtt InputBucket.Arn - Effect: Allow Action: - s3:GetObject Resource: !Sub "${InputBucket.Arn}/incoming/*" - Effect: Allow Action: - s3:PutObject - s3:AbortMultipartUpload Resource: !Sub "${ReportBucket.Arn}/reports/*" functions: startNightlyImport: handler: src/handler.startNightlyImport timeout: 30 events: - schedule: method: scheduler rate: cron(15 2 * * ? *) timezone: Europe/Berlin resources: Resources: PartsTable: Type: AWS::DynamoDB::Table Properties: TableName: bike-parts-catalog-${sls:stage} BillingMode: PAY_PER_REQUEST AttributeDefinitions: - AttributeName: sku AttributeType: S KeySchema: - AttributeName: sku KeyType: HASH PointInTimeRecoverySpecification: PointInTimeRecoveryEnabled: true InputBucket: Type: AWS::S3::Bucket Properties: LifecycleConfiguration: Rules: - Id: expire-incoming Status: Enabled Prefix: incoming/ ExpirationInDays: 14 ReportBucket: Type: AWS::S3::Bucket Properties: LifecycleConfiguration: Rules: - Id: expire-reports Status: Enabled Prefix: reports/ ExpirationInDays: 30 Outputs: InputBucketName: Value: !Ref InputBucket ReportBucketName: Value: !Ref ReportBucket 

Two details in this file deserve a second look. The build.esbuild.exclude: [] section makes sure the SDK gets bundled. Otherwise the Serverless Framework leaves out @aws-sdk/* and relies on the SDK in the runtime, which may not know the new command yet. In addition, lifecycle rules clean up old input files and reports automatically so nothing piles up in S3.

npx serverless deploy --stage dev 

Expected result: After the deployment the framework shows the names of both buckets under “Stack Outputs”. Note them down, you will need them shortly.

Step 2: Generate input files with a condition

Each line of the input file contains an Item field in DynamoDB JSON and, optionally, a ConditionExpression with placeholders. The syntax is exactly that of PutItem. The script reads the CSV file, sets the catalog version as a number (here the date, for example 20261106) and attaches the same guard to every record:

// src/build-input.ts import { readFileSync, writeFileSync, mkdirSync } from "node:fs"; import { gzipSync } from "node:zlib"; import { marshall } from "@aws-sdk/util-dynamodb"; export interface PartRow { sku: string; name: string; priceCents: number; stock: number; } // One record in the bulk put items input format (one JSON Lines row) export interface BulkRecord { Item: Record<string, unknown>; ConditionExpression?: string; ExpressionAttributeNames?: Record<string, string>; ExpressionAttributeValues?: Record<string, unknown>; } // Simple CSV parser for "sku;name;priceCents;stock" without quotes export function parseCsv(csv: string): PartRow[] { const lines = csv.split(/\r?\n/).filter((l) => l.trim() !== ""); const [, ...rows] = lines; // skip the header row return rows.map((line, i) => { const [sku, name, price, stock] = line.split(";"); const priceCents = Number(price); const stockCount = Number(stock); if (!sku || !name || !Number.isInteger(priceCents) || !Number.isInteger(stockCount)) { throw new Error(`Line ${i + 2} is invalid: ${line}`); } return { sku: sku.trim(), name: name.trim(), priceCents, stock: stockCount }; }); } // Writes only if the part does not exist yet or our version is newer export function toBulkRecord(row: PartRow, catalogVersion: number): BulkRecord { return { Item: marshall({ sku: row.sku, name: row.name, priceCents: row.priceCents, stock: row.stock, catalogVersion, }), ConditionExpression: "attribute_not_exists(#v) OR #v < :v", ExpressionAttributeNames: { "#v": "catalogVersion" }, ExpressionAttributeValues: marshall({ ":v": catalogVersion }), }; } // Splits the records into similarly sized files and compresses them with gzip export function toChunks(records: BulkRecord[], perFile: number): Buffer[] { const files: Buffer[] = []; for (let i = 0; i < records.length; i += perFile) { const body = records .slice(i, i + perFile) .map((r) => JSON.stringify(r)) .join("\n"); files.push(gzipSync(body + "\n")); } return files; } // Usage: npx tsx src/build-input.ts parts.csv 20261106 out/ if (process.argv[1]?.endsWith("build-input.ts")) { const [csvPath, version, outDir = "out"] = process.argv.slice(2); if (!csvPath || !version) { console.error("Usage: build-input.ts <csv> <catalog-version> [output-dir]"); process.exit(1); } const rows = parseCsv(readFileSync(csvPath, "utf8")); const records = rows.map((r) => toBulkRecord(r, Number(version))); mkdirSync(outDir, { recursive: true }); toChunks(records, 50_000).forEach((buf, i) => { writeFileSync(`${outDir}/parts-${String(i).padStart(4, "0")}.json.gz`, buf); }); console.log(`Wrote ${records.length} records to ${outDir}/`); } 

A finished line then looks like this:

{"Item":{"sku":{"S":"KET-108"},"name":{"S":"Chain 108 links"},"priceCents":{"N":"1999"},"stock":{"N":"40"},"catalogVersion":{"N":"20261106"}},"ConditionExpression":"attribute_not_exists(#v) OR #v < :v","ExpressionAttributeNames":{"#v":"catalogVersion"},"ExpressionAttributeValues":{":v":{"N":"20261106"}}} 

It is important that a record contains no line breaks. JSON.stringify without indentation takes care of that reliably. Splitting into files of 50,000 records each follows an AWS recommendation: several similarly sized files can be processed in parallel better than one huge file.

To make sure you can rely on the conversion, here are a few tests using the test runner built into Node.js:

// test/build-input.test.ts import { test } from "node:test"; import assert from "node:assert/strict"; import { gunzipSync } from "node:zlib"; import { parseCsv, toBulkRecord, toChunks } from "../src/build-input.js"; const csv = "sku;name;priceCents;stock\nKET-108;Chain 108 links;1999;40\nSCH-28;Tube 28 inch;699;120\n"; test("parseCsv reads rows and skips the header", () => { const rows = parseCsv(csv); assert.equal(rows.length, 2); assert.deepEqual(rows[0], { sku: "KET-108", name: "Chain 108 links", priceCents: 1999, stock: 40 }); }); test("parseCsv reports invalid rows with their line number", () => { assert.throws(() => parseCsv("sku;name;priceCents;stock\nX;Y;abc;1"), /Line 2/); }); test("toBulkRecord sets the version guard", () => { const rec = toBulkRecord(parseCsv(csv)[0]!, 20261106); assert.equal(rec.ConditionExpression, "attribute_not_exists(#v) OR #v < :v"); assert.deepEqual(rec.ExpressionAttributeValues, { ":v": { N: "20261106" } }); assert.deepEqual(rec.Item["sku"], { S: "KET-108" }); }); test("toChunks writes JSON Lines without line breaks inside a record", () => { const recs = parseCsv(csv).map((r) => toBulkRecord(r, 1)); const files = toChunks(recs, 1); assert.equal(files.length, 2); const lines = gunzipSync(files[0]!).toString("utf8").trim().split("\n"); assert.equal(lines.length, 1); assert.ok(JSON.parse(lines[0]!).Item); }); 

npx tsc --noEmit node --import tsx --test test/*.test.ts npx tsx src/build-input.ts parts.csv 20261106 out 

Expected result: Four green tests and one or more files such as parts-0000.json.gz in the out/ folder.

Step 3: Upload the files to S3

Upload the files under a prefix that contains the catalog version. That keeps every run cleanly separated, and an import never accidentally reads the previous day’s files.

aws s3 cp out/ s3://<input-bucket>/incoming/20261106/ --recursive 

Step 4: Start the import with the AWS CLI

For a first test the CLI is enough. The output configuration requests both report types: ERROR for broken records and CONDITION_CHECK_FAILURE for records that failed their condition. Without the second one you learn nothing about skipped records, not even how many there were.

aws dynamodb start-bulk-put-items \ --table-name bike-parts-catalog-dev \ --client-token parts-20261106 \ --input-s3-configuration '{"S3Bucket":"<input-bucket>","S3KeyPrefix":"incoming/20261106/","InputFormat":"DYNAMODB_JSON","InputCompressionType":"GZIP"}' \ --output-s3-configuration '{"S3Bucket":"<report-bucket>","S3KeyPrefix":"reports/20261106","OutputType":["ERROR","CONDITION_CHECK_FAILURE"]}' aws dynamodb describe-bulk-operation --bulk-operation-arn <arn-from-the-response> 

The client-token makes the start idempotent. If you call the same command again within eight hours with the same parameters, you get the same operation back instead of a second import. With different parameters and the same token, DynamoDB responds with IdempotentParameterMismatchException.

Expected result: At first the status is IN_PROGRESS, and the counters stay at zero for a few minutes while DynamoDB prepares the operation. After that they rise in steps because the values are only refreshed periodically.

Step 5: Start and monitor the import from TypeScript

Day to day you do not want to type this by hand. The following module wraps starting and monitoring. waitForImport checks the status every 30 seconds and stops at one of the four final states.

// src/bulk-import.ts import { DynamoDBClient, StartBulkPutItemsCommand, DescribeBulkOperationCommand, type BulkOperationDescription, } from "@aws-sdk/client-dynamodb"; const ddb = new DynamoDBClient({}); export interface ImportOptions { tableName: string; inputBucket: string; inputPrefix: string; reportBucket: string; reportPrefix: string; // Same token = same import, even if the start is called twice clientToken: ...