SOCI lazy loading: start Lambda containers faster

SOCI lazy loading lets a Lambda function on Lambda Managed Instances start before its container image has been fully downloaded. AWS announced it on October 28, 2026, and quotes startup times that are up to 70 percent shorter. I care less about that percentage than about two questions: who actually benefits from the extra build step, and where does it cause you trouble?

Container images are often the escape hatch on Lambda for anything that does not fit into a 250 MB ZIP package: ML libraries, models, PDF tools, entire runtimes. Those images quickly grow to several gigabytes. In this article, we build an example from scratch together: a small book recommendation service for a library, an image with four language models, the SOCI index, a CDK stack with a capacity provider and a script that lets you measure the effect yourself.

In short

SOCI places a table of contents next to your container image, so Lambda on Managed Instances only fetches the files your function actually reads. It pays off most for large images where each function needs only part of the content. The biggest pitfall is the format: Lambda accepts only a SOCI index v2 inside an image index with exactly two entries, and only single-architecture images.

What you will learn in this article

• What Lambda Managed Instances are and when they fit your workload
• How SOCI lazy loading fetches files on demand
• How to create a SOCI index with nerdctl and push it to ECR
• How to set up a capacity provider and function with AWS CDK in TypeScript
• How to measure startup times with and without SOCI yourself

What are Lambda Managed Instances?

Lambda Managed Instances (LMI for short) are a second way to run Lambda functions. Instead of the shared Lambda fleet, your functions run on EC2 instances in your own account. Lambda still takes care of launching, patching, routing and scaling those instances. You only decide where they may run and on which hardware.

The central building block is the capacity provider. It describes the VPC, subnets, security groups, architecture, allowed instance types and an upper limit for vCPUs. You then attach a function to that provider and publish a version. Only when you publish does Lambda launch instances, three by default for spreading across Availability Zones, plus three execution environments, before the version counts as ACTIVE.

How it differs from regular Lambda

For Node.js, LMI uses worker threads, and initialization runs once per worker thread. That matters for our example: if you load a large model into memory, expect it to sit in memory once per thread. In addition, concurrent requests in the same thread share the global state. Anything you create outside the handler should therefore only be read and never modified.

If you want to refresh the regular Lambda basics first, my article Deploy a Lambda function with CDK and TypeScript covers the standard path with a ZIP package.

Why large images slow down Managed Instances

Without SOCI, Lambda has to download and unpack the complete image on every instance before an execution environment can start there. That does not happen just once when you publish. It also happens whenever the capacity provider launches new instances because load is rising. With a 5 GB image, you are mostly waiting on network and disk during this phase.

The catch: a large part of that data is often never read. An image with PyTorch, several models and half an operating system contains far more than a single invocation needs. This is where SOCI lazy loading helps.

How SOCI lazy loading works

SOCI stands for Seekable OCI. OCI is the open standard for container images. The project comes from AWS and is open source under Apache-2.0. SOCI does not change your image. Instead, it creates an additional index that records for each file in which layer and at which position it is stored. With this table of contents, Lambda can fetch individual files from ECR instead of downloading everything in one go.

On Lambda Managed Instances, the flow looks like this: you push the image and the SOCI index together to ECR under a separate tag. That tag points to an image index with two entries, the regular image manifest and the SOCI index. If your function points to this tag, Lambda detects the index when you publish the version and turns on lazy loading. There are no further settings.

🎬 Startup with and without SOCI – this animated diagram only runs on my site: watch it on sascha-ms.de

When the files are fetched

Files your init code reads are fetched by Lambda during initialization. Anything the handler touches only later gets pulled in during that invocation, which then takes correspondingly longer. According to the documentation, Lambda does not download the rest of the image in the background. That gives you the most important rule for your code: import every library and read every file you need in the init phase. Then your invocation durations stay consistent.

From a plain image to an image index with a SOCI index: five steps to the function.

Prerequisites for SOCI lazy loading

• An AWS account in a Region where Lambda Managed Instances are available, and AWS CLI v2
• Node.js 22 or later, Docker with buildx and AWS CDK v2 (here aws-cdk-lib 2.275.0)
• For the SOCI step, a Linux machine with containerd, for example a small EC2 instance with Amazon Linux 2023 and a Graviton processor
• A function packaged as a container image for one architecture, here arm64
• The jq command line tool and basic knowledge of TypeScript and CDK

A quick glossary: ECR is the AWS container registry. nerdctl is a Docker-like command line tool for containerd, and the SOCI snapshotter is the containerd extension that can create and read SOCI indexes.

Setting up SOCI lazy loading step by step

Our example is a recommendation service for a library. For every book ID, it returns the five most similar books. It computes the similarity from vectors stored in one binary file per language. The image contains models for German, English, French and Spanish, but the function reads only one of them. I chose this on purpose, because exactly this pattern shows up in real images all the time.

Step 1: Create the project

mkdir lmi-soci-library && cd lmi-soci-library mkdir -p src scripts lib bin test npm init -y 

Replace the generated package.json with this version. "type": "module" makes Node.js load the compiled files as ES modules, including inside the container.

{ "name": "lmi-soci-library", "version": "1.0.0", "private": true, "type": "module", "scripts": { "build": "tsc -p tsconfig.build.json", "models": "node --experimental-strip-types scripts/make-models.ts", "test": "node --experimental-strip-types --test test/*.test.ts", "measure": "node --experimental-strip-types scripts/measure-start.ts" }, "dependencies": {}, "devDependencies": { "@aws-sdk/client-lambda": "3.1159.0", "@types/node": "^22.19.0", "aws-cdk": "2.1146.1", "aws-cdk-lib": "2.275.0", "constructs": "^10.4.2", "typescript": "7.0.2" } } 

You run the scripts under scripts/ and the tests directly from TypeScript, using the Node.js 22 option --experimental-strip-types. For the container, on the other hand, you compile to dist/. Two small configuration files handle this:

// tsconfig.json { "compilerOptions": { "target": "es2023", "module": "nodenext", "moduleResolution": "nodenext", "strict": true, "allowImportingTsExtensions": true, "rewriteRelativeImportExtensions": true, "skipLibCheck": true, "types": ["node"], "outDir": "dist" }, "include": ["src", "scripts", "lib", "bin", "test"] } 

// tsconfig.build.json { "extends": "./tsconfig.json", "compilerOptions": { "rootDir": "src", "outDir": "dist" }, "include": ["src"] } 

npm install 

Step 2: Load and compare vectors

The module reads a file in the format “magic bytes, count, dimensions, values” and compares books using cosine similarity. Cosine similarity measures how closely two vectors point in the same direction: 1 means very similar, 0 means unrelated.

// src/vectors.ts import { readFileSync } from 'node:fs'; export interface BookVectors { count: number; dims: number; data: Float32Array; } const MAGIC = 'BKV1'; const HEADER_BYTES = 12; /** Writes vectors in a simple binary format: "BKV1", count, dims, float32 values. */ export function encodeVectors(v: BookVectors): Buffer { const header = Buffer.alloc(HEADER_BYTES); header.write(MAGIC, 0, 'ascii'); header.writeUInt32LE(v.count, 4); header.writeUInt32LE(v.dims, 8); const body = Buffer.from(v.data.buffer, v.data.byteOffset, v.data.byteLength); return Buffer.concat([header, body]); } export function decodeVectors(buf: Buffer): BookVectors { if (buf.subarray(0, 4).toString('ascii') !== MAGIC) { throw new Error('Unknown file format: magic bytes missing'); } const count = buf.readUInt32LE(4); const dims = buf.readUInt32LE(8); const expected = HEADER_BYTES + count * dims * 4; if (buf.byteLength !== expected) { throw new Error(`File size ${buf.byteLength} does not match ${expected} bytes`); } // Copy into an aligned buffer so Float32Array works on every offset const data = new Float32Array(count * dims); data.set(new Float32Array(buf.buffer.slice(buf.byteOffset + HEADER_BYTES, buf.byteOffset + expected))); return { count, dims, data }; } export function loadVectors(path: string): BookVectors { return decodeVectors(readFileSync(path)); } function cosine(v: BookVectors, a: number, b: number): number { let dot = 0; let na = 0; let nb = 0; const oa = a * v.dims; const ob = b * v.dims; for (let i = 0; i < v.dims; i++) { const x = v.data[oa + i]; const y = v.data[ob + i]; dot += x * y; na += x * x; nb += y * y; } return na === 0 || nb === 0 ? 0 : dot / Math.sqrt(na * nb); } export interface Recommendation { bookId: number; score: number; } /** Returns the books most similar to bookId (without bookId itself). */ export function similarBooks(v: BookVectors, bookId: number, limit = 5): Recommendation[] { if (!Number.isInteger(bookId) || bookId < 0 || bookId >= v.count) { throw new RangeError(`bookId must be between 0 and ${v.count - 1}`); } const best: Recommendation[] = []; for (let other = 0; other < v.count; other++) { if (other === bookId) continue; const score = cosine(v, bookId, other); if (best.length < limit || score > best[best.length - 1].score) { best.push({ bookId: other, score }); best.sort((x, y) => y.score - x.score); if (best.length > limit) best.pop(); } } return best.map((r) => ({ bookId: r.bookId, score: Math.round(r.score * 1000) / 1000 })); } 

Step 3: A handler that loads in the init phase

The handler loads the model at the top level, which means during initialization. With SOCI, this is the moment Lambda fetches the file from ECR. If the call to loadVectors lived inside the handler, the first invocation of every environment would take noticeably longer.

// src/handler.ts import { loadVectors, similarBooks, type Recommendation } from './vectors.ts'; // Load in the init phase: with SOCI, Lambda fetches the file here // and not during the first invocation. const MODEL_FILE = process.env.MODEL_FILE ?? '/opt/models/books-de.bin'; const started = Date.now(); const vectors = loadVectors(MODEL_FILE); console.log(JSON.stringify({ msg: 'model loaded', file: MODEL_FILE, books: vectors.count, ms: Date.now() - started })); export interface RecommendEvent { bookId: number; limit?: number; } export interface RecommendResult { model: string; recommendations: Recommendation[]; } // Read-only shared state: safe even when an execution environment // handles several invocations at the same time. export async function handler(event: RecommendEvent): Promise<RecommendResult> { const limit = Math.min(Math.max(event.limit ?? 5, 1), 20); return { model: MODEL_FILE, recommendations: similarBooks(vectors, event.bookId, limit) }; } 

Step 4: Generate test data and check locally

The script creates four model files with random values. With BOOKS=200000, every file grows to roughly 100 MB, and the difference at startup becomes clearly visible. Small files are enough for the first test.

// scripts/make-models.ts // Generates one vector file per language. Only one of them is used per function – // exactly the situation in which SOCI lazy loading pays off. import { mkdirSync, writeFileSync } from 'node:fs'; import { encodeVectors } from '../src/vectors.ts'; const count = Number(process.env.BOOKS ?? 20000); const dims = Number(process.env.DIMS ?? 128); const languages = (process.env.LANGS ?? 'de,en,fr,es').split(','); mkdirSync('models', { recursive: true }); let seed = 42; const random = () => { seed = (seed * 1664525 + 1013904223) % 2 ** 32; return seed / 2 ** 32 - 0.5; }; for (const lang of languages) { const data = new Float32Array(count * dims); for (let i = 0; i < data.length; i++) data[i] = random(); const file = `models/books-${lang}.bin`; const buf = encodeVectors({ count, dims, data }); writeFileSync(file, buf); console.log(`${file}: ${(buf.byteLength / 1024 / 1024).toFixed(1)} MB`); } 

BOOKS=5000 npm run models npm test npm run build MODEL_FILE=models/books-de.bin node -e "import('./dist/handler.js').then(async (m) => console.log(await m.handler({ bookId: 7, limit: 3 })))" 

Expected result: four files of 2.4 MB each, four passing tests and an output with three recommendations. Book IDs and scores are random; the structure looks like this:

{"msg":"model loaded","file":"models/books-de.bin","books":5000,"ms":4} { model: 'models/books-de.bin', recommendations: [ { bookId: 37, score: 0.337 }, { bookId: 1965, score: 0.311 }, { bookId: 3426, score: 0.274 } ] } 

Step 5: Build the image for arm64 and push it to ECR

The Dockerfile uses the official Lambda base image for Node.js 24. The models end up in /opt/models.

# Dockerfile FROM public.ecr.aws/lambda/nodejs:24 # Compiled code plus package.json (for "type": "module") COPY package.json ${LAMBDA_TASK_ROOT}/ COPY dist/ ${LAMBDA_TASK_ROOT}/ # All language models – each function only reads one of them COPY models/ /opt/models/ CMD ["handler.handler"] 

Two build options matter here. --platform linux/arm64, because Lambda accepts only single-architecture images. And --provenance=false, because buildx would otherwise attach an attestation to the image and turn it into an image index with several entries. For the SOCI step, you want a plain image without such extras.

BOOKS=200000 npm run models && npm run build ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) REGION=eu-central-1 REPO=library-recommender REGISTRY=$ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com aws ecr create-repository --repository-name $REPO --region $REGION aws ecr get-login-password --region $REGION | docker login -u AWS --password-stdin $REGISTRY docker buildx build --platform linux/arm64 --provenance=false --sbom=false \ -t $REGISTRY/$REPO:v1 --push . 

Step 6: Create the SOCI index

You run this step on a Linux machine with containerd. The easiest option is a small Graviton instance with Amazon Linux 2023, because its architecture already matches the image. The instance needs a role with read and write access to the ECR repository.

sudo yum install -y soci-snapshotter containerd nerdctl sudo systemctl start soci-snapshotter sudo systemctl restart containerd ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) REGION=eu-central-1 REPO=library-recommender REGISTRY=$ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com aws ecr get-login-password --region $REGION | sudo nerdctl login -u AWS --password-stdin $REGISTRY sudo nerdctl pull --platform linux/arm64 $REGISTRY/$REPO...