Most model calls end one way.

You send a request. You get an answer. Done.

Long answers are starting to break that shape.

On October 13, 2026, Google released Gemini 4 Argon. The launch post credits an "industry-leading 1M unified input-output token limit via the Gemini API" and asks developers to read up on "the new continuation protocol." The Gemini API release notes list gemini-4-argon as generally available the same day.

That protocol is the part I keep thinking about.

One request no longer has to hold the whole answer. Google's developer guide for Argon says a single request caps out at 65,536 tokens (32,768 for non-streaming direct HTTP). Past that, the API returns finish_reason=CONTINUATION and an encrypted continuation_token that contains "a hash of the original prompt and the generation state up to that point." You send it back, and the model keeps going.

The same guide is very direct about what not to do:

Don't append partial model output into contents or send a follow-up prompt asking the model to continue.

It also says "don't use prompt_token_count to trigger context compaction in agent harnesses."

Google's long decode page (last updated October 13), where the servers run the 64K hops for you, makes the same point: "total_input_tokens is the sum of every hop's input, not the size of your context."

The same day, Vercel's AI SDK merged PR #23263, adding gemini-4-argon with automatic continuation. It exposes initialPromptTokenCount "so context-window checks don't use the summed input tokens."

Different teams. Same warning.

A long answer is now a loop. Your token math has to know that.

So let's build a tiny version of the client side of that loop.

By the end, you'll run one command:

npx tsx continuation.ts 

And watch a continuation loop get through a thinking-only segment, skip a compaction that would have fired for no reason, survive a dropped stream without duplicating text, and resume after the caller's cap runs out.

This is my small model of the idea. It is not how Gemini or the Google Gen AI SDK works inside. The model is fake, one word counts as one token, and the 40-token segment size and 1,000-token window are example inputs so the demo fits on a screen. Argon's real numbers are in the docs linked above.

Table of Contents

  1. The Problem
  2. Step 1: Requests, Checkpoints, and Usage
  3. Step 2: A Fake Model That Decodes in Segments
  4. Step 3: The Loop
  5. Step 4: Count the Context, Not the Bill
  6. Step 5: The Loops People Write First
  7. Step 6: Run It
  8. Where It Breaks Down
  9. The Bigger Idea

1. The Problem

A continuation protocol turns one answer into several requests.

That sounds like plumbing. It breaks four things your code used to get for free.

The request has to stay identical. The token carries a hash of the original prompt, and the guide says to resend that prompt every time.

Text can arrive before it is safe. A stream can drop halfway through a segment.

A segment can be all thinking. Empty text does not mean done.

Usage adds up in a strange way. Each continuation re-sends the prompt plus everything decoded so far, so summed input tokens grow much faster than the context does.

None of those are model bugs. They are loop bugs.

The model gives you a checkpoint. Your loop decides what counts as committed.

2. Step 1: Requests, Checkpoints, and Usage

Save the following TypeScript blocks in order as continuation.ts.

// tiny-continuation-loop: finish a long decode without lying about tokens. // Fake model, fake tokens (1 word = 1 token). No API key. No network. import { createHash } from "node:crypto"; type Request = { contents: string; systemInstruction: string; tools: string[] }; type Finish = "CONTINUATION" | "STOP" | "MAX_TOKENS"; type Usage = { prompt: number; text: number; thoughts: number }; type Checkpoint = { text: string; finish: Finish; token?: string; usage: Usage }; const count = (s: string) => (s.trim() ? s.trim().split(/\s+/).length : 0); const hash = (r: Request) => createHash("sha256").update(JSON.stringify(r)).digest("hex"); const pack = (o: object) => Buffer.from(JSON.stringify(o)).toString("base64url"); const unpack = (t: string) => JSON.parse(Buffer.from(t, "base64url").toString()); const log = (s: string) => console.log(s); 

A Request is the three things the guide says to resend every time: contents, system instruction, and tools.

A Checkpoint is what arrives when a request ends: text, a finish reason, maybe a token, and usage. The guide says the token and usage come back "at the end of each request."

My token is plain base64 with a hash inside. Real tokens are encrypted. The idea we need is the hash.

3. Step 2: A Fake Model That Decodes in Segments

// Each request decodes at most `perRequest` tokens, thinking first, then text. // Text streams in chunks. The checkpoint (token + usage) arrives at the end. class FakeModel { requests = 0; private dropAt = -1; constructor( private perRequest: number, private thoughts: number, private answer: string[], ) {} dropNextStreamAfter(chunks: number): void { this.dropAt = chunks; } async *stream(req: Request, maxOutput: number, token?: string): AsyncGenerator<string, Checkpoint> { this.requests += 1; let pos = 0; if (token) { const t = unpack(token); if (t.h !== hash(req)) throw new Error("400: continuation token does not match this request"); pos = t.pos; } const total = this.thoughts + this.answer.length; const end = Math.min(pos + this.perRequest, total, maxOutput); const thoughts = Math.max(0, Math.min(end, this.thoughts) - pos); const words = this.answer.slice(Math.max(0, pos - this.thoughts), Math.max(0, end - this.thoughts)); for (let i = 0, n = 0; i < words.length; i += 10, n += 1) { if (n === this.dropAt) { this.dropAt = -1; throw new Error("stream dropped"); } yield words.slice(i, i + 10).join(" "); } const prompt = count(req.contents) + count(req.systemInstruction) + pos; // re-prefill const finish: Finish = end >= total ? "STOP" : end >= maxOutput ? "MAX_TOKENS" : "CONTINUATION"; const next = finish === "STOP" ? undefined : pack({ h: hash(req), pos: end }); return { text: words.join(" "), finish, token: next, usage: { prompt, text: words.length, thoughts } }; } } 

The fake decodes thinking first, then text, at most perRequest tokens at a time.

Three rules are borrowed from the docs on purpose.

A token whose hash does not match the request gets a 400. That is my stand-in for "the token encodes the original prompt."

The prompt count includes pos, because every continuation re-prefills what was already decoded.

If the caller's maxOutput runs out first, the finish reason is MAX_TOKENS and a token still comes back.

dropNextStreamAfter() exists only so the demo can kill a stream on purpose.

4. Step 3: The Loop

This is the most important function in the file.

type Run = { status: "done" | "raise-cap" | "failed"; parts: string[]; token?: string; segments: number; billed: Usage; // sum of every request: what you pay for initialPrompt: number; // what you sent once error?: string; }; // The loop. Same request every time. Only the token changes. async function longDecode(model: FakeModel, req: Request, maxOutput: number, prev?: Run): Promise<Run> { const frozen: Request = structuredClone(req); const run: Run = prev ? { ...prev, parts: [...prev.parts], billed: { ...prev.billed }, status: "done" } : { status: "done", parts: [], segments: 0, billed: { prompt: 0, text: 0, thoughts: 0 }, initialPrompt: count(frozen.contents) + count(frozen.systemInstruction), }; let token = prev?.token; let retries = 0; while (true) { const buffer: string[] = []; // uncommitted until the checkpoint arrives let cp: Checkpoint; try { const it = model.stream(frozen, maxOutput, token); let step = await it.next(); while (!step.done) { buffer.push(step.value); step = await it.next(); } cp = step.value; } catch (e) { const msg = (e as Error).message; if (msg === "stream dropped" && retries < 2) { retries += 1; log(` drop discarded ${count(buffer.join(" "))} uncommitted tokens, retry from the same token`); continue; } return { ...run, status: "failed", error: msg }; } retries = 0; run.segments += 1; run.parts.push(cp.text); // commit only now run.billed.prompt += cp.usage.prompt; run.billed.text += cp.usage.text; run.billed.thoughts += cp.usage.thoughts; const note = cp.usage.text === 0 ? " (thinking only, keep going)" : ""; log(` seg ${run.segments} ${cp.finish.padEnd(12)} prompt ${String(cp.usage.prompt).padStart(3)} thoughts ${String(cp.usage.thoughts).padStart(2)} text ${String(cp.usage.text).padStart(2)}${note}`); if (cp.finish === "CONTINUATION" && cp.token) { token = cp.token; continue; } if (cp.finish === "MAX_TOKENS" && cp.token) return { ...run, status: "raise-cap", token: cp.token }; return { ...run, status: "done", token: undefined }; } } 

The important part is the sequence:

  1. Freeze the request once. Every segment sends the same one.
  2. Stream text into a buffer, not into the answer.
  3. Only when the checkpoint arrives, commit the buffer and add the usage.
  4. CONTINUATION with a token: keep going, even if the text was empty.
  5. MAX_TOKENS with a token: stop and hand back the token so the caller can raise the cap.
  6. A dropped stream: throw away the buffer and retry from the same token.

That last rule is the one I like most. The previous token is still valid, so the retry starts from the last committed checkpoint. Nothing uncommitted ever reached the answer.

prev lets the caller resume with a higher cap. The Vercel PR describes the same move: pass the token back "with a higher maxOutputTokens to resume."

5. Step 4: Count the Context, Not the Bill

Two tiny functions.

// What actually sits in the context window after the decode. const contextSize = (r: Run) => r.initialPrompt + r.billed.text + r.billed.thoughts; const shouldCompact = (tokens: number, window: number) => tokens >= window * 0.8; 

run.billed.prompt is the sum of every request's input. That is closer to what you pay for. It is not what sits in the window.

What sits in the window is the prompt you sent once, plus the thoughts and text that came back.

The guide points harnesses at initial_prompt_token_count for exactly this, and says to wait until the decode finishes before compacting. The 80% threshold is an example.

Summed input tokens vs actual context, example inputs from the demo

6. Step 5: The Loops People Write First

Two loops I can picture myself writing on a tired afternoon.

// The two loops people write first. async function naiveDecode(model: FakeModel, req: Request, maxOutput: number, mode: "ask-to-continue" | "keep-partials"): Promise<string> { let r = req; let token: string | undefined; let text = ""; for (let attempt = 0; attempt < 8; attempt += 1) { try { const it = model.stream(r, maxOutput, token); let step = await it.next(); while (!step.done) { text += (text ? " " : "") + step.value; // straight into the answer step = await it.next(); } if (step.value.finish === "STOP") return text; token = step.value.token; if (mode === "ask-to-continue") r = { ...req, contents: `${req.contents} ${text} Please continue.` }; } catch (e) { log(` naive error: ${(e as Error).message}`); if (mode === "ask-to-continue") return text; } } return text; } 

ask-to-continue does the thing the guide says not to do. It pastes the partial output into the prompt and adds "Please continue." That changes the request, so the token stops matching.

keep-partials streams chunks straight into the answer. When a stream drops, it retries from the same token and appends the segment again.

Both look reasonable. Both only break on long outputs.

Then the demos:

const files = Array.from({ length: 150 }, (_, i) => `src/billing/file${i}.ts`).join(" "); const req: Request = { contents: `Plan a safe migration for these files: ${files}`, systemInstruction: "You are a careful refactoring agent.", tools: ["read_file", "run_tests"], }; const answer = ( "Move the shared money helpers into packages/money first and keep the old imports as re-exports. " + "Then migrate invoices, refunds, and tax in that order, because refunds import invoices and tax imports both. " + "Run the contract tests after each package, not at the end. " + "Keep the feature flag off until the refund snapshots match byte for byte. " + "Delete the re-exports only after one full release with zero imports from the old path. " + "If any step fails, stop, revert that package alone, and report which test broke and why." ).split(" "); const WINDOW = 1000; // example context window for the demo, not Argon's const model = () => new FakeModel(40, 60, answer); const joined = (r: Run) => r.Source:https://dev.to/bobbyhalljr/google-launched-gemini-4-argon-lets-build-a-tiny-continuation-loop-in-typescript-5248

Related posts