When building context layers for AI coding agents, maximizing token efficiency is critical. But if your compression pipeline corrupts a single code block delimiter, the entire agent run fails.

In TokenCap, we built src/compress/compress.js with strict preservation gates verified by property-based testing.

The Preservation Testing Contract

In test/compress-preserve.property.test.js, we generate randomized markdown documents containing complex code fences, file paths, URLs, and JSON objects:

test("preserves fenced code blocks and URLs byte-exact during compression", () => { const sample = fs.readFileSync("test/fixtures/complex-markdown.md", "utf8"); const compressed = compressMarkdown(sample); const originalBlocks = extractCodeBlocks(sample); const compressedBlocks = extractCodeBlocks(compressed); assert.equal(originalBlocks.length, compressedBlocks.length); for (let i = 0; i < originalBlocks.length; i++) { assert.equal(compressedBlocks[i].content, originalBlocks[i].content); } });

Dry-Run Compression Preview

You can test compression on your existing .tokencap/snapshot.md before applying:

tokencap compress .tokencap/snapshot.md --dry-run

CLI Output:

Original size: 18,420 tokens (74.2 KB) Compressed size: 11,250 tokens (45.1 KB) Savings: 38.9% reduction Code blocks preserved: 14 / 14 exact matches

Learn more about safe context compression at tokencap.vansharora.app