AI & ML
Computed Permutation Trellis Coding: Near-Optimal Quantization from Two Multiplications and a 64-Byte Table
arXiv math.IT0 views
arXiv:2611.22019v1 Announce Type: new Abstract: We propose Computed Permutation Trellis Coding (CPTC), a low-bit quantization format for tensors written once and read many times. Formats that round each value on its own, such as INT4 and FP4, waste a large part of their bits against the information-theoretic ideal. Trellis codes recover this loss by encoding values jointly, but prior decoders either need a large lookup table for their best quality or compute values with a hash that restricts what they can decode. CPTC's decoder multiplies its state by two coefficients and adds the top bits of the products to index a 64-byte table. Because that sum is not uniformly distributed, the rarely used ends of a small table can hold values far out in the tails. One decoder serves every bit width from 1 to 6, reproduces the standard integer and floating-point formats exactly, and comes within 0.07 bits of the ideal at 4 bits. It matches a random-permutation trellis code whose table is thousands of times larger, and exceeds every QTIP codebook. In LLM weight quantization without fine-tuning, CPTC has the lowest KL divergence to the unquantized model at every tested bit width on Llama 3.1 8B and 70B Instruct against QTIP's codebooks (by up to 27%). The decoder has been built into the AWS Trainium 4 AI accelerator, where it runs at the throughput of the native FP4 and FP8 formats for a negligible fraction of the systolic array's area. Fused encoding kernels quantize an 8B model's linear layers in under two minutes on one GPU and, in cycle-level simulation, on one Trainium 4 chip.
Source:https://arxiv.org/abs/2611.22019PreviousAn upper bound on non-overlapping codesNextSubsystem Stabilizer $(r,\delta)$-Locally Recoverable Codes: Characterization, Bounds and Constructions
Related posts
Le Groupe Figaro and OpenAI partner to bring Le Figaro journalism to ChatGPT
0OpenAI Blog
Web 上已经有 GPT-6.2,本地 Codex 却没有?先对入口,再更新客户端
0dev.to
영지식 증명 프라이버시와 규제 준수: 우리는 정말 두 마리 토끼를 다 잡을 수 있을까?
0dev.to
My context grew 6x in three months. That decides the cache question
0dev.to AI
How to connect Joist to any MCP client: Claude, ChatGPT, Cursor
0dev.to AI
Gemini Notebook cria sites interativos a partir de anotações
0dev.to AI