Backend
The State Is Not the Bottleneck: A Paged-State Gated-DeltaNet Tile on a Tri-Mode Multicast Mesh, Bit-Exact from RTL to Silicon
arXiv cs.RO0 views
arXiv:2611.18422v1 Announce Type: new Abstract: Linear-attention language models replace a growing key-value cache with a recurrent state of fixed size. On an edge FPGA, this state can exceed the on-chip memory and must be paged to external memory. This paper asks whether such paging dominates the cost of decoding (generating tokens). We built a hardware tile that computes one complete layer of a custom 369-million-parameter hybrid Gated DeltaNet model on an Agilex 5 FPGA and pages its recurrent state. The model state needs 6.2 MiB, whereas the device has 0.87 MiB of block memory. Paging adds 26.0 MB of memory traffic per decoded token, at most 11.2 percent of the calculated total. On the board at 75 MHz, with synthetic weights, one token takes 11.15 ms in this layer. The matrix-vector products with the weights take about 62 percent of this time and the state phase, including the paging, 13 percent. The tile is one node of a multicast mesh of tri-mode processing elements. We measure the error of the fixed-point arithmetic against floating point and check that the hardware agrees bit for bit with a fixed-point reference. In simulation, the state logic agrees with the reference on all 18 recurrent layers. On the board, six builds pass all 30 runs of 64 tokens and two runs of 1,024 tokens. For this model and memory interface, the delivery of the weights limits the decoding performance, and the recurrent state does not. State paging is therefore practical for such a model on an edge FPGA.
Source:https://arxiv.org/abs/2611.18422PreviousVision-Language-Action Models: A Life-Cycle Survey of Design, Training and DeploymentNextBeyond Attack-Based Accuracy: Rethinking Adversarial Robustness Evaluation Through Structural Gradient Alignment
Related posts
Clustering App Ad Hooks in Go: TF-IDF, Cosine Similarity and a Hand-Labelled Sample, and the Walls You'll Hit (2026)
0dev.to
Building a Simple REST API with FastAPI: A Step-by-Step Guide for Beginners
0dev.to
What It Takes to Trust an AI Agent With Real Work
0dev.to
How to check that a PDF converter does not upload your file anywhere
0dev.to
One TPU v6e chip holds 927 agent sessions with host-held state and 5 with a kept chat (memory only)
0dev.to
How to redesign a website without losing Google rankings
0dev.to UX