A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise

TL;DR

ISTA's DASLab published GGUF builds of DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model, at 1.75, 2.00 and 2.25 bits per weight. The Hugging Face counter for the repository shows about 2.4 million downloads in ten days, and its card says the 2.00 build keeps 100.0% of the base model's benchmark average. We did not run these files. What we did was read the card, the per-tensor allocation files the authors ship, and the GGUF headers of the actual shards, then recount the bit budget ourselves. Three things came out of it.

First, the "2 bit" label covers only the backbone. A separate 196.6B-parameter lookup table called Engram sits outside the budget at 4.5 bits, and it is 44% of the IQ2_XXS download. Counting every stored weight, that build is 2.66 bits per weight, not 2.00. Second, the searched allocation has no simple pattern you could copy by hand: the "IQ2_XXS" build is made mostly of IQ1_M tensors, and which matrix gets more bits flips between builds. Third, the GPQA-Diamond gaps between builds are smaller than the noise of a 198-question test, so "100.0% recovery" is a statement about resolution as much as about quality.

What exactly was released?

The base model is DeepSeek-V4.1-Flash-1017: 763.2B parameters, 384 routed experts per layer with 6 active per token, and a 1M-token context window, according to the card. The quantized repository (ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF) was created on October 23, 2026, last updated October 28, and stood at about 2.4 million downloads and 551 likes when we queried the Hugging Face API on November 2.

Two methods from the same lab make the files. GSQ (arXiv 2604.18556) quantizes each tensor at a given type by learning the grid assignment and per-group scales through a Gumbel-Softmax relaxation. RCO (arXiv 2605.00649) then decides which of K quantization types each of N tensors gets, under a total size budget that it enforces exactly by treating the budget as a manifold in logit space. The output is an ordinary GGUF that llama.cpp b11870 (October 25, 2026) or newer can load, because the type chosen per tensor is a standard one such as IQ1_M or IQ2_XS.

A uniform Q2 file spends the same bits on a tensor that barely matters and on one that breaks the model if you touch it. A searched file can move bits between them while the average stays fixed.

Where do the bytes of a "2-bit" 763B model actually go?

The authors ship an allocation file per build that lists the type of all 1,043 tensors. That tells you the types but not the sizes, so we also read the headers of shards 2 to 5 directly with HTTP range requests (the tensor table sits at the start of each file) and computed bytes from the shapes and the standard ggml block layouts. For all three builds our recount lands on the card's own measured figures: 1.7499, 2.0038 and 2.2501 bits per weight over the 551.88B backbone parameters. Here is the check for the 2.00 build:

import struct, urllib.request REPO = "https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF/resolve/main/" # ggml type id -> (name, weights per block, bytes per block) TY = {0:("F32",1,4), 8:("Q8_0",32,34), 13:("Q5_K",256,176), 16:("IQ2_XXS",256,66), 17:("IQ2_XS",256,74), 29:("IQ1_M",256,56), 30:("BF16",1,2)} def header(fn, n=4_000_000): req = urllib.request.Request(REPO + fn, headers={"Range": f"bytes=0-{n-1}"}) b = urllib.request.urlopen(req, timeout=120).read() p = 4 def u(f, k): nonlocal p v = struct.unpack_from("<" + f, b, p)[0]; p += k; return v def s(): nonlocal p k = u("Q", 8); v = b[p:p+k].decode(); p += k; return v assert b[:4] == b"GGUF" and u("I", 4) == 3 n_tensors, n_kv = u("Q", 8), u("Q", 8) ints = {2: ("H", 2), 4: ("I", 4), 5: ("i", 4)} for _ in range(n_kv): # shards 2-5 carry only 3 split.* integer keys  s(); f, k = ints[u("I", 4)]; u(f, k) out = [] for _ in range(n_tensors): name = s(); dims = [u("Q", 8) for _ in range(u("I", 4))]; ty = u("I", 4); u("Q", 8) out.append((name, dims, ty)) return out tensors = [] for i in range(2, 6): # shard 1 is metadata only, shard 6 is the Engram file  tensors += header(f"DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ2_XXS-0000{i}-of-00006.gguf") def size(dims, ty): n = 1 for d in dims: n *= d name, per, by = TY[ty] return n, n // per * by P = B = Pe = Be = 0 for name, dims, ty in tensors: n, by = size(dims, ty); P += n; B += by if "_exps" in name: Pe += n; Be += by print(f"tensors {len(tensors)} params {P/1e9:.3f}B bytes {B/1e9:.2f} GB bpw {B*8/P:.4f}") print(f"experts: {Pe/P:.1%} of params, {Be/B:.1%} of bytes, {Be*8/Pe:.4f} bpw") print(f"dense path: {(P-Pe)/1e9:.2f}B params, {(B-Be)/1e9:.2f} GB, {(B-Be)*8/(P-Pe):.3f} bpw") 

It prints:

tensors 1042 params 551.881B bytes 138.23 GB bpw 2.0038 experts: 98.5% of params, 95.5% of bytes, 1.9438 bpw dense path: 8.30B params, 6.16 GB, 5.938 bpw 

The routed experts are 98.5% of the backbone's parameters and 95.5% of its bytes, at 1.94 bits. The dense path (attention, shared experts, output head, embeddings) is only 8.30B parameters, 1.5%, but it is kept at 5.94 bits on average, mostly Q5_K, with the attention KV and query-down projections at Q8_0, and that costs 6.16 GB. So the non-expert part of the model is paid for at roughly three times the expert rate. That is a sensible trade, since every token passes through all of it, and a uniform scheme cannot make it.

Then there is the part that is outside the search entirely. The shared Engram file holds one tensor, per_layer_token_embd, with shape 256 by 768,022,850. That is 196.6B parameters stored at IQ4_NL (4.5 bits), 110.6 GB, identical in every build. The card states this plainly: the bit-width quoted is "over the backbone (all tensors except the Engram table)". It is a reasonable definition, since the table is read one row per token. But if you are sizing a disk or a download, the number that matters is different:

The two parts we can read add up to 748.5B parameters out of the 763.2B the card quotes for the base model. We did not try to account for the remaining 14.7B.

Does the search follow a rule you could apply by hand?

We went looking for one, because a rule would let you quantize your own MoE without the search. The allocation files do not offer one.

Start with the labels. In the IQ2_XXS build, 60 of the 120 routed-expert tensors (40 layers times gate, up and down) are IQ1_M, 42 are IQ2_XXS and 18 are IQ2_XS. The 2.00 average comes from a mix that is mostly below 2 bits. In the IQ1_M build, 83 of 120 expert tensors are IQ1_S, the 1.5625-bit type. The build name is the budget the search was given, not the type most tensors ended up with.

Depth shows a tendency, not a law. In the 2.00 build the first ten layers average 1.77 bits in the experts and the last ten average 2.05. The IQ1_M build leans the same way (1.63 against 1.71). The IQ2_XS build does not: its four groups of ten layers average 2.10, 2.19, 2.30 and 2.19, so the peak is in the middle.

Matrix type is the clearest case against hand rules. In the 2.00 build the down projection gets fewer bits than gate and up (1.87 against 1.98). In the 2.25 build it gets more (2.28 against 2.15). Gate and up share the same type in all 40 layers of all three builds, which could be a constraint of how the search was set up or something the search learned. The card does not say which.

The practical reading is that the allocation depends on the budget, and so on the task loss the search sees, rather than on a fixed sensitivity ordering of layers. If you copy "early layers get fewer bits" into your own pipeline, you are copying a tendency from one model at one budget.

Is "100.0% recovery" a real result?

The card measures three benchmarks at reasoning effort 50 and reports the task average against the base model:

The averages and recovery percentages recompute correctly from the columns. What the table cannot tell you is how much those columns move on a rerun, and two features matter.

AIME25 is saturated. The base model scores 100.00 on a 30-problem set, so every build that also scores 100.00 shows only that the test has no headroom. It cannot reveal damage.

GPQA-Diamond has 198 questions, so each question is 0.505 points, and the percentages in the table are whole-question counts: 180/198 for the base and IQ2_XXS, 179/198 for IQ2_XS and 175/198 for IQ1_M. The 95% Wilson intervals are 86.1 to 94.2, 85.5 to 93.8 and 83.2 to 92.1. They overlap almost completely.

The sharpest case is the largest gap, base against IQ1_M, five questions out of 198. Even if all five discordant questions went the same direction, which is the best case for significance, an exact McNemar test gives p = 0.0625. One question going back the other way (six against one) gives p = 0.125. The card does not publish per-question results, so the real paired test cannot be run, but the best case already fails the usual 0.05 bar. Note also that IQ2_XS (more bits) scores one question below IQ2_XXS. That is what noise looks like, and it is not a defect in the method.

import math from math import comb def wilson(k, n, z=1.96): p = k / n; d = 1 + z * z / n c = (p + z * z / (2 * n)) / d h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d return 100 * (c - h), 100 * (c + h) def mcnemar_exact(b, c): # two-sided exact test on discordant pairs  n = b + c return min(1.0, 2 * sum(comb(n, i) for i in range(min(b, c) + 1)) / 2 ** n) for name, k in [("base/IQ2_XXS", 180), ("IQ2_XS", 179), ("IQ1_M", 175)]: lo, hi = wilson(k, 198) print(f"{name:13s} {k}/198 = {100*k/198:.2f}% 95% CI {lo:.1f} to {hi:.1f}") print("best case, 5 questions all in one direction: p =", mcnemar_exact(5, 0)) print("one flip back the other way (b=6, c=1): p =", round(mcnemar_exact(6, 1), 4)) 

base/IQ2_XXS 180/198 = 90.91% 95% CI 86.1 to 94.2 IQ2_XS 179/198 = 90.40% 95% CI 85.5 to 93.8 IQ1_M 175/198 = 88.38% 95% CI 83.2 to 92.1 best case, 5 questions all in one direction: p = 0.0625 one flip back the other way (b=6, c=1): p = 0.125 

None of this says the quantization is bad. The 2.00 build matching the base score on all three benchmarks, down to the hundredth on LiveCodeBench, is encouraging. It says the test is too coarse to rank these builds, and that a "98.7% against 100.0%" gap should not drive a decision. The card also does not state how many samples were drawn per question or the decoding seed, which matters for a model that is run at a reasoning effort setting.

Can this really run on a 12 GB GPU?

The card says each build runs on a single 12 GB GPU with the Strata engine, with the rest of the model in system memory and the Engram file kept on an SSD. It does not give a system-RAM requirement or a tokens-per-second figure, so here is the arithmetic you would do before buying hardware.

From the shapes we can compute what one token has to read. With 6 of 384 experts active per layer, the routed experts contribute 6/384 of 132.07 GB, which is 2.06 GB per token for the 2.00 build. The dense path adds 5.70 GB (excluding the token embedding, which is a single-row lookup). The Engram table adds one row.

So the dense path (6.16 GB stored) can sit in 12 GB of VRAM with some room for the KV cache, and the 132 GB of experts cannot. They live in system RAM, or get paged from SSD through memory mapping. The card does not say how Strata places tensors, so this is our assumption about the likely arrangement, not a description of the engine. Under it, the speed limit is how fast 2.06 GB can reach the GPU each token. The numbers below are illustrative ceilings that assume all six experts in every layer are fetched cold, with no expert caching and no overlap with compute:

Real throughput will differ. Routing is not uniform, so popular experts can be cached, and transfers can overlap with compute, which raises the ceiling. Small random reads from an SSD run well below sequential speed, which lowers it. The point of the table is the order of magnitude: "runs on 12 GB" and "runs at a speed you would want to chat at" are different claims, and the first is only the memory side of it. It is the same bandwidth-bound logic as sparse models on CPUs, stretched across a PCIe link.

What should you do before shipping a build like this?

Report bits per weight over everything you store, not only over the part that was searched. For this release that is 2.66 against 2.00, a difference big enough to change a hardware plan.

Read the allocation file, or parse the headers as above. It takes minutes and shows which tensors the search protected, here the dense path at about 5.9 bits.

Run your own task, with paired per-item results. If two builds differ by a handful of items on a few hundred, you cannot rank them, and a paired exact test tells you that in one line. If your task is long-form or agentic, a 198-question multiple-choice score says little about it.

Measure the whole chain on your hardware: load time, first-token latency with a cold expert cache, sustained tokens per second at your context length, and disk wear if you page from SSD. Then pin the runtime: the card's files use the llama.cpp layout from October 27 onward, the Engram file must be linked as shard 6 of each build, and older downloads need the authors' converter script.

FAQ

What is GSQ-RCO?
It is a two-step quantization recipe from the Institute of Science and Technology Austria. GSQ quantizes each tensor accurately at a chosen type, and RCO chooses the type for every tensor under an exact total size budget. The result is a standard GGUF file.

Does "2.00 bpw" mean the download is 2 bits per weight?
No. The figure is measured over the 551.9B backbone parameters only. The 196.6B-parameter Engram table is stored at 4.5 bits in every build, so the IQ2_XXS download of 248.84 GB works out to about 2.66 bits per weight over both parts.

Which llama.cpp version do I need?
The card requires b11870 (October 25, 2026) or newer, and the files have used llama.cpp's own DeepSeek-V4.1 layout since October 27. Earlier downloads can be converted locally with the script in the repository.

Is the IQ2_XXS build really lossless?
Its scores match the base model on all three benchmarks, but two of those benchmarks are small or saturated. On GPQA-Diamond, one question is 0.5 points and the 95% interval is about plus or minus 4 points, so the evidence supports "no large loss on these tests", not "identical".

Can I run it on a 12 GB GPU?
The card says yes with Strata, with the experts in system memory and the Engram file on SSD. You need roughly 132 GB for the experts plus the dense path, and speed is bounded by moving about 2 GB of expert weights per token.

Further reading

• Why a Sparse 35B MoE Runs Faster on a CPU Than a Dense 27B Model
• 0.1 Bits Per Weight Is Not 0.1 Bits Per Weight: Where the Bytes Go in Sub-2-Bit LLMs

Sources: ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF model card and allocation files, GSQ, arXiv 2604.18556, RCO, arXiv 2605.00649, llama.cpp release b11870. Byte counts, bit-widths and statistics above are our...