TL;DR
Our partner VIDRAFT released POCKET-27B-Pi, a coding agent packaged as one 16,547,400,320 byte GGUF (Q4_K_M, 16.5 GB). On the partner's own Terminal-Bench 2.1 run under the pi agent, it solved 28 of 48 shared tasks (58.3%), against 19 of 48 (39.6%) for the original Qwen3.8-27B. For edge and on-prem builders the more useful part is the memory budget: the model keeps K/V cache in only 16 of its 64 layers, so a 64K context costs 4.0 GiB in f16, 2.13 GiB in q8_0 and 1.13 GiB in q4_0. Weights plus a 64K cache fit in about 19.5 GiB, which is inside one 24 GB GPU. Below: the byte math, the config fields to read, a serving command, and the benchmark table.
Disclosure: VIDRAFT is a technical partner of Ginigen. The benchmark numbers are VIDRAFT's own measurements, and the memory math is ours, computed from the public files.
What exactly is in the 16.5 GB file?
POCKET-27B-Pi is the open Qwen3.8-27B base with the pi coding-agent adapter merged into the weights, then quantized. The Hugging Face repo FINAL-Bench/POCKET-27B-Pi-GGUF ships two files, and the sizes below come from the repo's file listing:
| File | Bytes | Size |
|---|---|---|
| POCKET-27B-Pi-Q4_K_M.gguf | 16,547,400,320 | 15.41 GiB (16.5 GB) |
| POCKET-27B-Pi-Q6_K.gguf | 22,082,529,920 | 20.57 GiB (22.1 GB) |
The license tag on the repo is Apache-2.0, and the adapter is already merged, so there is one file to load and no adapter step at serving time. This is the size class that matters for a workstation, a mini PC with a 24 GB GPU, or an on-prem box in a developer's office.
How much RAM does a 64K context cost?
Weights are a fixed cost. The KV cache grows with every token, and it decides whether a long coding session fits. The usual shortcut "layers x KV heads x head dim" overestimates here, because Qwen3.8-27B is a hybrid. Read these fields from its config.json (text_config):
import json, urllib.request cfg = json.load(urllib.request.urlopen( "https://huggingface.co/Qwen/Qwen3.8-27B/resolve/main/config.json"))["text_config"] layers = cfg["num_hidden_layers"] # 64 full = cfg["layer_types"].count("full_attention") # 16 (1 in every 4) kv_heads = cfg["num_key_value_heads"] # 4 head_dim = cfg["head_dim"] # 256 bytes_per_tok_f16 = 2 * full * kv_heads * head_dim * 2 # K and V, 2 bytes each print(bytes_per_tok_f16) # 65,536 print(bytes_per_tok_f16 * 65536 / 2**30) # 4.0 GiB at 64K Only the 16 full-attention layers store K and V per token. The other 48 are linear-attention layers whose state does not grow with context. Per token that is 2 x 16 x 4 x 256 x 2 = 65,536 bytes in f16. For the cache types llama.cpp offers, the block formats cost 1.0625 bytes per element for q8_0 and 0.5625 for q4_0:
| Cache type | Bytes per token | 64K context | 256K context |
|---|---|---|---|
| f16 | 65,536 | 4.00 GiB | 16.0 GiB |
| q8_0 | 34,816 | 2.13 GiB | 8.5 GiB |
| q4_0 | 18,432 | 1.13 GiB | 4.5 GiB |
The grey bars are illustrative: they show what the same dimensions would cost if all 64 layers cached K/V. The hybrid layout gives a cache four times smaller at every context length, which is why a 27B model can hold a long agent session on a single card.
Does it fit in 24 GB?
Add it up for the Q4_K_M build at 64K context with an f16 cache:
- Weights: 15.41 GiB
- KV cache (16 attention layers): 4.00 GiB
- Linear-attention state: 48 layers x 48 value heads x 128 x 128 x 4 bytes (fp32) = 144 MiB, constant in context
- Subtotal: about 19.55 GiB, leaving a little over 3 GiB of a 24 GiB card for compute buffers
Switching the cache to q8_0 returns about 1.9 GiB of that, which is useful if you want a second concurrent session. The partner's benchmark ran eight agent sessions in parallel on one GPU, so concurrency headroom is the number to watch when you size your own box. Use this checklist before you ship:
- Read
layer_types,num_key_value_headsandhead_dimfrom the config, not from a blog post. - Compute bytes per token for the cache type you will run.
- Multiply by context length times the number of parallel sessions.
- Add the file size and about 3 GiB for compute buffers, then compare with free VRAM.
Which result did VIDRAFT measure?
The partner put four builds on the same machine with the same job: Terminal-Bench 2.1 under the pi coding agent, thinking off, one attempt per task, one hour per task, eight agent sessions in parallel on one GPU. The table counts the 48 tasks that all four builds completed.
| Build | File size | Tasks solved | Score |
|---|---|---|---|
| Qwen3.8-27B, original weights | 22.4 GB | 19 of 48 | 39.6% |
| Pi model, full precision | 53.8 GB | 21 of 48 | 43.8% |
| POCKET-27B-Pi Q6_K | 22.1 GB | 27 of 48 | 56.3% |
| POCKET-27B-Pi Q4_K_M | 16.5 GB | 28 of 48 | 58.3% |
That is +47% over the original model and +33% over the 53.8 GB full-precision pi build, from a file 3.3 times smaller than the latter. Head to head on 50 shared tasks, the Q4_K_M build solved 30 and the full-precision original 23. Between the two POCKET sizes, Q4_K_M and Q6_K each solved 39 of 62 head-to-head tasks, so the smaller file is the one to ship.
The partner also reports that, on the same 50 tasks, POCKET-27B-Pi solved 29 with a median of 15 agent turns and no run reaching the 100-turn cap, while POCKET-27B (the unchanged base weights) solved 19 with a median of 36.5 turns and 14 runs at the cap. Fewer turns means fewer generated tokens, which on a local box is time you get back.
Why does a smaller file finish more tasks?
An agent works against a clock. Every task has a one-hour budget and every turn waits for the model. A 16.5 GB build reads fewer bytes per generated token than a 53.8 GB build, so on the same GPU it completes more turns, and more whole tasks, before time runs out. Decode speed on consumer hardware is largely a memory-bandwidth question: tokens per second is roughly bandwidth divided by the bytes read per token, so a file one third the size has a ceiling about three times higher. It is the same lesson as our earlier byte-math posts on KV cache and MoE slimming, applied to a coding agent.
How do I run it locally?
llama-server -hf FINAL-Bench/POCKET-27B-Pi-GGUF:Q4_K_M \ -ngl 999 -c 65536 --port 8080 --alias pocket-27b-pi \ --temp 0.8 --top-p 1.0 --top-k 0 --min-p 0 \ --chat-template-kwargs '{"enable_thinking":false}' Add --cache-type-k q8_0 --cache-type-v q8_0 if you need the extra concurrency headroom computed above, and run your own task set to confirm. Then point the pi agent at the local endpoint through ~/.pi/agent/models.json with baseUrl set to http://localhost:8080/v1. Once the file is downloaded, the whole loop runs with no network, and source code stays on the machine.
What should I measure on my own hardware?
A budget on paper is a starting point, so confirm it with three quick checks before you commit a team to a local coding agent:
- Peak VRAM during a real task, not an idle prompt. Watch
nvidia-smiwhile the agent runs a long repository search, because tool output pushes the context toward the limit. - Turns and generated tokens per solved task. If your median turn count is far above 15 on your own tasks, the bottleneck is the prompt or the tool setup, not the model file.
- Wall-clock time per task at one session and at four. The gap tells you whether to spend spare memory on a quantized cache and more concurrency, or on a longer context.
Record the numbers next to the llama.cpp build and the GGUF file hash, so a later model swap is a controlled comparison rather than a feeling.
FAQ
How big is POCKET-27B-Pi?
The Q4_K_M file is 16,547,400,320 bytes (16.5 GB) and the Q6_K file is 22,082,529,920 bytes (22.1 GB).
How much memory does a 64K context need?
4.0 GiB in f16, 2.13 GiB in q8_0 and 1.13 GiB in q4_0, because only 16 of the 64 layers keep a K/V cache.
Does it fit on a 24 GB GPU?
Q4_K_M plus a 64K f16 cache is about 19.55 GiB before compute buffers, so yes, with room for a quantized cache and more sessions.
What score did it get?
28 of 48 shared Terminal-Bench 2.1 tasks (58.3%) under the pi agent with thinking off, measured by VIDRAFT, against 39.6% for the original Qwen3.8-27B.
What license is it under?
The repo is tagged Apache-2.0.
Links
- Model: https://huggingface.co/FINAL-Bench/POCKET-27B-Pi-GGUF
- Base model config: https://huggingface.co/Qwen/Qwen3.8-27B
- Original write-up by VIDRAFT AI Media: https://vidraft-ai-media.static.hf.space/a/pocket-27b-pi-coding-agent-on-one-gpu-2026-11-20.html

