Your next local AI setup could start with a browser tab.
Community developer Mohd Mahmodi converted Spark-X2.6-1.7B into a 4-bit ONNX build that runs inference in the browser through WebGPU. The first visit downloads about 1.21 GB, which the browser caches for later visits.
The numbers below are the author's measurements, not a new SparkLLM benchmark run: Windows 11, desktop Chrome, NVIDIA RTX 3090 Ti, one request at a time; benchmark quality checks use thinking off. The 49 ms first-token result is for a 28-token prompt, not a long document.
A longer conversation does not have to mean slower decoding
The author's original speed sweep records 108, 104, 107, 109 and 107 tokens/s at prompt lengths of 28, 1k, 4k, 8k and 16k tokens. First-token latency does increase: from 49 ms at 28 tokens to 5.9 seconds at 16k. Decoding speed and waiting for the first token are different measurements.
The included loader keeps its KV cache on the GPU and reuses the cached prefix for follow-up chat turns. That is a useful implementation detail for developers building a local browser assistant.
What survives the conversion?
With thinking off and the same prompts and greedy decoding, the author reports:
| Check | Official BF16 in Transformers | Official INT4 in Transformers | Browser build |
|---|---|---|---|
| GSM8K, all 1,319 problems | 75.7% | 72.7% | 72.7% |
| BFCL v3 simple function calling, 200 cases | 76.5% | 52.5% | 52.5% |
| Needle retrieval, 12 cases at 1k–16k tokens | 12/12 | 12/12 | 12/12 |
The browser build matches the official INT4 scores on these checks, not the BF16 model. The function-calling gap remains, and these limited checks are not a claim of identical outputs or universal accuracy. The author also reports a 30-minute stability run with 877 generations and no errors.
Open the demo, then build your own
Try the live browser demo in desktop Chrome or Edge. Allow time for the first model download. The repository also includes a JavaScript loader, thinking controls, multi-turn chat and tool-call parsing.
Only Windows 11/Chrome/RTX 3090 Ti was tested by the author; phones, integrated GPUs and other platforms are not validated by this report. CPU/WASM is not supported by the app. Start with one tab: the author reports that two tabs in one Chrome instance can exhaust D3D12 descriptor heaps.
What would you put in a local browser assistant: private notes, a coding scratchpad, or a small document tool?
Credit: Mohd Mahmodi. Original images reproduced unchanged from the Apache-2.0 repository. Source README · Validation report · License. Source snapshot: 1a2cedace485d8bdae557729adaa8f34ea05eed1.
