Tencent's WeVisOne-VL-4B picked up 5,986 Hugging Face downloads last month, and I suspect a lot of those people expected another small chat-about-your-image model. It's a perception model instead: boxes, points, counts, text locations, segmentation masks and alpha mattes, all from one 4B checkpoint built on Qwen3-VL. You can run WeVisOne-VL-4B locally on an 8 GB GPU with a Q4_K_M GGUF in llama.cpp, and the Hub offers a one-line Ollama pull too. You just lose the masks when you do.

I went through the model card, the GitHub README, the config and the GGUF repos. This post covers what the model is, Tencent's numbers against Qwen3-VL-8B (and the baseline they left out), VRAM per format, and the two ways to run it.

What WeVisOne-VL-4B actually is

The model card says it comes from WeChat Vision at Tencent and is built on Qwen3-VL. The config.json confirms it: Qwen3VLForConditionalGeneration, 36 text layers, hidden size 2560, a 24-layer ViT with patch size 16. That's the Qwen3-VL-4B layout with a 152,680-token vocabulary.

The extra tokens are the trick. Coordinates are emitted as <0> to <1000> (normalized boxes and points), and every predicted box in dense tasks is followed by an <OBJ> token (ID 152678). A separate 9.6M-parameter pixel decoder, shipped as mask_decoder.pt, reads those <OBJ> positions plus multi-level ViT features and produces masks and stride-1 alpha mattes, so there's no SAM in the loop.

The task list from the GitHub repo: detection, referring, point, count_box / count_point, ocr / ocr_grounding, segmentation, matting.

Other facts worth knowing before you download:

  • License: Apache-2.0 for Tencent-authored code; third-party components keep their own licenses.
  • Context: max_position_embeddings is 262144, same as the Qwen3-VL base. Tencent's evals cap images at 1,960 visual tokens, so context isn't the constraint here.
  • Single-turn, single image. The pixel decoder rejects video grids, and the model returns structured output without chain-of-thought.
  • It answers None when the thing you asked for isn't in the image, which matters more than it sounds (see the benchmarks).
  • There's also a 2B sibling. Paper is arXiv:2610.31329, submitted 22 Oct 2026.

WeVisOne-VL-4B vs Qwen3-VL benchmarks, and who's missing

All numbers below are from the GitHub README. Tencent compares against Qwen3-VL-8B, Qwen3.5-9B, Rex-Omni-3B, LocateAnything-3B and a few specialists.

Task (metric) Qwen3-VL-8B Qwen3.5-9B Rex-Omni-3B LocateAnything-3B WeVisOne-VL-4B
LVIS detection (F1) 42.9 43.7 46.9 50.7 55.3
Dense200 detection (F1) 13.8 18.0 58.3 58.7 71.3
RefCOCOg test (F1) 74.9 74.6 74.3 77.6 81.8
WeRef rejection rate 18.3 70.7 43.3 73.0 96.4
PixMo-Count acc n/a 80.4 71.9 82.8 89.6
TotalText OCR (F1) 35.4 n/a 40.6 43.3 52.0
SROIE OCR (F1) 43.3 n/a 44.8 40.3 68.1

On matting (MAD ×10³, lower is better), WeVisOne-VL-4B scores 9.8 on Tencent's WeRefMat referring benchmark against 22.4 for a Qwen3.5-9B + SAM2-Matting pipeline. On ReasonSeg val it gets 71.2 gIoU vs 65.0 for a fine-tuned LISA-13B.

The rejection numbers are the ones I care about. Qwen3-VL-8B on the WeRef set rejects absent targets 18.3% of the time, which means it draws a box around something that isn't there most of the time. At 96.4% you can put the detector in a pipeline; at 18.3% you end up babysitting it. One caveat: WeRef-Bench and WeRefMat are Tencent's own benchmarks.

The 2B sibling is closer than you'd expect: 53.3 on LVIS, 92.9% WeRef rejection, 88.1 on PixMo-Count. If the 4B doesn't fit your card at bf16, the 2B through the official package is the one to try before you give up on masks.

I do have complaints about the comparison. Qwen3-VL-4B isn't in the tables. This model is a 4B built on Qwen3-VL, and Tencent compares it with the 8B. I'd like to know how much of the gain comes from the fine-tuning and how much from the base, and the one baseline that would tell us is missing.

There's no Gemma and no general VLM benchmark either: no DocVQA, no OCRBench, no MMMU, no chart QA. That fits what the model is for, but it leaves the "vs Qwen VL and Gemma" question unanswered for general image chat, and nobody has published those numbers yet.

People are already questioning the baseline selection. Discussion #3 on the model page asks how the baselines were selected, pointing out that the GroundingPI paper reports additional baselines and higher scores on some overlapping benchmarks, and how the model compares with frontier proprietary models. No reply at the time of writing.

How much VRAM it needs: bf16 vs Q8_0 vs Q4_K_M

Tencent publishes no VRAM figures. The table below is estimated from file sizes. These are weight sizes only; activations and KV cache come on top.

Format Weights Vision projector Total on disk 8 GB card?
f16 GGUF 8.84 GB 0.84 GB (f16 mmproj) ~9.7 GB No
Q8_0 GGUF 4.70 GB 0.45 GB (Q8_0 mmproj) ~5.2 GB Yes
Q4_K_M GGUF 2.72 GB 0.45 GB ~3.2 GB Easily

Sizes are the actual file sizes in mradermacher/WeVisOne-VL-4B-GGUF; the repo's README table rounds them up (2.8 GB for Q4_K_M, 0.6 GB for the Q8_0 mmproj). The Hub lists the original checkpoint at 5B params in BF16, so the official bf16 path is roughly 10 GB of weights. That doesn't fit on an 8 GB card without offloading.

There is one measured data point, from the PALLER-NOVUS GGUF repo: in a 20-query pilot on an RTX 2060 6 GB (context 4096, 1,024 to 1,960 image tokens), 4B Q4_K_M peaked at 4.38 GiB of whole-device VRAM at 4.17 s per request. The 4B F16 GGUF had to spill onto the CPU on that card and took about 60 s per query. My guess is that 12 GB is the realistic floor for the Tencent package at bf16, but nobody has published a measured number yet.

Run it with the official package (the only way to get masks)

The Tencent package is what loads mask_decoder.pt. Install it from the repo:

git clone https://github.com/Tencent/WeVisOne-VL.git && cd WeVisOne-VL conda create -n wevisone-vl python=3.10 -y && conda activate wevisone-vl pip install -r requirements.txt && pip install -v -e . 
from PIL import Image from wevisone_vl import WeVisOneVL from wevisone_vl.visualization import save_result model = WeVisOneVL("tencent/WeVisOne-VL-4B", backend="transformers") # or backend="vllm"  image = Image.open("demo/examples/images/seabreeze.jpg").convert("RGB") result = model.predict(image, task="matting", query="person") save_result("outputs/matting", image, result)

That's the README example. Swap task for detection (with a list of categories as query), ocr, count_box and so on. The README also lists options for device ("auto", "cuda:0", "npu:0", "cpu"), dtype, min_pixels / max_pixels and the usual decoding arguments like max_new_tokens and temperature. If you're short on memory, lower max_pixels before anything else; fewer visual tokens means less activation memory. There's a Gradio demo too:

CUDA_VISIBLE_DEVICES=0 python demo/app.py --model tencent/WeVisOne-VL-4B

Add --backend vllm to run it on vLLM instead of transformers.

The Hub's "Use this model" menu also offers a plain vllm serve "tencent/WeVisOne-VL-4B" and an SGLang launch command. Since the architecture is stock qwen3_vl, those serve the language model only: you get coordinate tokens back, parse them yourself, and never see a mask.

The Hub's transformers snippet for the model asks "What animal is on the candy?" That's the generic image-text-to-text example, so don't judge the model by free-form chat; it was trained for structured outputs.

GGUF in Ollama and llama.cpp: boxes yes, masks no

There are three community GGUF repos for the 4B so far: mradermacher, prithivMLmods and PALLER-NOVUS, which ships 2B and 4B in one repo. The Hub's "Use this model" menu on the mradermacher repo gives:

llama-server -hf mradermacher/WeVisOne-VL-4B-GGUF:Q4_K_M ollama run hf.co/mradermacher/WeVisOne-VL-4B-GGUF:Q4_K_M

The problem is that a GGUF is the language model plus an mmproj vision projector. The pixel decoder is a separate PyTorch file, and neither llama.cpp nor Ollama knows what to do with an <OBJ> token. So on an 8 GB card you should get detection, referring boxes, pointing, counting and OCR grounding as coordinate text, but not segmentation or matting. PALLER-NOVUS says the same: putting mask_decoder.pt next to a GGUF doesn't give you pixel output. Their Q4_K_M runs did return structured boxes on a pinned, patched llama.cpp build (they say the patch carries changes for another model and adds no WeVisOne capabilities, but stock upstream is untested), launched with --special so the coordinate tokens survive decoding. I haven't seen anyone confirm that Ollama's hf.co pull picks up the separate mmproj for this architecture, so test image input there before you build on it. There are no imatrix quants yet either; mradermacher only has static ones so far.

Is it the vision model to run on an 8 GB card?

For grounding work, I think yes. If your local pipeline is GroundingDINO for boxes plus SAM for masks, or a general VLM you keep asking for coordinates, this is the first small model I'd test as a replacement, mostly because of the rejection rate. Q4_K_M plus its projector at ~3.2 GB leaves plenty of room.

For "read this screenshot and tell me what the error says", I wouldn't switch. The OCR benchmarks here measure finding and transcribing text regions with boxes, not answering questions about a document. Stay on whatever general Qwen3-VL build you already use until someone posts DocVQA or OCRBench numbers for this one.

And if you want masks on 8 GB, you're waiting on a quantized path for the official package, or on someone wiring the pixel decoder into llama.cpp.

Which small vision model do you actually use for OCR or screenshots locally right now, and would you trust a 4B model's boxes without a human checking them?

Sources