Korean AI Startup VIDRAFT Hits #1 on Google's Fast Gemma Challenge with 6× Inference Speedup

TL;DR: Seoul-based Pre-AGI startup VIDRAFT topped the global leaderboard of Google and Hugging Face's Fast Gemma Challenge, achieving 510.58 tokens per second on the shared benchmark model — roughly 6× faster than the baseline — while maintaining output quality. The result was independently verified by the contest organizers, making it directly relevant to any engineer thinking about inference optimization and cost efficiency at scale.

What it is

The Fast Gemma Challenge is a competitive inference-optimization benchmark co-organized by Google's Gemma team and Hugging Face. Every participant receives:

• The same public multimodal model (gemma-4-E4B-it)
• The same hardware (a standardized GPU instance provided by the organizers)
• A mandate to improve software-only throughput — swapping the model, disabling modalities (text, image, audio), or any other architectural shortcut is explicitly prohibited

Rankings are determined exclusively by VERIFIED scores: the organizers re-run each submission against a private prompt set and independently measure both speed and output quality. Self-reported numbers are not accepted. This design makes the leaderboard resistant to gaming or cherry-picked benchmarks.

VIDRAFT's autonomous AI agent vidraft-darwin claimed the #1 verified position on this leaderboard.

How it works

At a conceptual level, VIDRAFT's result reflects a pure software optimization approach applied on top of an unchanged model checkpoint. The organizers' rules forced competitors to extract performance gains entirely through the software stack — covering areas such as kernel-level execution, memory layout, batching strategies, and scheduling — rather than through model compression or capability removal.

VIDRAFT applied optimization techniques drawn from two of its internal technology pillars:

• VK-series inference engine — VIDRAFT's proprietary inference runtime, developed in-house to accelerate large model serving.
• POCKET platform — VIDRAFT's on-device AI platform, designed to run large models in resource-constrained environments without a dedicated GPU. The engineering discipline required for on-device efficiency appears to have informed the team's approach to squeezing throughput from a fixed hardware budget.

The key insight the team emphasizes is that inference optimization is not a trade-off between speed and quality — the verified result explicitly requires both. The 510.58 TPS figure was accepted only after the organizers confirmed that output perplexity remained within acceptable bounds (PPL 2.3929).

Benchmarks & results

All numbers below come directly from the verified leaderboard record and the press announcement:

The practical implication the company highlights: a 6× throughput increase means you could serve the same workload with approximately one-sixth the GPU capacity compared to an unoptimized baseline — a direct reduction in inference infrastructure cost.

The verified results are publicly visible on the Fast Gemma Challenge official leaderboard hosted on Hugging Face.

How to try it

The source article does not announce a publicly released model checkpoint, open-source repository, or API endpoint associated with this specific challenge submission. If you want to explore VIDRAFT's public work:

• Leaderboard: Check the Fast Gemma Challenge leaderboard on Hugging Face Spaces to see the verified submission details firsthand.
• VIDRAFT on Hugging Face: Search for vidraft on huggingface.co to find any publicly released models or demos from the team (including their Darwin and AETHER model families mentioned in company materials).
• Official channels: Monitor VIDRAFT's announcements for any planned open release of the VK inference engine or POCKET platform.

No public pip install, API endpoint, or model download command is confirmed in this release — do not use any that are not officially published by VIDRAFT.

FAQ

Q: Was the speedup achieved by quantizing the model or stripping out modalities?
A: No. The Fast Gemma Challenge rules explicitly prohibit replacing the model or disabling any of its capabilities (text, image, audio). The 6× speedup is a software-stack-only result on the unmodified gemma-4-E4B-it checkpoint.

Q: How trustworthy is the #1 ranking — could participants inflate their scores?
A: The contest uses a VERIFIED scoring system: the organizers re-run every submission on a private prompt set using their own infrastructure. Only these independently reproduced scores count toward the leaderboard. Self-reported or locally measured numbers are not accepted, so score inflation is structurally prevented.

Q: What hardware did VIDRAFT use, and can I replicate it?
A: All participants were given the same standardized GPU instance by the organizers. Because the optimization is software-only and hardware is held constant across all competitors, the gains are attributable entirely to VIDRAFT's software stack — not access to exotic hardware.

Q: Does this result apply to other models beyond Gemma?
A: The challenge was scoped to gemma-4-E4B-it. Whether the underlying techniques generalize to other model families is not addressed in this announcement, though VIDRAFT's VK inference engine and POCKET platform are described as applying to large models broadly.

Originally reported by AI이코노미뉴스 (2026-08-04) — source article.