TL;DR: The S1MB decision-model leaderboard now has a Japanese track. Darwin-27B-ZTC-v5 took first place there with a Borda score of 84.57, ahead of Perplexity (80.86), torchcast (80.49), and Jev 1.13 (75.43). The same model is also first on the English track (90.80 across 160 models). The striking part: it received no Japanese-specific training. Decision skills learned in English transferred to Japanese.
What is S1MB and the Japanese track?
S1MB (System One Mosaic Benchmark) scores decision models, models that return a judgment instead of generated text, across many specialized benchmarks and averages the per-benchmark ranks into one Borda score. The operator added a Japanese track (japanese-v1, 27 benchmarks) and measured the models directly. Live board: https://hotchpotch-s1mb-leaderboard.hf.space/?category=japanese-v1
The result
- Borda 84.57, which is 3.71 points ahead of second place.
- Task average 60.05, ahead of the next model at 58.43.
- First place on 10 of 27 benchmarks, top three on 16 of 27.
Why cross-lingual transfer matters here
Darwin ZTC was trained as a decision engine in English. It was not given a Japanese decision-training set. Yet the capabilities that make a good judge, reasoning over traps, moral judgment, and probability calibration, carried over to Japanese almost intact.
That is the useful signal for anyone deploying decision models in more than one language: a strong judge is not just memorizing one language's patterns. It is learning a transferable skill.
Head-to-head on the Japanese track
Against each of the top three competitors, Darwin ZTC wins the majority of benchmarks: 15 to 9 versus Perplexity, 15 to 11 versus torchcast, 15 to 10 versus Jev 1.13.
Selected wins:
- JMMLU (Japanese knowledge exam): 88.4, tied best.
- Commonsense moral judgment: 88.0, first.
- SOBACO (Japanese cultural commonsense): 72.5, first.
- JaNLI (Japanese natural-language inference with trap sentences): 83.7.
- JSICK semantic relatedness: 56.6.
- Operator-authored math and formal logic: first on all three types.
FAQ
Was the model fine-tuned on Japanese decision data?
No. It was trained as an English decision engine. The Japanese result comes from transfer.
What is a Borda score?
Each benchmark's ranking is converted to points and averaged, so no single task dominates the headline.
Where can I verify this?
On the public board: https://hotchpotch-s1mb-leaderboard.hf.space/?category=japanese-v1 and the English track at ?category=english-v1
Why does a two-language lead matter?
Routing, scoring, and safety gating run in every language a product serves. A judge that transfers lowers the cost of going multilingual.
Further reading
- VIDRAFT Engineering Notes: Decision models that judge in one forward pass
- VIDRAFT Engineering Notes: What first place on S1MB English means
