AI & ML
TORUS: A Unified Closed-Loop Benchmark for Unified Audio Models
arXiv cs.SD0 views
arXiv:2607.28896v2 Announce Type: replace Abstract: Unified audio models capable of audio understanding, audio generation and, increasingly, audio editing are proliferating rapidly. However, each capability is evaluated in isolation on specialized benchmarks and a basic question remains unanswered: can these models understand the audio they generate/edit i.e. do they exhibit audio understanding-generation self-coherence (UGC_OWN)? We introduce TORUS, an audio-native unified benchmark for holistic, closed-loop evaluation of unified audio models comprising 1,053 questions (117 tests) spanning 21 tasks and 8 audio domain settings across all 3 audio types. We evaluate unified models and cascades of specialized models across objective and subjective metrics. Best unified models are able to answer only 52.4% questions about their generations, while gpt-audio can answer 53.6% questions. We hypothesize that a truly unified audio intelligence should be highly self-coherent and thus position understanding-generation coherence as an essential test for future unified audio systems. Further details are on our project page: https://torus-benchmark.github.io/
Source:https://arxiv.org/abs/2607.28896PreviousDescriptor: Certus Caliber Classification Gunshot Dataset (C3GD)NextBeyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions
Related posts
Introducing the Grid Protectors Program
0OpenAI Blog
The world has a new plan to save nature, but it still needs the money
0Ars Technica

Could data center backlash flip Florida?
0The Verge

Vivo’s first Pro Max phone proves its Ultra is still in a league of its own
0The Verge

Why Peacock is making a huge bet on YouTube
0The Verge
Ninja’s first-ever microwave just replaced two of my kitchen appliances at once
0TechCrunch