AI & ML
ASR-Protected Fusion and Reference-Anchored Policy Optimization for Joint ASR and Word-Level Speech Anti-Spoofing
arXiv cs.SD0 views
arXiv:2610.34156v1 Announce Type: new Abstract: Existing anti-spoofing systems report either an utterance-level score or a time-based spoof interval: the former cannot show which words are spoofed, while the latter does not directly match the transcript and must be post-aligned with words, so alignment errors can lead to wrong word-level labels. Standard ASR model outputs only text; spoof information in the audio is lost after recognition and is not passed with the transcript. Yet changing only a few words can change the meaning of a sentence, leaving downstream systems to use incorrect text without knowing which words are spoofed. We therefore treat partial anti-spoofing as tagged ASR: the model transcribes speech and marks spoofed words at the same time, directly linking spoof labels to output words. We adopt the audio LLM and use ASR-Protected Subspace Gated Fusion (APSGF) to fuse spoof cues with ASR features. Spoof cues are added only outside an ASR-protected subspace, so they do not disturb recognition features. After cross-entropy training, we fine-tune the model with Reference-Anchored Policy Optimization (RAPO). Since the SFT policy is already concentrated, multiple samples from the same input tend to receive similar rewards, resulting in weak group-wise advantages. RAPO instead generates a single sample with low sampling randomness and anchors its advantage to the reward of a reference model. This allows the model to reinforce responses that outperform the reference while suppressing those that underperform it. Experiments on five datasets demonstrate the effectiveness of our approach in jointly detecting spoofed words and preserving ASR accuracy. The code is publicly available.
Source:https://arxiv.org/abs/2610.34156PreviousEmoNet: A Lightweight Attention-Based CNN for Speech Emotion RecognitionNextAudioFM: Towards General Purpose Audio Representation Learning
Related posts
Should You Rent AI or Run Your Own?
0dev.to Machine Learning
Who's hiring this week: only 1 in 30 new tech jobs is junior, and 1 in 6 'AI jobs' is AI-training work (Oct 26-Nov 1, 2026)
0dev.to Career
ClarityHope – An Online Counselling Platform for Accessible Support
0dev.to Career
The 3 documents to have before you start any freelance project (free deposit clause inside)
0dev.to Career
How I Diagnose a Slow, Loud, or Overheating PC: A Practical Checklist
0dev.to Beginners
You Did Great for Three Days. Then Life Happened. Here Is a Habit System That Survives That.
0dev.to Beginners