AI & ML
Zero-Shot Spoken Language Understanding with SLAM-ASR: What Do Monolingual ASR-Trained Linear Projectors Enable?
arXiv eess.AS0 views
arXiv:2610.20844v1 Announce Type: cross Abstract: The SLAM-ASR framework connects the features of a frozen speech encoder as input to a frozen LLM decoder to enable speech transcription. In this work we ask whether this training-on speech transcription-provides the LLM with the capability to also perform spoken language understanding (SLU) tasks, by training monolingual projectors on 11 languages. When prompted to answer a yes/no question about the input speech, the mean instruction following rate (IFR) is 49.8%. When not following the instruction, these models default back to transcription. The transcription quality-measured by the character error rate (CER)-is not sufficient to predict IFR. Remarkably, when the model is instructed to answer the topic question as a single digit option, the mean IFR rises to 81.3%. Our models score a mean 23.2% topic balanced accuracy (BAcc) against 14.3% chance. However, the same decoder scores a mean 71.5% BAcc when using reference transcripts as text input. This indicates a failure of the projector to transfer information for the SLU tasks.
Source:https://arxiv.org/abs/2610.20844PreviousBridging the Gap: Converting Read Text to Conversational DialogueNextMIDI-REPA: Improving Text-to-Music Generation with Symbolic Representation Alignment
Related posts

What Arconatese 3-4 Pontedera tells us about Serie D - Girone D
0dev.to Watercooler
Your AI agent got an HTTP error. Should it retry?
0dev.to Discuss
Is AI dangerous, or just misunderstood?
0dev.to Discuss
Why we price a developer tool per fix instead of per seat
0dev.to Productivity
My LLM-Powered Sidekick for Painless PR Descriptions
0dev.to Productivity
Reward-Adaptive Backtracking GFlowNet: A Technique Worth Understanding
0dev.to Productivity