AI & ML
Collapse, Repair, and Drift: Dissecting Fine Tuning Dynamics in Speech Language Models
arXiv cs.SD0 views
arXiv:2610.18192v1 Announce Type: new Abstract: Supervised fine tuning (SFT) is commonly monitored through training and evaluation loss, but loss does not fully characterize how model functionality evolves along the same trajectory. We study this gap through dialect adaptation of Qwen3-ASR, MiMo-Audio, and FireRed-LLM across learning rates from $10^{-5}$ to $10^{-3}$. Character error rate (CER) reveals pronounced transient collapse and recovery whose severity and temporal structure are only weakly reflected by loss. Bidirectional layer patching reveals strong cross checkpoint functional incompatibility during recoverable collapse, with model dependent depth profiles that reorganize during recovery. Pairwise gradient similarity alone does not reliably indicate optimization stability, whereas raw gradient spectral analysis provides a clearer structural distinction between stable and unstable optimization: successfully converged runs approach a broad and temporally stable spectral energy distribution, while failed runs remain spectrally concentrated or exhibit persistent fluctuations. Finally, AdamW update transfer bypasses pronounced early collapse, yet substantial Mandarin degradation still emerges later in training. These results motivate three separable SFT dynamics: collapse, repair, and drift, and show that transient collapse is not necessary for persistent source domain degradation.
Source:https://arxiv.org/abs/2610.18192PreviousFish Audio Drama 3 Technical ReportNextGeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognition
Related posts

Copilot Fusion's Local Model Preview: Only One of Three Stages Runs on Your Machine
0dev.to AI

Meituan shipped LongCat-2.5
0dev.to AI
Standard bf16 Rounding Discards 99% of Your RL Policy Updates
0dev.to AI
Why Are Startups Choosing Coworking Space in Pune in 2026?
0dev.to AI
What a 32GB GPU actually lets you run locally
0dev.to AI

LLM weak random number generator detection test
0dev.to Security