AI & ML
Location Matters: Layer-wise Prompt Conditioning for Voice--Style Disentanglement in Text-to-Speech
arXiv cs.SD0 views
arXiv:2611.24125v1 Announce Type: cross Abstract: Independent control of voice and emotional style characteristics of speech is a long-standing problem in text-to-speech (TTS) systems. We investigate how the conditioning location of voice and style prompts across TTS decoder layers affects the trade-off between preservation and expressiveness for voice identity and emotional style. We introduce a layer-dependent gated fusion mechanism and systematically study which layers should be influenced by which prompt type. Our findings show an asymmetric relationship. Voice benefits from early conditioning for identity preservation, while style benefits from early conditioning for expressiveness. Our findings provide practical guidance that helps neural network architects make an appropriate trade-off for the architecture of TTS systems according to their use cases.
Source:https://arxiv.org/abs/2611.24125Previous21Codec: Decoder-side Speaker Control for Text-to-SpeechNextWhy Multi-Source Audio Fails to Compose: A Geometric Diagnosis and Its Remedy
Related posts
Probing Physics in Motion Diffusion Models Against Their Own Estimate
0arXiv cs.GR
NIRJALA: Draining the Sea out of Gaussian Splatting with Three-Path Optics and Reversible Physics
0arXiv cs.GR
How Music LLMs Actually Work: From Tokens to Waves (MusicGen, DiT, and Building Your Own Engine)
0dev.to Machine Learning
蒸馏 Lora RHLF 都是什么意思
0dev.to Machine Learning

Teaching an offline plant doctor to spot rust
0dev.to Machine Learning
Is Your Model Good Enough? Precision vs Recall
0dev.to Machine Learning