AI & ML
Persona-Model Collapse in Emergent Misalignment
arXiv cs.CR0 views
arXiv:2605.12850v3 Announce Type: replace-cross Abstract: Fine-tuning large language models on narrow data with harmful content produces broadly misaligned behavior on unrelated prompts, a phenomenon known as emergent misalignment. We propose that emergent misalignment involves persona-model collapse: a deterioration of the internal representations that encode and distinguish personas, undermining the nuance and consistency of their expression. We first test this hypothesis behaviorally, by comparing how models fine-tuned on harmful data and benign controls answer questionnaires while role-playing diverse personas. Across four model families, harmful fine-tuning produces more polarized, all-or-nothing answers, suggesting more binary, less nuanced persona expression. Histograms show increased concentration at the scale endpoints, and Kullback--Leibler (KL) divergence quantifies the distributional shift. We also find reduced within-persona consistency: repeated answers to the same question under the same persona become less stable. We reproduce these observations across diverse harmful datasets and questionnaires that probe different aspects of persona-conditioned behavior, while capability and simple instruction-following controls argue against a uniform loss of general ability. Finally, a mechanistic analysis of persona-conditioned activations in Qwen3-235B shows that representations elicited by different persona prompts span fewer effective dimensions in the misaligned variant than in the original and benign-control variants, most clearly in the final layers. Together, these findings provide behavioral and representational support for persona-model collapse as an aspect of emergent misalignment.
Source:https://arxiv.org/abs/2605.12850PreviousAnalysis of LLM Vulnerability to GPU Soft Errors: An Instruction-Level Fault Injection StudyNextGeneric Number-of-Copies Amplification for Pseudorandom States
Related posts
ASUS Launches New Zenbook 14 in India With Ceraluminum Design and OLED Display
0FOSS Bytes
TOP BEARINGS SRL conduce în Intermedieri produse diverse Argeș
0dev.to Data
How a Daily Silver Benchmark Is Computed (And Why Unit Conversion Is the Hard Part)
0dev.to Data
Lakewood Shooting: One Police Officer Killed, Three Wounded in Colorado
0dev.to News
OpenAI: Images 2.6, GIF e Workspace
0dev.to News
How to Plan a Healthtech Product Video Without Sounding Like a Brochure
0dev.to Design