AI & ML
Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment
arXiv cs.IR0 views
arXiv:2610.07023v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable capabilities but remain vulnerable to jailbreak attacks that elicit harmful or unsafe outputs. Existing safety alignment approaches, including Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), often require substantial attack-specific supervision and computational resources, while remaining susceptible to shallow safety alignment and over-refusal. To address these challenges, we introduce SSRFT(Supervised Safe-Role Fine-Tuning), the first framework that reformulates safety alignment as the internalization of a predefined safe role. SSRFT constructs a Safe-Role Question-Answer (SRQA) dataset from psychometric questions, limited jailbreak prompts, and a safe-role description. Role-consistent responses are synthesized, validated, and expanded into diverse scenarios, enabling models to internalize safety-oriented values and principles rather than explicit refusal patterns. Experiments across multiple Base and Instruct models show that SSRFT achieves more robust and generalizable safety alignment than standard SFT. SSRFT shows substantially greater robustness to prefilling attacks and better generalization to unseen jailbreak domains, while reducing over-refusal on benign queries and preserving the model's general capabilities. These results establish safe-role internalization as an effective alternative to refusal-centric safety alignment. Warning: This paper contains examples of harmful and toxic language.
Source:https://arxiv.org/abs/2610.07023PreviousTree Navigation Without LLM Summaries: A Matched-Cost Study of Hierarchical Retrieval for Long-Document QANextSmart Content Ingestion for Generative AI Workloads
Related posts
Shutting the Door on the XCPE Botnet (Threat Intelligence)
0LWN.net

Canva Brings Free All-in-One Affinity App to iPad
0MacRumors
GPT-Rosalind-6 System Card Addendum
0Hacker News New
Mediroute: An open-source AI platform exposing global drug price disparities
0Hacker News New
An intuition for how neural nets can understand text
0Hacker News New
An open LLM read 291 retracted papers: 7.7% of passages mention the retraction
0Hacker News New