AI & ML
Shielded Trajectory Sampling for Personalized Relational Safety in Conversational AI
arXiv cs.HC0 views
arXiv:2609.30780v2 Announce Type: replace Abstract: Anthropomorphic conversational systems can provide companionship and emotional support, but repeated, seemingly harmless interactions can encourage excessive dependence and reduced human contact. We formulate safety in user-AI relationships as a sequential decision problem, using conversational cues to track the relationship and selecting responses according to their anticipated effects over time. A conditional Generative Flow Network learns from sampled response plans to select replies that preserve the largest estimated mass of safe and useful continuations. Model-based checks screen plans during training and responses during interaction, requiring at least one sequence of future response strategies that meets a prescribed safety bound under every candidate user model, independently of the learned network. Strategies are selected directly from learned first-step probabilities, avoiding online plan sampling. We evaluate the approach in simulation against reactive safeguards, filters with and without memory, and alternative planners, using exact computation to validate the learned approximation. We also evaluate relational cue extraction from real user messages in WildChat. To our knowledge, this is the first approach to select conversational replies by anticipating their effects on the user-AI relationship while imposing an explicit model-based safety bound on future response sequences. Results show close agreement with exact planning, lower decision costs at longer horizons, and improved safety-utility trade-offs when protective responses are less effective. The approach balances immediate support with the longer-term effects of repeated interaction. The source code is available at https://github.com/benedettapicano/ANTHROPOMORPHIC_SAFETY_TRAJ.
Source:https://arxiv.org/abs/2609.30780PreviousHuman-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMsNextAI Simulations Grounded in Rich Self-Reports Increase Accuracy and Reduce Distortion
Related posts

Debugging Claude Fable 5.2 Code Art: Truncated SVG, Broken Canvas, Working Render
0dev.to
Hydra · Coeziune OM–IA — the complete DOI registry (2026-11-18): 76 published works
0dev.to OpenSource
Your Rust Code Sets an Environment Variable and the 2024 Edition Calls That Unsafe
0dev.to Rust
Your Go Function Returned a Nil Error and the Caller Saw an Error Anyway
0dev.to Go
Kengele: an alarm for when an activist goes missing, that nobody can switch off
0dev.to Go
How OpenAI scales PostgreSQL to 800 million users
0dev.to Database