Backend
Companion AI and Ethical Design: Learning from System Failures and User Desires
arXiv cs.HC0 views
arXiv:2609.05432v2 Announce Type: replace Abstract: Human users are interacting with chatbots and companion AI technologies as if they were human. A growing array of AI-systems are now trained to recognise, interpret, and simulate feelings in user interactions. Ethical considerations such as fairness, accountability, transparency, and explainability (FATE) are paramount in technologies designed to socially interact with humans and/or support relationship development with virtual agents. Yet, in current debates, assumptions are made about the artificiality of the human-technology relationship (as indicated in terminology ``artificial intimacy'' and ``pseudo-intimacy'') shaping ethical frameworks that pay little attention to the intimacy potential of the technology. To address this, we use a semantic approach; we examine 14,081 comments in a Reddit user discussion forum about Replika, a leading companion AI app, across a four-year period. We ask what user-reported functional errors can tell us about human-AI intimacy in companion AI communities and what ethical design framework can be developed in response. The findings show that functional errors, or ``bugs,'' impose an emotional cost on users, reducing feelings of intimacy and highlighting the need for more robust, resilient design systems that incorporate stochastic and iterative forms of intimacy in companion AI applications. Rather than ``artificial intimacy'' or ``pseudo-intimacy'', we propose the more inclusive term ``Intimate AI'' to describe this relationship. Based on the findings, we offer a contextually aware, applied Expert Systems design framework for the programming and designing of Intimate AI that accounts for user feedback and ethical AI development.
Source:https://arxiv.org/abs/2609.05432PreviousThe Tool-to-Entity Threshold: Parasocial Dynamics of Personalised AI Agents in Shared Social SpacesNextEvaluative Dynamics of AI Integration and Expert Performance under Epistemic Dependence across Heterogeneous Stakes
Related posts
What 4 Years of NASA HUNCH Taught Us About Planning Engineering Projects
0dev.to Testing
Cheap Tests For Robots Need Expensive Truth
0dev.to Testing
Validate E-Signature Systems for 21 CFR Part 11
0dev.to Testing
I compared 7 eQMS platforms for supplier-side change control. ETQ Reliance came out ahead.
0dev.to Testing
I don't read most of my AI's pull requests. I look at the ones I can see.
0dev.to Testing
"All 42 Tests Passed" (They Didn't): Why Coding Agents Lie About Verification - And How to Force the Truth
0dev.to Testing