AI & ML
AffectGPT-RL: Revealing Roles of Reinforcement Learning in Open-Vocabulary Emotion Recognition
arXiv cs.HC0 views
arXiv:2605.06126v2 Announce Type: replace Abstract: Open-Vocabulary Multimodal Emotion Recognition (OV-MER) aims to predict emotions without being constrained by predefined label spaces, thereby enabling fine-grained emotion understanding. Unlike traditional discriminative methods, OV-MER leverages generative models to capture the full spectrum of emotions and employs emotion wheels (EWs) for metric calculation. Previous approaches primarily rely on token-level loss during training. However, this objective is misaligned with the metrics used in OV-MER, and these metrics cannot be directly optimized via gradient backpropagation. To address this limitation, we turn our attention to reinforcement learning, as this strategy can optimize non-differentiable objectives. We term this framework AffectGPT-RL. Furthermore, we conduct extensive experiments to elucidate the role of reinforcement learning in this task, revealing the necessity of the reasoning process, the impact of different rewards, and the generalizability to other emotion tasks such as sentiment analysis and basic emotion recognition. Experimental results demonstrate that AffectGPT-RL yields significant performance improvements on OV-MER. Beyond this task, we also achieve remarkable performance gains on basic emotion recognition, attaining state-of-the-art results on MER-UniBench. To the best of our knowledge, this is the pioneering work exploring the role of reinforcement learning in OV-MER, providing valuable guidance for subsequent researchers. Our code is provided at: https://github.com/zeroQiaoba/AffectGPT/tree/master/AffectGPT-RL.
Source:https://arxiv.org/abs/2605.06126PreviousBeyond Checklist Compliance: A Tale of the Accessibility-Usability Dilemma across Government and Non-Government Web Systems in BangladeshNextFriction in AI-Assisted Clinical Decision-Making: A Case Study on The Role of Questions and 'What-if' Scenarios
Related posts
Show HN: Subvocal – Talk to your computer without making noise
0Hacker News Best

The new and four-star-rated Bose QuietComfort Headphones are already down to a record-low price ahead of Black Friday
0TechRadar

AI may be the future, but customer experience must meet the price-point
0TechRadar

The cassette player renaissance continues with this Bluetooth-toting high-tech Kickstarter model
0TechRadar

FBI confirms it thinks North Korean hackers hit Binance in record cryptocurrency heist
0TechRadar

These 3 hand-cranked radios are affordable and offer extra features
0TechRadar