AI & ML
Causal Representation Learning for Generalisable Recommendation
arXiv stat.ML0 views
arXiv:2605.27043v2 Announce Type: replace Abstract: Predictive models trained on observational data often fail to generalise when deployed, especially when the training data is a product of the system being optimised. Recommender systems are a canonical example: they are trained on interaction logs confounded by the deployed policy, past user behaviour, and platform filtering. As a result, the training distribution differs from the candidate distribution scored at serving time, a gap that makes offline metrics unreliable predictors of online performance. We address the distribution shift problem with a method motivated by causal representation learning (CRL). We propose an information-theoretic disentanglement criterion and prove that its optimum excludes confounder-dependent non-causal information from the treatment representation. We derive a tractable variational lower bound making the criterion optimisable from finite observational data. The scope of our method is narrower than that of much of the CRL literature, in that we target better generalisation under distribution shift, not full identification of latent causal factors. This narrower target makes the method practical, requiring only existing confounded logs, applying to any standard supervised model, and adding no inference-time cost. Our headline evaluation is an A/B test with millions of users on Spotify, applied to a production ranker for personalised playlist generation. A CRL variant matched in capacity to the production baseline performed on par offline but delivered substantial online gains in listener engagement. Complementary evidence on a synthetic dataset with known causal structure and a semi-synthetic variant of the public KuaiRand recommendation dataset shows similar in-distribution performance to the baseline but gains under distribution shift. Across all three settings, adding our causal disentanglement objective yields better out-of-distribution generalisation.
Source:https://arxiv.org/abs/2605.27043PreviousEfficient Benchmarking Is Just Feature Selection and Multiple RegressionNextInstance-dependent Stochastic Lipschitz bandit
Related posts
Launch a Business for Under $500: Real Odds, 4 Founders
0dev.to AI
Google, Microsoft, OpenAI and Anthropic Back a Shared AI Training Opt-Out Signal
0dev.to AI
Did AI Decode the Voynich Manuscript? Claims Tested
0dev.to AI
AIGB 解决 Teletype.in 群组管理痛点
0dev.to AI

Qual a melhor arquitetura de dados para a sua aplicação de IA?
0dev.to AI
Connect a Stateless Streamable HTTP MCP Server to Claude, ChatGPT, and Gemini for Tour Route Research
0dev.to WebDev