AI & ML
Orthogonal Norm Growth in Two-Tower Retrieval: A Popularity Mechanism for Unified Discovery
arXiv cs.IR0 views
arXiv:2611.09294v1 Announce Type: new Abstract: Unified search and recommendation systems may update the same item vectors from both interaction streams. We study how repeated updates can increase an item vector's norm and affect retrieval under an unnormalized dot product. A cosine-only loss makes the output gradient perpendicular to the current item vector - tangent to its constant-norm sphere. Training changes encoder parameters, however, so the realized item-vector movement need not remain tangent. We derive sufficient conditions under which it does; every nonzero update then increases squared norm by its squared length. When the tasks share an item encoder, sequential updates from both can accumulate in the same norm, which raw-inner-product serving can expose for either task. This conditional result is not a theorem about joint SGD. Controlled ablations test these conditions. On MovieLens 32M and Amazon Books, accumulated tangent movement accompanies frequency-norm association and additional head-item exposure under raw-dot rather than normalized-item ranking. In a nonlinear tower combining a per-item ID vector with a shared content encoding, active item movement remains near-tangent. We separate its norm change into the ID-update contribution and other output changes, then use a matched training intervention that removes immediate norm growth caused by the ID-row update. The intervention reduces norm-related exposure but also recommendation utility, so it is diagnostic rather than a proposed mitigation. Our logs are recommendation-only: the formulation supplies a shared-representation audit, while prevalence in joint traffic remains open.
Source:https://arxiv.org/abs/2611.09294PreviousStructure and Lineage Evidence for Vietnamese Statutory Article Retrieval on DRiLLNextSPARSEUP: Towards Better Recipes for Training Modern Learned Sparse Retrievers
Related posts

Some United Airlines flights now let you sign into Spotify on your flight — and its' totally free
0TechRadar

Straight Talk just dropped an unlimited data plan to $20/mo ahead of Black Friday — a brand new record
0TechRadar

Who Gives A Crap: Data breach at ethical toilet paper company exposes customer information across UK, US and Australia
0TechRadar

Since tokenmaxxing is the wrong metric for AI ROI, what is the right one?
0TechRadar

Narwal V50 Cube review: a self-emptying stick vacuum with brilliant smart features
0TechRadar

Security researcher had to jailbreak their car just to update the clock
0Tom's Hardware