AI & ML
Orthogonal Norm Growth in Two-Tower Retrieval: A Popularity Mechanism for Unified Discovery
arXiv cs.IR0 views
arXiv:2611.09294v1 Announce Type: new Abstract: Unified search and recommendation systems may update the same item vectors from both interaction streams. We study how repeated updates can increase an item vector's norm and affect retrieval under an unnormalized dot product. A cosine-only loss makes the output gradient perpendicular to the current item vector - tangent to its constant-norm sphere. Training changes encoder parameters, however, so the realized item-vector movement need not remain tangent. We derive sufficient conditions under which it does; every nonzero update then increases squared norm by its squared length. When the tasks share an item encoder, sequential updates from both can accumulate in the same norm, which raw-inner-product serving can expose for either task. This conditional result is not a theorem about joint SGD. Controlled ablations test these conditions. On MovieLens 32M and Amazon Books, accumulated tangent movement accompanies frequency-norm association and additional head-item exposure under raw-dot rather than normalized-item ranking. In a nonlinear tower combining a per-item ID vector with a shared content encoding, active item movement remains near-tangent. We separate its norm change into the ID-update contribution and other output changes, then use a matched training intervention that removes immediate norm growth caused by the ID-row update. The intervention reduces norm-related exposure but also recommendation utility, so it is diagnostic rather than a proposed mitigation. Our logs are recommendation-only: the formulation supplies a shared-representation audit, while prevalence in joint traffic remains open.
Source:https://arxiv.org/abs/2611.09294PreviousStructure and Lineage Evidence for Vietnamese Statutory Article Retrieval on DRiLLNextSPARSEUP: Towards Better Recipes for Training Modern Learned Sparse Retrievers
Related posts

Nvidia drops ~5% pre-market after its board opens a probe into Jensen Huang; Nvidia previously dismissed the interactions as "nothing unusual or unexpected" (Demetri Sevastopulo/Financial Times)
0Techmeme

Nvidia Senior Director Madison Huang, daughter of Jensen, cancels a high-profile Seoul appearance citing a "family matter", throwing the event into disarray (Yoolim Lee/Bloomberg)
0Techmeme

Jensen Huang maintains deep control over Nvidia, managing 60+ direct reports and owning a 3.56% stake, presenting a challenge for Nvidia's 10-person board (New York Times)
0Techmeme

Amsterdam police arrested between 150 and 200 protesters for trespassing and violence at a Microsoft data center construction site owned by Pure Data Centers (Jowi Morales/Tom's Hardware)
0Techmeme

The US OCC grants Coinbase full approval for a trust bank charter; sources say it could expand Coinbase's stablecoin business and let it issue stablecoins (Olga Kharif/Bloomberg)
0Techmeme

US voters in Harwood, North Dakota rejected a ballot measure that opposed Applied Digital's $3B data center, set to open in 2027, bucking a nationwide backlash (Miriam Waldvogel/Washington Post)
0Techmeme