Database
Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders
arXiv cs.IR0 views
arXiv:2610.09227v1 Announce Type: new Abstract: Mixed-precision post-training quantization needs a per-module sensitivity signal; for a text embedder the obvious one -- the retrieval quality a module costs when quantized -- needs relevance labels that deployments rarely have. We measure a label-free substitute: quantization-induced representation drift, obtained by quantizing one module, re-encoding the corpus, and recording how far the output embeddings moved from their full-precision positions. What is specific is the observable: the deployed output representation a dense retriever ranks with. Across five development embedders, configuration-level drift orders sampled mixed-precision plans against held-out retrieval quality at a macro Spearman of 0.911, the sensitivity transports across calibration corpora and retrieval domains in the usable regime, module drifts compose rank-consistently but not numerically, and relevance-derived sensitivity adds no consistent value. The method is one additive allocation under a hard packed-byte budget, with no labels and no search. On three embedders held untouched until method, baselines and hypotheses were frozen and sealed, the pre-registered directional hypothesis against the prior LieQ criterion holds (3/3 at the main budget, no collapse) and drift scores above a two-sided LieQ steelman in 2/3; but at the main budget drift is numerically lower than same-budget uniform precision on all three (-0.99, -0.85, -1.01 points), having reduced module and whole-model drift as designed. Output drift is thus a robust coarse sensitivity signal, not a universally optimal allocation objective: it avoids the catastrophic failures of the transferred signed-geometry adaptation and can remain usable at stressed budgets where uniform collapses, but fine-grained redistribution around a strong uniform operating point remains unresolved.
Source:https://arxiv.org/abs/2610.09227PreviousWhat Transfers from a VLM Teacher? Comparing Supervision Signals for Visual Document RetrievalNextFrom Chunks to Functional Evidence: Function-Aware Retrieval for EDA Documentation QA
Related posts
Goravel events and listeners for notifications
0dev.to Go
Goravel queues and jobs: retries and failed jobs
0dev.to Go
Goravel feature testing for a REST API
0dev.to Testing

How to find and fix slow PostgreSQL queries
0dev.to Database
Goravel models and migrations with Postgres
0dev.to Database
Goravel belongs to relationships and foreign keys
0dev.to Database