Database
A Model of Diverse Sampling from Language Models
arXiv stat.ML0 views
arXiv:2611.25003v1 Announce Type: cross Abstract: Language models often produce high-quality individual samples but poor sets, with repeated samples clustering around the same semantic modes. We formalize diverse generation as sampling a size-$K$ subset of complete strings from a determinantal point process (DPP), where the likelihood encodes item quality, and the embedding geometry encodes repulsion between similar outputs. Exact inference over all strings is intractable, so we propose PoolDPP: draw a finite candidate pool from a tractable proposal, importance-weight the candidates toward the target quality, and run a $K$-DPP sampler on the induced pool kernel; a greedy MAP variant provides deterministic selection. We prove a finite-sample bound with an explicit constant: the KL divergence between the sampler's output and the target $K$-DPP decays as O(1/N) in the pool size. Empirically, PoolDPP extends the quality--diversity frontier on open-ended generation and the precision--recall frontier on ambiguous text-to-SQL (Ambrosia), and on mathematical reasoning (GSM8K), it raises diversity at about the same pass@$K$.
Source:https://arxiv.org/abs/2611.25003PreviousStatistical, Machine Learning and Deep Learning Methods for Missing Covariates: An Empirical Evaluation with Applications to Alzheimer's Disease ResearchNextNear-Quadratic Sample Compression for All VC Classes
Related posts
PostgreSQL 19
0Hacker News
Why PostgreSQL 19 shipped late
0Databricks Blog
A language model that writes 1,000 tokens a second. What changes when generation is nearly instant?
0dev.to ELI5

Room Migration v19: Adding a Unified Persons Directory Without Breaking Existing Data
0dev.to Database
Stop Removing Stopwords: How an Inherited Preprocessing Default Distorts Legal Text-as-Data
0arXiv cs.IR
AWS Cloud Quest: Cloud Practitioner Badge - How to Earn It
0dev.to AWS