AI & ML
A low-rank decoder bottleneck bounds reliability in foundation-model perturbation prediction
arXiv stat.ML0 views
arXiv:2611.05849v1 Announce Type: cross Abstract: Foundation-model predictors of perturbation response are evaluated primarily by aggregate accuracy. Using their predictions to prioritize experiments requires assessing each prediction's reliability. We audit the state-transition (ST) decoder used in STATE across three encoders (SE-600M, scGPT, and Tahoe-x1) and four cell lines under CRISPRi. The decoder's first attention layer has a low-rank output. Projecting this output onto 4-32 fixed directions estimated from training activations preserves mean per-perturbation Pearson delta correlation (PDC). This low-rank attention bottleneck bounds the decoder's reliability by distance to training responses: PDC declines as the cosine distance from a perturbation's observed response to its nearest training response increases. We propose predicted-NN-distance, the cosine distance from a predicted response to its nearest training response, as a per-prediction confidence score. The score has a mean absolute Spearman correlation of 0.81 with PDC, close to the 0.88 obtained with the observed response. Thresholds calibrated on this score transfer across encoders and cell lines, and the score remains predictive of PDC for drug perturbations.
Source:https://arxiv.org/abs/2611.05849PreviousTowards the Query Complexity of Sampling under Strong Log-Concavity and Log-Sobolev InequalitiesNextPrior-data fitted networks for variable selection
Related posts
Scalable Source Selection for Machine Learning Tasks
0arXiv cs.DB
AvalancheBench: Evaluating Enterprise Agent-Driven Semantic Query Processing Through Latent World Recovery
0arXiv cs.DB
ConStruM: A Structure-Guided LLM Framework for Context-Aware Schema Matching
0arXiv cs.DB
From Certified Pairs to a Certified Hub: Reasoner-Checked Crosswalks for the Information Exchange Standard
0arXiv cs.DB
A Comparative Study of Importance Attribution to Columns in Deep Learning over Relational Databases
0arXiv cs.DB
Community Search under Member-Specific Tolerances
0arXiv cs.DB