Database
Stop Removing Stopwords: How an Inherited Preprocessing Default Distorts Legal Text-as-Data
arXiv cs.IR0 views
arXiv:2609.19153v3 Announce Type: replace-cross Abstract: Empirical legal scholarship increasingly treats judicial text as data, and much of it still runs on sparse, interpretable pipelines -- TF-IDF features and linear classifiers -- because the textual feature is often the object of study rather than a means to prediction. Yet these pipelines inherit preprocessing defaults from mid-century information retrieval that were never validated against classification accuracy. The most entrenched of these is stopword removal. This study introduces an exhaustive single word ablation that measures the effect of a preprocessing step directly against the downstream objective, and applies it to stopword removal as the hardest case to dislodge. Matching Supreme Court Database labels to Caselaw Access Project opinion texts, the study examines two binary tasks that bracket F1 headroom, ideological direction (no-removal baseline F1 ~0.68) and constitutional versus non-constitutional law type (~0.92), across 7,668 and 7,001 opinions. For each task, the ablation removes each of about 18,500 candidate words in turn, and a task-specific stoplist is built from the measurements. Common generic stoplists fall below the no-removal baseline in all twelve held-out tests. The task-specific stoplists move held-out F1 by +0.0023 (95% CI [-0.0124, +0.0170]) on ideology and by +0.0001 ([-0.0082, +0.0085]) on law type. Neither task shows a detectable benefit from removal, and supplemental analysis finds that word-level statistics predict removal effects poorly, because true removal effects differ by less than the measurement can register, leaving little to predict. The method generalizes to any inherited preprocessing default; the result is a caution specific to interpretable legal text-as-data, where a step that reshapes which features a model sees can distort the doctrinal and ideological signal the research is meant to recover. The burden of proof sits with removal.
Source:https://arxiv.org/abs/2609.19153PreviousFrom transcription to semantic corpus analysis: unsupervised learning of sentence representations for ancient languagesNextGeometry, Local Minimax Risk, and the Limits of Sample Average Approximation in Linear Stochastic Optimization
Related posts
PostgreSQL 19
0Hacker News
Why PostgreSQL 19 shipped late
0Databricks Blog
A language model that writes 1,000 tokens a second. What changes when generation is nearly instant?
0dev.to ELI5

Room Migration v19: Adding a Unified Persons Directory Without Breaking Existing Data
0dev.to Database
A Model of Diverse Sampling from Language Models
0arXiv stat.ML
AWS Cloud Quest: Cloud Practitioner Badge - How to Earn It
0dev.to AWS