Backend
When Does Combining Software-Effort Estimates Help? Evidence from Four Public Project Datasets
arXiv cs.SE0 views
arXiv:2611.06948v1 Announce Type: new Abstract: Combining estimates can reduce individual prediction errors, but an ensemble may also dilute an unusually accurate estimator or repeat errors shared by its members. We study these competing effects on four archival software-effort datasets (Kitchenham: 145 projects, Desharnais: 77, China: 499, Maxwell: 63) with 100 repeated 70/30 training and test partitions per dataset. Three conventional models (parametric regression, analogy, and random forest) are combined with each other and, in Kitchenham, with a recorded human estimate. Out-of-fold calibration errors drive five combination rules: equal weighting, inverse-MSE weighting, a nonnegative simplex combination, signed Bates-Granger weighting, and Bates-Granger with covariance shrinkage. All preprocessing is learned within training folds. Model-error correlations range from 0.54 to 0.97. In Kitchenham, the human estimate dominates: equal weighting raises mean test log-MAE from 0.239 to 0.455, a loss that remains significant under a corrected resampled t-test with Holm adjustment, whereas the simplex rule limits the increase to 0.250. In Desharnais, China, and Maxwell, every rule lowers mean log-MAE (by 1.3 to 8.6%) and BG with shrinkage beats the calibration-selected individual on 75, 83, and 83 of 100 splits, but none of these gains is statistically distinguishable from zero once the overlap between repeated splits is accounted for. On a completion-year split of Desharnais, equal weighting becomes slightly worse and inverse-MSE is near the individual baseline, while simplex and shrinkage retain mean gains of about 5%. Combining correlated effort estimates is therefore a conditional decision: the harm of naive pooling next to a dominant source is clear, while the benefit elsewhere is small and uncertain.
Source:https://arxiv.org/abs/2611.06948PreviousHow Python Audio Libraries Change Across Releases: Module Structure and DocumentationNextKG-AST for Cheaper CI/CD: Budgeted Test Selection and AI Bug Review
Related posts
Metrics for Your AI Workers: Build a Personal Agent Benchmark
0dev.to
Characterization Tests First: Freeze a Messy Function, Then Make One Safe Edit
0dev.to
48-Hour Field Notes: I Moved a Nightly Job Onto a Free Server and Timed What the Cold Boot Ate
0dev.to
We benchmarked SQLite, ObjectBox, and ToStore for On-Device RAG in Flutter: 5-dimension results & architecture comparison
0dev.to
Cost per Green Run: Seven Free-Tier Myths and a Ledger You Can Run
0dev.to
Node.js Daily Email Report Webhook Processing: Durable Worker Draining
0dev.to