One overall number hides the hourly misses
A fixed 90% prediction interval covered 3,894 of 4,258 test hours. The overall rate is 91.4514%, only 1.4514 percentage points from the nominal target. That single number reads as close enough. An operator using intervals makes hour-level decisions, and for staffing or dispatch the 08:00 to 09:00 window matters more than a six-month average. This article answers a descriptive question: where do the 364 missed hours sit in the day, and what can be read from that distribution.
The data are the UCI Bike Sharing hourly system-wide rental totals from Washington D.C.'s Capital Bikeshare in 2011 and 2012. They are not available bikes at a station and not unmet demand. I split the rows into four time-separated roles by target timestamp: training before 2012, Q1 for epoch selection, Q2 for frozen calibration scores, Q3 and Q4 for testing. The point model is a float64 network from 35 inputs through 32 tanh hidden units to 1 output, with 1,185 parameters, trained for 40 epochs and selected at epoch 39 by Q1 error. The three nominal levels, 95%, 90%, and 80%, reuse the same point model and the same Q2 calibration scores.
Calibration uses 2,134 absolute residuals. The 90% threshold follows the order statistic k = ceil((n+1)(d-a)/d). With n=2134 and a/d=1/10, the rank is 1,922 and the threshold is 96.621898 rentals per hour. Widths below are in hourly rental counts, not probabilities.
The three fixed levels:
| Nominal level | Threshold | Covered / 4,258 | Observed coverage | Mean width |
|---|---|---|---|---|
| 95% | 127.099692 | 4,037 | 94.8098% | 223.8812 |
| 90%, primary | 96.621898 | 3,894 | 91.4514% | 172.9837 |
| 80% | 62.429339 | 3,511 | 82.4566% | 114.2475 |
The 364 misses point in two directions: 76 truths fell below the lower bound and 288 truths exceeded the upper bound. The asymmetry is a lead worth following, since it suggests point forecasts often sit low around peak hours, but it is not an attribution on its own.
What the hour slices show
A few representative clock hours:
| Hour | Covered / hour rows | Coverage | Misses above upper bound |
|---|---|---|---|
| 00 | 176 / 178 | 98.8764% | 2 |
| 03 | 175 / 175 | 100% | 0 |
| 07 | 149 / 177 | 84.1808% | 23 |
| 08 | 94 / 177 | 53.1073% | 81 |
| 17 | 125 / 178 | 70.2247% | 43 |
| 18 | 142 / 178 | 79.7753% | 24 |
Overnight hours miss rarely, and the misses gather near the morning and evening peaks. I also made a post hoc coarse split of test targets from 07:00 through 19:59 in one group and all other hours in another. This split was not preregistered. The daytime group has 2,308 rows and 1,968 covers, or 85.2686%; the other group has 1,950 rows and 1,926 covers, or 98.7692%. The two groups add back to 4,258 rows and 3,894 covers.
Months vary too. September covered 631 / 720 = 87.6389%, October 589 / 684 = 86.1111%, and December 674 / 696 = 96.8391%. I do not attribute those differences to season alone, because the data do not support that single-cause reading.
Width and coverage are separate observations
The interval is [max(0, point-q), point+q]. When the point forecast is below q, the lower bound clips to zero and the width becomes point+q, smaller than the full 2q. For 90%, 2q is 193.2438 rentals while the test mean width is 172.9837, and the gap comes from that lower-bound clip. The saved hour-level mean widths show where the clip bites. Hour 03 has a mean width of 108.1440, hour 08 has 190.1621, and hour 17 has 192.9269. Low-count overnight hours carry narrower absolute intervals, and peak hours sit closer to 2q.
Coverage direction and width direction are different facts. Overnight hours have narrow intervals and almost all covers. Peak hours have wider intervals and more misses, with 81 truths above the upper bound at hour 08. That combination is consistent with several mechanisms: point predictions running low on some high-count hours, residual scale changing by hour, or a distribution shift between the Q2 calibration period and the Q3/Q4 test period. The coverage table cannot separate those. I do not claim any one of them as the identified cause, and I do not claim that switching to another score would necessarily help. The next diagnostic step is to read the saved point-predictions.csv and look at the sign of the signed difference truth - point computed from its truth and prediction columns, plus the magnitude in its absolute_residual column, by hour bucket and by point level.
What these rows can support
Every test target needs its previous 24 nominal hours, and evaluation rolls one hour forward, using only actual counts that have already occurred. Adjacent nominal targets share 23 input hours. Members of one hour bucket, such as the 177 targets at hour 08, sit about 24 hours apart. The overlap of adjacent windows does not by itself prove that all rows within one hour bucket are dependent. The precise claim I can hold is that independence and exchangeability are not shown to hold on this sequence, and not refuted either. The hour and month tables are therefore descriptive summaries. I run no significance test, build no error bars, and promise no probability for a single input or a future month.
The test role has 4,376 original rows, and 118 targets lacked their preceding hours, so they stay out of the coverage denominator and remain in the inventory. My conclusion covers only those 4,258 eligible hours. The source dataset and the earlier benchmark were already inspected in prior work, so this check is not blind. The usual marginal coverage theorem gives its guarantee when calibration and test scores are exchangeable and the fitted procedure is independent of the calibration labels. That premise is not shown to hold on this sequence, so the experiment does not claim the theorem's guarantee and does not assert that the sequence has been proved non-exchangeable.
Recheck the saved summaries
The script below uses the Python3 standard library and should run from the root of the complete fixed package. It reads only the saved scores and grouped summaries in calibration-001/calibration.json and evaluation-001/results.json. It loads no weights, rebuilds no 24-hour windows, fits no model, and does not re-derive the full interval CSV row by row. I do not treat this printout as a new independent performance test.
python3 - <<'PY' from pathlib import Path import json import math root = Path(".") cal = json.loads((root / "calibration-001/calibration.json").read_text(encoding="utf-8")) scores = cal["scores"] a, d = 1, 10 k = math.ceil((len(scores) + 1) * (d - a) / d) q = sorted(scores)[k - 1] res = json.loads((root / "evaluation-001/results.json").read_text(encoding="utf-8")) primary = next(r for r in res["intervals"] if r["alpha_denominator"] == 10) print("n_scores", len(scores), "rank", k, "q", q) print("overall", primary["overall"]["n"], primary["overall"]["covered"], primary["overall"]["miss_above"], primary["overall"]["miss_below"]) for h in sorted(primary["hour"], key=lambda s: int(s)): row = primary["hour"][h] print(h, row["n"], row["covered"], round(row["coverage"], 6)) PY Expected output gives n_scores=2134, rank=1922, and overall values 4,258, 3,894, 288, 76. The 95% and 80% groupings live in the same evaluation-001/results.json. Calibration scores are in calibration-001/calibration.json, and the order-statistic rule is in quantiles.py.
The fixed-commit repository is WideChaos/forecast-interval-lab, and the standalone app entry is https://widechaos-forecast-interval-lab.static.hf.space/index.html. The app runs the fixed model and three frozen thresholds in browser memory, with no inference API and no autosave. The explicit JSON download contains the full input, so users should inspect it before sharing. The single actual training run finished on 2026-11-04; this publication reuses that model, calibration, and complete results, and performs no new training.
This 1200×1000 original figure shows the loss trace, monthly coverage and width, and the four-role denominators. It does not show hourly coverage.
The data are Hadi Fanaee-T's Bike Sharing dataset, UCI page https://archive.ics.uci.edu/dataset/275/bike+sharing+dataset, DOI 10.24432/C5W894. The method background is Angelopoulos and Bates, arXiv:2107.07511v6; I use only the absolute-residual and order-statistic framework and do not copy their code, figures, or empirical studies. The data and derived weights, calibration scores, results, and figures keep CC BY 4.0 attribution. My code and documentation are Apache-2.0, by Renchao Wu.



