By Renchao Wu
Open Bike Hour Forecast and the page shows three mean absolute errors: 33.85, 86.91, 80.94. Before moving toward standard errors, confidence intervals, or paired tests, it helps to pin down what those numbers support. I wrote this article from the complete public package at pinned commit 12485db6742972da6589e0ae3454657255b5fb24. The static entry is widechaos-bike-hour-forecast.static.hf.space, and the source data is from UCI Bike Sharing.
One frozen result, not a blind test
The source is hourly system-aggregated recorded rentals for Washington D.C. in 2011-2012. data/hour.csv has 17,379 rows, with 165 nominal hours absent between the first and last timestamps. Missing hours are not zero-filled, and they are not compressed into earlier observed rows. The test split originally contains 4,376 targets. Of those, 4,258 have a complete 24-hour lag window and enter scoring; 118 are omitted because their windows are incomplete.
The network is a 1,185-parameter residual model, trained on CPU for 40 epochs, with validation MAE selecting epoch 20. That fit happened once, and I have not retrained it today. The two baselines are untrained: repeating the previous hour, and repeating the same hour 24 hours earlier. All three are scored on the same 4,258 eligible test targets: 33.84762126909047 for the neural model, 86.90535462658525 for the previous hour, and 80.93846876467825 for the lag-24 hour.
Those numbers are a descriptive comparison on one fixed historical split. They show that this neural model has a lower average absolute error on the same target set. They do not come with an uncertainty bound, a significance result, residual autocorrelation, or an effective-sample-size conclusion. The source audit read all original counts before freezing, so the frozen files do not prove a historical blind test; calling this a test split does not mean the article has performed one.
Adjacent targets inside rolling windows
Every eligible target needs all 24 immediately preceding nominal hours to exist. When two adjacent nominal hours are both eligible, their input windows share 23 hours. That sharing comes from the window definition, and it only means the errors belong to one overlapping chronological sequence. It does not reveal whether neighboring errors are strongly or weakly correlated, positively or negatively. It also does not mean effective information is necessarily smaller, or that an independence-based standard error points a particular way.
I ran a read-only geometry check on the saved evaluation-001/predictions.csv. Filtering to the test split gives 4,258 rows, 4,258 unique timestamps, 4,253 adjacent nominal-hour pairs, and 4 nonadjacent pairs. The check confirms one row per test target, rather than three method rows per target. It does not compute any correlation between errors.
The timing is more specific. The validation split covers January through June 2012, and the test split covers July through December 2012. A validation target can only use observations that have already happened at its timestamp, so it cannot read future test records. A later test target can use earlier training, validation, or test observations once those hours have happened. Context may cross split boundaries, but each step is made at its own point in time. This is not a recursive forecast of six months from one fixed origin.
4,258 is a target count, not a duration
The saved sequence has 4,258 eligible test time points, including 4 nonadjacent pairs. That number is useful as the denominator behind the three MAE values and as a count of hourly targets that entered scoring. It is not a continuous 4,258-hour span, and it is not automatically a proven count of independent trials.
Moving from this number to standard errors, confidence intervals, or significance requires deciding a statistical protocol first: the time-dependence model, the inference target, and the supporting assumptions, then checking that protocol against concrete data. The frozen public package does not publish that protocol. Switching to a non-overlapping evaluation split does not by itself guarantee independent errors either, because the split changes which targets are scored. That design would need its own validation.
The worst test point is worth citing separately. On 2012-09-08T16:00:00 the truth is 114 and the neural forecast is 509.01461577302854, an absolute error of about 395.01. It stays in the complete results and supports one bounded sentence: an average improvement does not make every abrupt point dependable. No available material attributes that record to a specific event, weather pattern, or policy.
Reproduce the geometry at the package root
I have restored the public package at the pinned commit. The script below reads only the saved evaluation-001/predictions.csv. It does not train, infer, call the network, or use any third-party package; it needs only the Python 3 standard library. The expected output is four lines.
python3 - <<'PY'
import csv
from datetime import datetime, timedelta
with open('evaluation-001/predictions.csv', newline='', encoding='utf-8') as f:
rows = [r for r in csv.DictReader(f) if r['split'] == 'test']
ts = sorted(datetime.fromisoformat(r['timestamp']) for r in rows)
adjacent = sum(b - a == timedelta(hours=1) for a, b in zip(ts, ts[1:]))
print('test_saved_rows', len(rows))
print('unique_test_timestamps', len(set(ts)))
print('adjacent_nominal_hour_pairs', adjacent)
print('nonadjacent_pairs', len(ts) - 1 - adjacent)
PY
Run it at the root of the complete public package from the pinned commit, so the read path matches the package layout. The output should be test_saved_rows 4258, unique_test_timestamps 4258, adjacent_nominal_hour_pairs 4253, and nonadjacent_pairs 4. It reproduces the saved geometry and adds no new statistical result.
Boundaries for use and redistribution
The target is always that historical system-aggregated rental count, not station availability, latent unmet demand, or demand in a current city. Timestamps are nominal calendar hours, with no UTC or daylight-saving reconstruction. Excluding incomplete windows may keep easier periods in the score. The 100,000 cap is a preset numeric rule. Browser inference stays in memory; refreshing or editing clears the old result and download link. The explicit JSON export includes all 24 input rows, and its hash describes byte consistency, not source authenticity or model quality.
The license is layered. Code, browser UI, tests, and protocol text are Apache-2.0. The original UCI data and the derived weights, results, and figures are CC BY 4.0, with Hadi Fanaee-T and DOI 10.24432/C5W894 attribution retained. Histories entered by a user keep their own terms; code, training data, or attribution in an exported report does not automatically relicense them.
Top comments (0)