The latest articles on DEV Community by Senanur Çetin (@senanurcetin). https://dev.to/senanurcetin https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4164153%2Fafd11d71-41a8-47b3-95b4-edf8393a880e.jpg https://dev.to/senanurcetin en Senanur Çetin Mon, 05 Oct 2026 20:03:09 +0000 https://dev.to/senanurcetin/my-ensemble-scored-0129-on-the-leaderboard-here-is-what-explained-the-gap-5aba https://dev.to/senanurcetin/my-ensemble-scored-0129-on-the-leaderboard-here-is-what-explained-the-gap-5aba <p>My ensemble scored 0.129 on the leaderboard, below the field median. I published it anyway, then tested six explanations for the gap.</p> <p>Before submitting I wrote down a forecast: 0.143. The graded scores were 0.128 (single model) and 0.129 (ensemble).</p> <h2> Setup </h2> <p>804.5M raw market rows reduced to 292 BigQuery features, walk-forward validation with a one-month embargo, six months held out and read exactly once (+0.15171).</p> <h2> What I found, without spending another submission </h2> <ul> <li>One hypothesis confirmed: my hold-out sat at the 83rd percentile of period difficulty. De-biasing for that gives +0.14084, within 0.00004 of the walk-forward mean computed a different way.</li> <li>Four falsified, including the two I liked most. One unsettled.</li> <li>Period difficulty explains 46% of the gap. The rest is documented, not explained away.</li> </ul> <p>Five recorded forecasts, five overshoots, all in the same direction. That points to one cause, not five mistakes.</p> <h2> The rule I kept </h2> <p>An internal gain smaller than the fold-to-fold noise tells you which model to prefer, not what a leaderboard will show.</p> <h2> Links </h2> <ul> <li>Case study: <a href="https://senanur-cetin.vercel.app/projects/mscapital-market-forecasting?utm_source=devto&amp;utm_medium=social&amp;utm_campaign=w1-mscapital" rel="noopener noreferrer">MSCapital market forecasting</a> </li> <li>Notebook: <a href="https://www.kaggle.com/code/senanuretin/ms-capital-my-hold-out-was-a-lucky-stretch" rel="noopener noreferrer">My hold-out was a lucky stretch (Kaggle)</a> </li> </ul> datascience machinelearning kaggle modelevaluation