We put twelve model families through exactly the same test for ranking stocks one month ahead: simple rules, ridge regression, LightGBM, a PyTorch MLP, k-nearest neighbours, an LSTM, a GRU, a regime-switching model and others.
Same features, same walk-forward schedule, same costs, same measure. None showed a statistically significant edge.
That raised a practical question: if nothing clearly wins, which one do you keep?
Our rule
If a more complex model doesn't clearly beat the simpler one, the simpler one wins.
"Clearly" means: better on the development data by more than the noise, and still better after correcting for the number of things we tried.
Why
Overfitting is learning patterns that were chance in your sample. More knobs give a model more ways to find them:
- A deep network with tens of thousands of weights can fit a lot of noise in a few years of monthly outcomes.
- A ridge regression with a handful of coefficients can't fit much.
When both score about the same on unseen data, the complex one has likely spent its extra capacity on noise, and will be more fragile when the market changes.
There are practical reasons too: a simple model is faster to retrain, easier to debug, and easier to explain to a reader who wants to check the work.
How we check for it
- Results year by year: a model that only shines in one year has probably fit that year.
- Results without the three best names, and at double the trading cost.
- A paired test against the simpler model (Newey-West t-statistic), not just a higher average.
Takeaways
- Fewer knobs, fixed rules, and a test set you never peek at.
- Report the simple baseline next to every fancy model.
- When they tie, keep the simple one.
The full leaderboard with all twelve families, their scores and their t-statistics is public.
From Quantooo Lab, the makers of quantooo.com, a free, open and educational quant lab: every formula, every test and every failure is public. See it here: https://quantooo.com/models
Educational, not investment advice.
Top comments (0)