You split your data 80/20, get 91% accuracy, and ship it. But was that 91% luck or skill? A single split can fool you. Cross-validation gives you a trustworthy number. Here's k-fold, visualized.
🔁 Watch the folds rotate: https://dev48v.infy.uk/ml/day18-cross-validation.html
The problem with one split
One train/test split is high-variance: a lucky test set flatters your model, an unlucky one trashes it. You're judging on a single roll of the dice.
k-fold cross-validation
Split the data into k equal folds. Then, k times: train on k−1 folds, validate on the held-out one. You get k scores — report the mean ± std. Every data point gets used for both training and validation (in different rounds), so the estimate is stable.
The demo rotates each fold through validation, fits a real model per fold, fills in the per-fold scores, and shows the average — next to a single split you can reshuffle to watch it swing.
The disciplines that matter
- Use CV to tune hyperparameters, but keep a final test set you never touch.
- Fit scalers/encoders inside each fold (or you leak).
- Stratify for imbalanced classes; don't shuffle time-series.
Cost: k× the training. Worth it for an honest score.
🔨 Built from scratch (split into folds → train/score each → mean±std → grid-search) on the page: https://dev48v.infy.uk/ml/day18-cross-validation.html
Part of MachineLearningFromZero. 🌐 https://dev48v.infy.uk
Top comments (0)