This was my final project for IBM's Deep Learning and Reinforcement Learning course. Fashion-MNIST is a familiar benchmark, so I framed it as a business task instead of a leaderboard entry: an online shop wants product photos tagged into ten categories automatically, and a person reviews whatever the model is unsure about.
That framing changes the question. "How accurate is the model?" becomes "what share of images can it tag on its own, and how accurate is it on those?"
Four architectures, one budget
All four models used the same split (54,000 train, 6,000 validation, 10,000 test), the same optimiser, early stopping on validation loss and a 40-epoch cap.
| Model | Parameters | Test accuracy | Macro F1 | Train time (CPU) |
|---|---|---|---|---|
| Dense network (MLP) | 235k | 89.3% | 0.893 | 1 min |
| Small CNN | 422k | 91.6% | 0.916 | 2 min |
| Deep CNN, BatchNorm + Dropout | 289k | 93.7% | 0.937 | 24 min |
| Deep CNN + augmentation | 289k | 92.6% | 0.926 | 33 min |
The deep CNN is the best model with fewer parameters than the small CNN: three convolutional blocks with batch normalisation and dropout do more with less. The augmented version was still improving when the 40-epoch cap hit, so it is 1.1 points behind here and is the one I would continue with more training time.
The confidence gate
A 93.7% model is wrong on about one image in sixteen, which is too many for a shop to publish unchecked. But the model's own confidence is informative. If I accept only predictions whose top-class probability clears a threshold and send the rest to a reviewer:
| Threshold | Tagged automatically | Accuracy on those | Sent to review |
|---|---|---|---|
| none | 100% | 93.7% | 0% |
| 0.90 | 84.5% | 98.5% | 15.5% |
| 0.95 | 80.1% | 99.1% | 19.9% |
| 0.99 | 70.6% | 99.6% | 29.4% |
At 0.95, the model tags four images in five at 99.1% accuracy and a person handles the fifth. That table is the deliverable I would hand to the business: it turns a single accuracy number into a choice between workload and error rate.
Where it fails, and why
Shirts are the hard class, with 79% recall. The top confusions are shirt with T-shirt (78 and 75 of 1,000 in each direction) and shirt mistaken for coat (57). Trousers, bags and sandals are all above 98.5%.
The gate does not catch everything: a fifth of the mistakes are made at 0.95 confidence or more. Looking at those images, most are items a person would also hesitate over. That is useful to know before promising a client 99%.
What the network looks at
Grad-CAM highlights the parts of an image that drove a prediction. For the recommended model it lands on collars, sleeves and the overall silhouette rather than the background. Checking this is a quick guard against a model that is right for the wrong reasons.
Limits
The models were trained on a CPU, so the 40-epoch cap was a time budget rather than a choice. Fashion-MNIST images are small, centred and greyscale; real product photos are not, and the gate's numbers would have to be measured again on real data.
- Code, notebook and the 10-page report: github.com/engomaressam/fashion-mnist-cnn-gradcam
- Kaggle notebook (GPU): Fashion-MNIST CNNs, confidence gating and Grad-CAM
- Model: omaressamrme/fashion-mnist-deep-cnn
Omar Essam El Sherbiny. Head of Data at WalaPlus. Data platforms on Google Cloud, data governance, applied ML. More at omaressam.web.app.
Top comments (0)