DEV Community

Cover image for ๐—ง๐—ต๐—ฒ ๐—บ๐—ผ๐˜€๐˜ ๐—ฐ๐—ผ๐˜‚๐—ป๐˜๐—ฒ๐—ฟ๐—ถ๐—ป๐˜๐˜‚๐—ถ๐˜๐—ถ๐˜ƒ๐—ฒ ๐˜๐—ฟ๐˜‚๐˜๐—ต ๐—ถ๐—ป ๐—”๐—œ:
Armaan Khan
Armaan Khan

Posted on

๐—ง๐—ต๐—ฒ ๐—บ๐—ผ๐˜€๐˜ ๐—ฐ๐—ผ๐˜‚๐—ป๐˜๐—ฒ๐—ฟ๐—ถ๐—ป๐˜๐˜‚๐—ถ๐˜๐—ถ๐˜ƒ๐—ฒ ๐˜๐—ฟ๐˜‚๐˜๐—ต ๐—ถ๐—ป ๐—”๐—œ:

The algorithm is rarely the bottleneck.
The data almost always is.

You can have the most sophisticated model architecture published in the latest research paper. If your data is biased, incomplete, or insufficient โ€” the model will fail.

And the reverse holds:

A simple model trained on excellent data will frequently outperform a complex model trained on mediocre data.

This is the truth that separates practitioners from spectators:

๐Ÿ”น Practitioners obsess over data quality.
๐Ÿ”น Spectators obsess over model architecture.

Here's what "obsessing over data" actually means:

๐Ÿ”น Is there enough of it? (Volume)
๐Ÿ”น Does it represent the real world? (Variety)
๐Ÿ”น Is it accurate and consistent? (Quality)
๐Ÿ”น Is it labeled correctly? (Annotation)
๐Ÿ”น Does it actually relate to the problem? (Relevance)
๐Ÿ”น Is it free from historical bias? (Fairness)

Six questions. All of them about data. None of them about algorithms.

๐—ง๐—ต๐—ฒ ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น ๐—ถ๐˜€ ๐˜๐—ต๐—ฒ ๐—ฒ๐—ป๐—ด๐—ถ๐—ป๐—ฒ.
๐—ง๐—ต๐—ฒ ๐—ฑ๐—ฎ๐˜๐—ฎ ๐—ถ๐˜€ ๐˜๐—ต๐—ฒ ๐—ณ๐˜‚๐—ฒ๐—น.
An engine without fuel is a sculpture.

Spend 80% of your effort on the fuel.
The engine is the easy part.

Day 4 of 20. Tomorrow: Why "we need AI" is the most expensive sentence in tech.

AI #DataQuality #MachineLearning #DataScience #AIFundamentals

Top comments (0)