K-Means is often the first unsupervised algorithm people learn, and it’s easy to run: a few lines of scikit-learn and you have clusters. The hard part is getting clusters that mean something. During my internship at Valentius Kryptix, I clustered 178 wines using 13 chemical measurements, with the grape labels hidden. Here are the three mistakes I worked to avoid.
Mistake 1: Forgetting to scale
K-Means groups points by distance, so any feature with a large numeric range quietly takes over. In the Wine dataset, proline runs from roughly 280 to 1,680, while hue stays between about 0.5 and 1.7. Unscaled, proline would drive nearly every decision and the other twelve features would barely matter.
The fix is one line: StandardScaler(). Every feature then has a mean of 0 and a standard deviation of 1, so each one gets an equal say.
Mistake 2: Guessing k
K-Means makes you choose the number of clusters up front, and a wrong guess still gives you tidy-looking output. I tested k from 2 to 10 with two methods. The elbow method looks at inertia, which always falls as k grows, and asks where the gains flatten out. The silhouette score measures how well each point fits its own cluster compared with the nearest other one.
Both pointed to k=3. The silhouette peaked at 0.285, which is moderate. That’s worth stating honestly: the structure is real, but the clusters aren’t sharply separated.
Mistake 3: Trusting a 2D plot too much
You can’t plot 13 dimensions, so I used PCA to project the data down to two. It’s tempting to treat the resulting scatter plot as the full picture, but my two components captured only 55.4% of the variance (36.2% and 19.2%). Roughly 45% of the information isn’t in the plot.
That means two clusters can overlap on screen while being separate in the full space. I ran K-Means on the full scaled data and used PCA only for visualization, so the clustering never depended on the projection.
Did it work?
The clusters had distinct chemical profiles. One was full-bodied and rich in phenolics, one was dark but low in phenolics, and one was light and low in alcohol. To test this, I compared the clusters with the hidden cultivar labels after the fact. The adjusted Rand index was 0.897, and only about 6 of the 178 wines were grouped differently from their true variety. The algorithm recovered the grape types from chemistry alone.
Takeaways
Scale your features before any distance-based algorithm.
Back up your choice of k with more than one method.
Report explained variance alongside any PCA plot.
Describe what your clusters mean in plain language. That interpretation is the real result.
The full walkthrough, with plots and code, is in my main article.
👉 Read the full article: https://valentiuskryptix.com/finding-hidden-structure-using-k-means-clustering-and-pca/
Code on GitHub: https://github.com/VivekChaudhari786/K-means-Cluster-and-PCA
Top comments (0)