When I first started learning machine learning, I naturally gravitated toward supervised learning.
It made sense.
Give the model some data, tell it what the correct answer is, train it to recognize the relationship, and then use it to make predictions.
But eventually I came across a different question:
What happens when we don't have the answers?
What if our dataset doesn't have a target column telling us what each observation is?
What if, instead of asking a model to predict something, we want it to help us discover what is already hiding inside the data?
That is where unsupervised learning becomes interesting.
So, what exactly is unsupervised learning?
Unsupervised learning is a branch of machine learning where algorithms work with data without predefined target labels.
Instead of learning:
"These are the correct answers. Learn to predict them."
we essentially say:
"Here is the data. Find something interesting."
The algorithm looks for patterns, similarities, structures, relationships, or unusual observations within the data.
This can be incredibly useful because real-world datasets don't always come neatly labeled.
Imagine having thousands of customers but no column saying:
- Customer type A
- Customer type B
- Customer type C
Instead, you might have information about their income, spending habits, transaction frequency, savings, age, and other characteristics.
You can use unsupervised learning to investigate whether naturally occurring groups exist.
And this is one of the things I find most interesting about it:
The model isn't necessarily confirming what we already know. It can help us discover what we didn't know to look for.
The Main Families of Unsupervised Learning
Unsupervised learning isn't just one algorithm.
It is more useful to think of it as a collection of approaches designed to answer different questions.
Some of the major categories include:
- Clustering
- Dimensionality reduction
- Density estimation
- Anomaly/outlier detection
- Mixture models
- Matrix factorization and decomposition
Let's break these down.
1. Clustering
Clustering is probably the first thing most people encounter when learning unsupervised learning.
The goal is simple:
Group observations that are similar to each other.
Suppose I have customer data:
Customer Income Spending Savings
A 30,000 8,000 5,000
B 32,000 7,500 4,800
C 90,000 25,000 30,000
D 95,000 28,000 35,000
There is no target telling us which customers belong together.
A clustering algorithm might discover that A and B are quite similar, while C and D form another group.
That's the basic idea.
There are several clustering algorithms, and they don't all define "similarity" in the same way.
K-Means
K-Means is probably the most recognizable clustering algorithm.
The basic idea is to divide observations into K clusters, with each observation assigned to the cluster whose center is closest according to the algorithm's objective.
One of the first things I learned with K-Means was that choosing K isn't as simple as saying:
K = 3
and moving on.
You need to investigate whether that number of clusters actually makes sense.
Two common techniques are the:
- Elbow Method
- Silhouette Score
The Elbow Method looks at how the within-cluster sum of squares changes as K increases.
The silhouette score evaluates how well observations fit within their assigned cluster compared with neighboring clusters.
This is an important lesson with clustering:
The algorithm can create clusters, but that doesn't automatically mean the clusters are meaningful.
K-Means works particularly well when clusters have relatively simple, compact geometry, but it can struggle with irregularly shaped clusters. (GitHub)
Hierarchical Clustering
Another approach I find particularly interesting is hierarchical clustering.
Instead of simply producing a flat set of clusters, hierarchical clustering builds a hierarchy of relationships between observations.
In agglomerative hierarchical clustering, every observation starts as its own cluster. The algorithm progressively merges similar clusters until everything is eventually part of one large cluster. (scikit-learn)
The result can be visualized using a dendrogram.
The dendrogram allows us to look at the different levels at which observations or groups merge and decide where we might "cut" the hierarchy.
This makes hierarchical clustering particularly useful when I want to understand not just which observations are grouped together, but also how those groups relate to one another.
DBSCAN
Not every dataset naturally forms neat circular or compact groups.
This is where algorithms such as DBSCAN become useful.
DBSCAN is a density-based clustering algorithm.
Instead of asking:
"How many clusters should I create?"
it looks for areas where observations are densely packed together.
One advantage is that it can identify clusters with more irregular shapes and can also identify observations that don't belong to dense regions.
That makes it particularly interesting for datasets where unusual observations are part of the problem rather than something we simply want to remove.
2. Dimensionality Reduction
Now let's say my dataset has 100 features.
Trying to visualize or work with all 100 dimensions isn't exactly convenient.
This is where dimensionality reduction comes in.
The goal is to represent high-dimensional data using fewer dimensions while attempting to preserve useful information or structure.
One of the most widely used approaches is Principal Component Analysis (PCA).
PCA
PCA transforms the original features into a smaller number of new variables called principal components.
The first component captures as much of the variance in the data as possible, followed by subsequent components capturing additional variance subject to the method's constraints. (Scikit-learn)
For example:
from sklearn.decomposition import PCA
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)
Now a dataset with many features can be represented using two principal components.
Why is this useful?
One reason is visualization.
I can plot:
Principal Component 1
vs.
Principal Component 2
and potentially see structures that were difficult to observe in the original feature space.
PCA can also be useful before other machine-learning techniques when a dataset has many features.
But there is an important caveat:
PCA doesn't magically make information disappear without consequences.
Reducing dimensions means making a trade-off between simplicity and information retention.
t-SNE
Another dimensionality-reduction technique you'll often encounter is t-SNE — t-distributed Stochastic Neighbor Embedding.
Unlike PCA, which focuses on preserving variance through linear transformations, t-SNE is primarily useful for visualizing complex high-dimensional relationships.
It is particularly popular for creating 2D or 3D visualizations of high-dimensional data.
However, I wouldn't treat a t-SNE plot as proof that a particular number of clusters exists.
It's a visualization tool, and its output can be influenced by its parameters and the structure of the data.
3. Anomaly Detection
Sometimes we're not interested in finding groups.
We're interested in finding the observations that don't fit.
This is anomaly detection.
Consider:
- fraudulent transactions
- unusual network activity
- abnormal sensor readings
- unusual customer behavior
- manufacturing defects
In these situations, the unusual observation may actually be the most important one.
Algorithms such as Isolation Forest, Local Outlier Factor (LOF), and One-Class SVM can be used for different forms of anomaly or novelty detection.
This is one of the areas where unsupervised learning becomes particularly practical.
Instead of asking:
"What class does this transaction belong to?"
we might ask:
"Does this transaction look unusual compared with the rest?"
Scikit-learn treats novelty and outlier detection as a distinct part of its unsupervised-learning toolkit. (Scikit-learn)
4. Gaussian Mixture Models
Another approach is the Gaussian Mixture Model (GMM).
At a high level, GMM assumes that the data can be represented as a mixture of several underlying probability distributions.
This is different from simply assigning every observation to one cluster.
A GMM can provide probabilities representing how strongly an observation belongs to different components.
For example:
Customer A
Cluster 1: 0.85
Cluster 2: 0.10
Cluster 3: 0.05
This can be useful when boundaries between groups aren't perfectly clear.
Scikit-learn includes both Gaussian Mixture and Variational Bayesian Gaussian Mixture models within its unsupervised-learning tools. (Scikit-learn)
5. Matrix Factorization and Decomposition
Another family of unsupervised methods focuses on breaking complex data into underlying components.
This includes techniques such as:
- PCA
- Independent Component Analysis (ICA)
- Non-negative Matrix Factorization (NMF)
- Latent Dirichlet Allocation (LDA)
These approaches can be useful when we believe that complicated observations may be explained by a smaller number of underlying factors or components.
For example, in text analysis, LDA can be used for topic modeling — helping identify groups of words that tend to occur together and may represent underlying topics.
So Which Algorithm Should I Use?
This is probably one of the most important questions.
There isn't one "best" unsupervised learning algorithm.
The right choice depends on the question you're asking and the structure of your data.
| Problem | Possible approach |
|---|---|
| Find natural groups | K-Means |
| Understand hierarchical relationships | Hierarchical Clustering |
| Find irregularly shaped dense groups | DBSCAN |
| Reduce many features | PCA |
| Visualize complex high-dimensional data | t-SNE |
| Find unusual observations | Isolation Forest / LOF |
| Model probabilistic groups | Gaussian Mixture Models |
| Discover latent components/topics | NMF / LDA / other decomposition methods |
Scikit-learn's own comparison of clustering methods emphasizes that different algorithms make different assumptions about cluster geometry, scalability, and the type of structure they are designed to capture. (GitHub)
What I Have Learned About Unsupervised Learning
One thing that has stood out to me while working through clustering is that unsupervised learning requires a different mindset.
In supervised learning, evaluation can feel more straightforward.
You have:
Actual value
↓
Prediction
↓
Compare them
↓
Measure performance
With unsupervised learning, the question becomes much more open-ended.
If K-Means gives me three clusters, how do I know those clusters are actually useful?
If PCA reduces my features to two components, what information did I lose?
If an anomaly detection algorithm identifies an observation as unusual, is it actually an error — or is it simply an important but rare case?
These questions require more than just running .fit() and looking at the output.
They require domain knowledge, exploration, visualization, and critical thinking.
My Practical Unsupervised Learning Workflow
When working with a real dataset, I wouldn't immediately jump into K-Means.
I'd start with understanding the data.
My workflow would look something like:
Load dataset
↓
Basic checks
↓
Missing values
↓
Duplicates
↓
Data types
↓
Outlier analysis
↓
Correlation analysis
↓
Feature selection
↓
Scaling
↓
Choose an unsupervised technique
↓
Train the model
↓
Evaluate the structure
↓
Visualize
↓
Interpret the results
And preprocessing matters.
For distance-based algorithms such as K-Means and many hierarchical clustering approaches, features with very different scales can distort the distance calculations. That's why scaling is an important part of the workflow. Scikit-learn also highlights feature scaling as relevant when methods depend on relationships between features. (Scikit-learn)
The Bigger Picture
For me, unsupervised learning is interesting because it changes the role of the data scientist.
Sometimes we're not trying to build a model that predicts an answer.
Sometimes we're trying to ask better questions.
Maybe the dataset contains customer segments we hadn't considered.
Maybe there are unusual transactions hiding among millions of normal ones.
Maybe dozens of variables can be represented by a much smaller number of meaningful components.
Maybe the groups we assumed existed don't actually appear in the data.
And that last possibility is important.
Unsupervised learning doesn't guarantee that the pattern we hoped to find exists.
Sometimes the most valuable result is discovering that our assumptions were wrong.
Final Thoughts
I'm still learning my way through machine learning, but unsupervised learning has made me appreciate something I think is easy to overlook:
Data doesn't always need to give us the answer before we can learn something from it.
Sometimes the most interesting insights come from allowing the data to reveal its own structure.
K-Means, hierarchical clustering, DBSCAN, PCA, t-SNE, Gaussian Mixture Models, anomaly detection, and decomposition methods all approach that idea differently.
The real skill isn't memorizing every algorithm.
It's learning to ask:
What am I trying to discover, what assumptions does this algorithm make, and does the structure it finds actually make sense?
That's where unsupervised learning becomes more than just another machine-learning technique.
It becomes a way of exploring data.
Top comments (0)