Introduction
Machine learning is a branch of artificial intelligence that allows computers to learn patterns from data. Instead of writing every rule manually, we provide a computer with examples and let it discover useful relationships. Machine learning is used in many parts of daily life, including recommendation systems, fraud detection, voice assistants, online shopping, medical research, social media, and banking.
There are several major types of machine learning. One of the most important is called unsupervised machine learning. It is especially useful when we have a large amount of data but do not already know the correct answers or categories for that data.
What is Unsupervised Machine Learning?
In traditional Supervised Machine Learning, models learn with a teacher or a supervisor. We feed the computer input data paired with correct answers (called labels). For example, we show thousands of pictures labeled "Cat" or "Dog", and the model learns the relationship between the pixels and the labels.In Unsupervised Machine Learning, there is no teacher, no supervisor, and—most importantly—no ground truth labels.We feed the algorithm raw data without target outputs. The algorithm's sole goal is to inspect the data, uncover hidden mathematical structures, detect repeating patterns, and group similar data points together.
Supervised vs. Unsupervised Learning
Why Do We Need Unsupervised Learning?
Data Labeling is Expensive and Slow: In the real world, human annotation is time-consuming and costly. Unsupervised learning allows us to make sense of vast datasets before or without labeling them.
Discovering Unknown Patterns: Humans are inherently limited by their own biases and domain knowledge. Unsupervised learning can discover connections in data that humans never thought to look for.
Data Compression and Feature Extraction: Unsupervised techniques help simplify complex datasets, making them easier to visualize, store, and feed into downstream predictive models.
Core Branch 1: Clustering
Clustering is the task of partitioning a dataset into distinct groups (or clusters) such that data points in the same group are more similar to each other than to those in other groups.
Three most important clustering algorithms.
A. K-Means Clustering
K-Means is the workhorse of clustering algorithms due to its speed and simplicity.
How K-Means Works Step-by-Step:
Choose
$K$: Decide how many clusters ($K$) you want to discover.Initialize Centroids: Randomly place $K$ points in the feature space. These points act as the initial center points (centroids) of your clusters.
Assign Points: Calculate the distance (usually Euclidean distance) between every data point and all
$K$centroids. Assign each data point to its nearest centroid.Update Centroids: Recompute the position of each centroid by taking the average (mean) of all data points assigned to that cluster.
Repeat: Repeat steps 3 and 4 until the centroids stop moving (convergence) or a maximum number of iterations is reached.
The Math Behind Distance Calculation
To measure how close two data points $A = (a_1, a_2, \dots, a_n)$ and $B = (b_1, b_2, \dots, b_n)$ are, K-Means uses the standard Euclidean distance formula:
$d(A, B) = \sqrt{\sum_{i=1}^{n} (a_i - b_i)^2}$$
Finding the Right $K$:
The Elbow Method
We plot the Within-Cluster Sum of Squares (WCSS) against various values of $K$. As $K$ increases, WCSS decreases because clusters become smaller and tighter. The optimal $K$ is located at the "elbow" point—where the rate of decrease dramatically slows down.
$$\text{WCSS} = \sum_{k=1}^{K} \sum_{x \in C_k} \vert{}\vert{}x - \mu_k\vert{}\vert{}^2$$
(Where $C_k$ is the set of points in cluster $k$, and $\mu_k$
is the mean/centroid of cluster $k$.)
B. Hierarchical Clustering
Unlike K-Means, which requires you to pre-define $K$, Hierarchical Clustering creates a nested hierarchy of clusters presented as a tree-like diagram called a Dendrogram.
There are two primary approaches:
Agglomerative (Bottom-Up): Starts with every single data point as its own individual cluster. In each step, the two closest clusters are merged until only one grand cluster remains.
Divisive (Top-Down): Starts with all data points inside a single master cluster and recursively splits them into smaller sub-clusters.
What is a Dendrogram?
A dendrogram illustrates how clusters are progressively combined or split. By drawing a horizontal line across the dendrogram at a specific height threshold, you can "cut" the tree and choose the number of clusters that best fits your problem.
C. DBSCAN (Density-Based Spatial Clustering of Applications with Noise)
Both K-Means and Hierarchical Clustering struggle when clusters have arbitrary, non-spherical shapes (like concentric circles or crescent shapes) or when data contains significant background noise.DBSCAN clusters points based on local data density rather than centroid distances.
Key Concepts of DBSCAN:
$\epsilon$ (Epsilon): The maximum radius around a point to search for neighbors.
MinPts: The minimum number of points required within the $\epsilon$-neighborhood to consider that area a "dense region".
Point Classification in DBSCAN:
1. Core Point: Has at least MinPts within its $\epsilon$-radius.
2. Border Point: Lies within the $\epsilon$-radius of a Core Point but has fewer than MinPts in its own radius.
3. Noise Point (Outlier): Any point that is neither a Core Point nor a Border Point.
Advantage of DBSCAN: It automatically identifies outliers as noise and can discover complex cluster topologies without needing $K$ specified upfront
Core Branch 2: Dimensionality Reduction
Modern datasets often suffer from the Curiosity of High Dimensionality.
Dimensionality Reduction reduces the number of random variables under consideration by obtaining a set of principal features, compressing the data while retaining as much critical information as possible.
A. Principal Component Analysis (PCA)
PCA is a linear dimensionality reduction technique. It reorients the dataset into a new coordinate system such that maximum variance is captured in the fewest possible axes.
How PCA Works (Intuition):
Center the Data: Subtract the mean from each feature vector so the data centers around the origin
$(0,0)$.Compute Covariance Matrix: Calculate how each variable correlates with every other variable.
-
Find Eigenvectors and Eigenvalues:
- Eigenvectors represent the directions of the new coordinate axes (called Principal Components).
- Eigenvalues represent the amount of variance captured along each Principal Component.
Select Top Components: Sort the components by their eigenvalues and select the top
$k$components that explain most of the total variance (e.g.,$95\%$of original variance).
B. t-SNE (t-Distributed Stochastic Neighbor Embedding)
While PCA searches for global linear relationships, t-SNE is a non-linear technique designed specifically for visualizing high-dimensional data in 2D or 3D space.
Intuition Behind t-SNE:
- Computes similarity probabilities between pairs of data points in high-dimensional space using a Gaussian distribution.
- Constructs a low-dimensional map (usually 2D) with points positioned such that neighboring points in the high-dimensional space remain close neighbors in the low-dimensional map.
- Uses a Student-t distribution in the low-dimensional space to solve the "crowding problem," allowing clusters to spread out nicely for clear visualization.
Core Branch 3: Anomaly Detection
Anomaly Detection (or Outlier Detection) is the process of identifying rare events, items, or observations that raise suspicion by differing significantly from the vast majority of the data.
Because anomalies are rare by definition, labeled anomaly datasets are extremely scarce. Unsupervised algorithms excel here by learning what "normal" data looks like and flagging anything that deviates from the norm.
Key Algorithms for Anomaly Detection:
Isolation Forest: An tree-based algorithm that isolates anomalies instead of profiling normal points. Because anomalies are rare and different, they require fewer splits in a decision tree to be isolated compared to normal points.
One-Class SVM: A variation of Support Vector Machines that fits a tight boundary around normal data points. Anything falling outside this decision boundary is flagged as an anomaly.
Isolation Forest Split Tree Depth:
Normal Points: Root -> Split 1 -> Split 2 -> Split 3 -> Split 4 (Deep)
Anomaly Point: Root -> Split 1 (Isolated early!)
Core Branch 4: Autoencoders (Deep Unsupervised Learning)
When we combine unsupervised concepts with Deep Neural Networks, we get Autoencoders.
An Autoencoder is a neural network designed to copy its input to its output through a constrained bottleneck layer.
[ Input Data X ] ---> ( Encoder ) ---> [ Bottleneck / Latent Space ] ---> ( Decoder ) ---> [ Reconstruction X' ]
The Architecture:
1. Encoder: A series of layers that compresses the input data $X$ into a lower-dimensional representation (the Latent Space or Code).
2. Bottleneck: The narrowest layer of the network that restricts the flow of information, forcing the network to learn only the most essential features.
3. Decoder: A series of layers that attempts to reconstruct the original input from the latent code.
Loss Function:
The loss function measures how closely the reconstructed output $\hat{X}$ matches the original input $X$ (often measured via Mean Squared Error):
$$\text{Reconstruction Loss} = \vert{}\vert{}X - \hat{X}\vert{}\vert{}^2$$
Applications of Autoencoders:
Image Denoising: Train the network using noisy images as input and clean images as output; the network learns to strip out background noise.
Dimensionality Reduction: Non-linear compression that often outperforms linear PCA.
Anomaly Reconstruction: If an autoencoder is trained only on normal data, it will fail to reconstruct abnormal inputs, yielding a high reconstruction error that alerts operators.
Real-World Applications
Unsupervised Machine Learning powers dozens of critical services across modern industries:
Customer Segmentation in Marketing: E-commerce platforms group customers by browsing behavior, purchase history, and spending habits to build personalized marketing campaigns.
Fraud Detection in Banking: Credit card processors monitor transactional patterns and flag unusual purchases occurring in unexpected geographical locations.
Gene Expression Analysis in Genomics: Scientists cluster human genetic patterns to uncover previously unknown biological sub-types of complex diseases.
Recommendation Engine Pre-processing: Streaming platforms use dimensionality reduction to handle massive user-item interaction matrices before generating recommendations.
Document Topic Modeling: Natural Language Processing (NLP) models organize millions of unstructured news articles into distinct topic groups automatically.
Summary & Key Takeaways
Unsupervised machine learning provides the toolkit to transform raw, unlabeled chaos into structured knowledge.
Clustering (K-Means, Hierarchical, DBSCAN) groups similar data points together based on distance or density metrics.
Dimensionality Reduction (PCA, t-SNE) compresses high-dimensional feature spaces down to manageable components while preserving critical structural information.
Anomaly Detection isolates rare events by measuring deviations from learned normal distributions.
Autoencoders leverage deep learning bottlenecks to extract rich latent representations from complex, unstructured data like images and video.
Conclusion
Unsupervised machine learning is a powerful way of learning from data that does not have predefined answers or labels. Instead of being told exactly what to look for, the model searches for hidden structures, similarities, relationships, and unusual observations.
Its main uses include clustering similar items, finding products that occur together, simplifying complex data, and detecting unusual behavior. It is useful in business, health, cybersecurity, research, education, online platforms, and many other fields.
The most important idea to remember is this:
Unsupervised machine learning helps computers discover patterns in data when no one has already provided the correct categories or answers.
Top comments (0)