Support Vector Machines

MjolnirBearer — Mon, 22 Jun 2026 15:39:18 +0000

Today I brushed up SVM, and ngl, it felt confusing at first.
But once the main idea clicked, everything started making sense.

🎯 The Core Idea

SVM doesn’t just find any boundary between classes.

It finds the best possible boundary — the one with the maximum margin.

Think of it like this:

If two groups are standing apart, SVM tries to build the widest road between them.

Wider road = safer separation.

And safer separation = better predictions on new data.

📏 What is Margin?

Margin is simply:

Distance between the decision boundary and the nearest data points.

Why does it matter?

✅ Bigger margin = better generalization
✅ Less chance of messing up on unseen data

Simple but powerful.

⭐ Support Vectors

This part was interesting.

Not every point matters.

Only the points closest to the boundary actually affect where the boundary is placed.

These important points are called:

Support Vectors

That’s literally where SVM gets its name from.

Main characters fr 😤

🛣 Large Margin Classification

The whole goal:

Find the boundary with the largest margin possible.

Not just separation.

Confident separation.

That distinction made a lot of sense today.

🔒 Hard Margin SVM

Works when data is perfectly separable.

Means:

❌ No mistakes allowed
✅ Strict separation

Problem?

Very sensitive to outliers.

One weird point can mess things up.

🌊 Soft Margin SVM

This is more practical.

Real-world data is messy.

Soft margin allows:

✅ Some mistakes
✅ Better flexibility
✅ Better generalization

Basically:

“Okay, a few mistakes are fine if the overall boundary is better.”

Makes way more sense.

🎛 C Hyperparameter

This controls how strict SVM is.

Big C:

Tries hard to avoid mistakes
Smaller margin
More strict

Small C:

Allows more mistakes
Bigger margin
More relaxed

So basically:

C = strictness controller

🌀 When Linear Boundaries Fail

Sometimes data looks weird.

Like circles inside circles.

A straight line won’t work there.

That’s where SVM gets cooler.

✨ Kernel Trick

Probably the coolest part.

Kernel trick helps SVM handle non-linear data by transforming it into higher dimensions.

But the crazy part?

It does this without actually computing the transformation directly.

Felt like magic.

🔧 Types of Kernels

Polynomial Kernel

Useful when patterns are polynomial-like.

Creates curved boundaries.

RBF Kernel

Most common one.

Super flexible.

Works great for complex data.

This was SVMs in a nutshell for me..
Simple idea..yet powerful

DEV Community: MjolnirBearer