Feature Scaling: Normalization

What’s the Intuition Behind It?

Imagine you’ve got two measuring sticks. One’s for how tall your friends are (say, 4 to 6 feet), and the other’s for how much candy they can eat in a minute (maybe 10 to 100 pieces).

If you try to compare “height” and “candy-eating power” directly, it’s a mess! Six feet sounds tiny next to 100 candies, even though 6 feet is super tall for a person. The numbers are on totally different scales, right?

Normalization is like giving you a magic ruler that shrinks everything down to a scale from 0 to 1, so you can compare them fairly.

Here’s the formula:

$$x' = \frac{x - \min(x)}{\max(x) - \min(x)}$$

Don’t let it scare you—it’s simpler than it looks! Let’s break it down with that candy-and-height idea.

Step 1: Picture Your Numbers

Say your friends’ candy-eating scores are: 10, 50, and 100 pieces. The smallest (min) is 10, and the biggest (max) is 100. Right now, those numbers are all over the place. We want them between 0 and 1, like a percentage of “awesomeness” compared to the best candy-eater.

Step 2: Slide Everything Down

First, subtract the smallest number (10) from each score. Why? Because we want the lowest score to become 0—it’s our starting line.

• 10 - 10 = 0
• 50 - 10 = 40
• 100 - 10 = 90

Now our numbers are 0, 40, 90. See? The smallest is 0, just like we wanted. This is the $x - \min(x)$ part.

Step 3: Shrink the Range

Next, look at the original range: 100 - 10 = 90. That’s how “wide” your candy-eating universe is. We want that whole range to fit between 0 and 1, so we divide everything by 90:

• 0 ÷ 90 = 0
• 40 ÷ 90 = 0.444… (about 0.44)
• 90 ÷ 90 = 1

Boom! Now your scores are 0, 0.44, and 1. The worst candy-eater is 0, the best is 1, and your buddy in the middle is 0.44—perfectly scaled! That’s the
$x' = \frac{x - \min(x)}{\max(x) - \min(x)}$ trick.

Why Does This Make Sense?

Think of it like a stretchy rubber band. You’ve got this band that’s 90 units long (from 10 to 100), and you want to squeeze it into a tiny 1-unit box (0 to 1). First, you slide it so the start is at 0 (subtract the min), then you squish it down to fit (divide by the range). It keeps the shape of your data—the gaps between your friends’ scores stay the same, just smaller—so you can compare candy-eating to height or anything else fairly.

Why Machines Love It

Now, imagine you’re teaching a robot to guess who’s the best candy-eater. Robots like numbers that play nice together. If one number’s huge (100) and another’s tiny (6 from height), the robot might freak out and pay too much attention to the big one. Normalization evens the playing field, so the robot looks at patterns, not just bigness. That’s why it’s great for stuff like distance-based tricks (like finding “who’s closest” in a game) or anything where inputs need to be tidy and bounded.