Softmax Function

Softmax = a simple trick that turns scores into probabilities (numbers between 0 and 1 that add up to exactly 1).

Imagine you are waiting for Bus 49 and want to guess:

• Will there be lots of empty seats?
• Will there be only a few empty seats?
• Will there be no empty seats at all?

We give each situation a “happiness score”:

• Lots of empty seats → score 3 (yay! ❤️)
• Few empty seats → score 2 (okay 😐)
• No empty seats → score 1 (ugh 😩)

Now softmax magic happens in just two steps:

Step 1: Make each score much bigger using exponential (e^score). This makes good things really stand out!

• e³ ≈ 20
• e² ≈ 7
• e¹ ≈ 3

(We use easy round numbers here — actual values are 20.1, 7.4, 2.7, but close enough!)

Step 2: Add them up and divide to get probabilities.

Total = 20 + 7 + 3 = 30

Now the chances are:

• Lots of empty seats → 20 / 30 = ⅔ ≈ 67%
• Few empty seats → 7 / 30 ≈ 23%
• No empty seats → 3 / 30 = 10%

→ 67% + 23% + 10% = 100% ✓

That’s it! Softmax just says:

“Turn your scores into chances — the better score gets much more chance, but everyone still gets something, and it all adds to 100%.”

The Tiny Formula (you can almost remember it)

For any score z:

probability = eᶻ / (sum of e for all scores)

That’s why in apps, games, or AI models (like ChatGPT choosing the next word), the final answer often comes from softmax — it picks the most likely thing, but softly, with percentages.