Exploring the SoftMax Function: The Better Way to Interpret Neural Network Outputs

In the previous article, we looked into the ArgMax function, how it is used, and its limitations.

In this article, we will explore the softmax function.

Let's find out how SoftMax works in our existing example.

Our output values are:

• Setosa = 1.43
• Versicolor = -0.4
• Virginica = 0.23

The softmax value for Setosa is calculated as:

SoftMax_setosa = e^setosa / (e^setosa + e^versicolor + e^virginica)
               = e^1.43 / (e^1.43 + e^-0.4 + e^0.23) 
               = 0.69

Next, let's calculate the softmax output value for Versicolor:

SoftMax_versicolor = e^versicolor / (e^setosa + e^versicolor + e^virginica)
                   = e^-0.4 / (e^1.43 + e^-0.4 + e^0.23) 
                   = 0.10

Finally, the softmax value for Virginica is:

SoftMax_virginica = e^virginica / (e^setosa + e^versicolor + e^virginica)
                   = e^0.23 / (e^1.43 + e^-0.4 + e^0.23) 
                   = 0.21

Now, let's summarize the softmax output values:

• Setosa (1.43) = 0.69
• Versicolor (-0.4) = 0.10
• Virginica (0.23) = 0.21

Observations:

• The highest raw value, Setosa, has the highest softmax value.
• The lowest raw value, Versicolor, has the lowest softmax value.
• Virginica, which has a raw value in between, has a softmax value in between.

Another important point is that all softmax values are between 0 and 1. The softmax function always ensures this.

Additionally, if we add all softmax output values, the result is 1.

That’s it for the softmax function. In the next article, we will discuss the general form of softmax and its derivatives.

Looking for an easier way to install tools, libraries, or entire repositories?
Try Installerpedia: a community-driven, structured installation platform that lets you install almost anything with minimal hassle and clear, reliable guidance.

Just run:

ipm install repo-name

… and you’re done! 🚀

🔗 Explore Installerpedia here