Python sigmoid()

$$
f(x) = \frac{1}{ (1 + e^{-x}) }
$$

$$
f'(x) = f * (1-f)
$$

pip install numpy matplotlib sympy

import numpy as np
import matplotlib.pyplot as plt
import sympy as sp

x = sp.Symbol('x')
f1 = 1 / (1 + sp.exp(-1*x))
df1 = sp.diff(f1, x)

f1 = sp.lambdify(x, f1, 'numpy')
df1 = sp.lambdify(x, df1, 'numpy')

x = np.linspace(-10, 10, 400)
y = f1(x)
y_df = df1(x)

plt.figure(figsize=(8, 5))
# 1行2列
plt1 = plt.subplot(1,2,1)
plt2 = plt.subplot(1,2,2)
plt1.plot(x, y, label='f(x)', color='blue')
plt2.plot(x, y_df, label='df(x)', color='red')

plt1.grid()
plt2.grid()
plt1.legend()
plt2.legend()
plt.show()

sigmoid 函数可以将任意的输入映射到 (0, 1) 之间
输入值在 [-6, 6] 之间输出值才会有明显差异，
输入值在 [-3, 3] 之间才会有比较好的效果。

The sigmoid function maps any input to a value within the interval (0, 1).
The output varies significantly only when the input is within the range [-6, 6],
and performs well especially when the input is within [-3, 3].

sigmoid 导数范围是 (0, 0.25)
当输入 <-6 或者 >6 时，sigmoid 激活函数图像的导数接近为 0，
此时网络参数将更新极其缓慢，或者无法更新。

The derivative of the sigmoid function lies in the range (0, 0.25).
When the input is less than -6 or greater than 6, the derivative of the sigmoid activation function approaches zero,
causing the network parameters to update very slowly or even stall completely.