Global or Local Minima and Maxima and Saddle Points in Deep Learning

<Global and Local Minima>

• A global minimum is the globally minimal point whose gradient is zero but not a local minimum.

• A local minimum is the locally minimal point whose gradient is zero but not a global minimum.

*Memos:

• I used math3d.
• The left formula is 10x/e^(x^2+y^2)(-x)^e. *x∈ is [-3, 3] and y∈ is [-3, 3].
• The right formula is -4x/e^(x^2+y^2)(x)^e. *x∈ is [-3, 3] and y∈ is [-3, 3].

---

<Global and Local Maxima>

• A global maximum is the globally maximal point whose gradient is zero but not a local maximum.

• A local maximum is the locally maximal point whose gradient is zero but not a global maximum.

*Memos:

• I used math3d.
• The left formula is -10x/e^(x^2+y^2)(-x)^e. *x∈ is [-3, 3] and y∈ is [-3, 3].
• The right formula is 4x/e^(x^2+y^2)(x)^e. *x∈ is [-3, 3] and y∈ is [-3, 3].

---

<Saddle Points>

A saddle point is the combination point of a local minimum and maximum.

*Memos:

• I used math3d.
• The formula is x^2-y^2. *x∈ is [-4, 4] and y∈ is [-4, 4].