Mathematics for Machine Learning - Day 12

An offer you literally can refuse.

You're not gonna like today, it's going to be really short since I'm only going to cover one section and it's been a long day. Though I'd rather be consistent than anything else, it doesn't negate the fact that this is a poor time management from me. Since I'm not a politician, I won't sugarcoat that this will happen and I can't guarantee this won't happen again but I'll keep moving forward.

Rank

So, you're stuck in low ELO hell, be it in chess, Valorant or Dota 2, we've all been there but how do you get from low ELO (or dare I say deficient rank) to a full rank? But we're going too far, let's start all the way back.

What's a rank?

A rank is the number of linearly independent columns of a matrix

$$A \in \reals^{m\times n}$$

equals the number of linearly independent rows. The rank of a is denoted by:

$$\text{Rank of A } A = rk(A)$$

Is it really that simple?

Of course not.

Properties of rank

1. The column rank equals the row rank

$$rk(A) = rk(A^T)$$

2. The columns of

$$A \in \reals^{m\times n} \text{ span a subspace } U \subseteq \reals^m$$

with the dimensions of U equal the rank of A. Later on, this subspace will be called the image/range.

3. The rows of

$$A \in \reals^{m\times n} \text{ span a subspace } W \subseteq \reals^n$$

with the dimensions of W equal the rank of A. A basis of W can be found by applying Gaussian elimination to A.

4. For all

$$A \in \reals^{m\times n} \text{ and all } b \in \reals^m$$

it holds that the linear equation system Ax = b can be solved if and only if the rank of A equals the rank of A|b, where A|b denotes the augmented system.

5. For all

$$A \in \reals^{m\times n} \text{ the subspace of solutions for } Ax = 0$$

posesses dimensions

$$n - rk(A)$$

Later on we'll call ths subspace the kernel or the null space.

6. A matrix

$$A \in \reals^{m\times n}$$

has full rank if its rank equals the largest possible rank for a matrix of the same dimensions. This means that the rank of a full rank matrix is the lesser of the number of rows and columns. i.e. rk(A) = min(m,n). A matrix is said to be rank deficient if it doesn't have full rank.

Conclusion.

If you're friend is asking for tips to climb the rank? Just tell them to be a matrix with each column being a pivot column.

Because honestly, a full rank rank is just a basis but in matrices and not vectors.

---

Acknowledgement

I can't overstate this: I'm truly grateful for this book being open-sourced for everyone. Many people will be able to learn and understand machine learning on a fundamental level. Whether changing careers, demystifying AI, or just learning in general, this book offers immense value even for fledgling composer such as myself. So, Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong, thank you for this book.

Source:
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for Machine Learning. Cambridge: Cambridge University Press.
https://mml-book.com