Mathematics for Machine Learning - Day 17

What an absolute clown. When it's just separate examples, it's working fine, but when combined I'm suddenly confused like a headless chicken. I need to stop finding shortcuts only to just re-learn it again the right way.

Well, another lesson learned. No use dwelling on yesterday's idiocy.

Basis Change.

It's combining a few sections we've discussed before into one. So far what I've known to be related directly are:

1. Transformation Matrix
2. Linear mapping

Which combined will become... Transformation mapping. I know, so original.

So what is it?

It's transforming matrices while taking into consideration linear mapping if the basis is changed.

So, with

$$\Phi : V \to W$$

Consider 2 ordered basis of V and W.

$$B = (b_1, \dots, b_n), \tilde{B} = (\tilde{b}_1, \dots, \tilde{b}_n) \to V \\ C = (c_1, \dots, c_n), \tilde{C} = (\tilde{c}_1, \dots, \tilde{c}_n) \to W$$

Note:
If you forgot, here's what the above basis are for.

$$B \to \text{ Basis Change } \to \tilde{B} \\ C \to \text{ Basis Change } \to \tilde{C} \\ B \to \text{ Linear Mapping } \to C \\ \tilde{B} \to \text{ Linear Mapping } \to \tilde{C} \\$$

With both basis change and linear mapping can be inversed. So anything can become anything, you just gotta believe in yourself.

Oh yeah, also

$$A_{\Phi} \in \reals^{m \times n} \text{ and } \tilde{A_{\Phi}} \in \reals^{m \times n}$$

are the transformation matrix for the basis change.

So what?

So, consider this transformation matrix:

$$A = \left[\begin{array}{cc} 2 & 1 \\ 1 & 2 \\ \end{array}\right]$$

with respect to the standard basis, if we define a new basis.

$$B = ( \left[\begin{array}{c} 1 \\ 1 \\ \end{array}\right], \left[\begin{array}{c} 1 \\ -1 \\ \end{array}\right] )$$

we obtain the diagonal transformation matrix

$$\tilde{A} = \left[\begin{array}{cc} 3 & 0 \\ 0 & 1 \\ \end{array}\right]$$

which is easier to work with than A.

Huh... How?

I hope that's what you're thinking right now, because I spent well over an hour knowing how the hell did this happen.

If you already know how... Please review to see if it's really correct or not :D I'm new in this.

Step 1: Transform the new basis into a matrix

$$B = ( \left[\begin{array}{c} 1 \\ 1 \\ \end{array}\right], \left[\begin{array}{c} 1 \\ -1 \\ \end{array}\right] ) = \left[\begin{array}{cc} 1 & 1 \\ 1 & -1 \\ \end{array}\right]$$

Step 2: Find Inverse B

$$B^{-1} = \frac{1}{\Vert{B}\Vert} adj(A) \\ B^{-1} = \frac{1}{-2} \left[\begin{array}{cc} -1 & -1 \\ -1 & 1 \\ \end{array}\right] \\ B^{-1} = \left[\begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} \\ \end{array}\right]$$

Step 3: Calculate (Inverse B) AB

$$B^{-1}AB = \left[\begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} \\ \end{array}\right] \left[\begin{array}{cc} 2 & 1 \\ 1 & 2 \\ \end{array}\right] \left[\begin{array}{cc} 1 & 1 \\ 1 & -1 \\ \end{array}\right] \\$$

Calculate AB

$$B^{-1}AB = \left[\begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} \\ \end{array}\right] \left[\begin{array}{cc} 3 & 3 \\ 3 & -1 \\ \end{array}\right]$$

Calculate Inverse B to AB

$$B^{-1}AB = \left[\begin{array}{cc} 3 & 0 \\ 0 & 1 \\ \end{array}\right]$$

For some of you, this is enough.

You might already understand the steps and what does each step mean, to which I say, congratulations the other texts below aren't need for you so feel free to leave.

But honestly, I still don't get it :D.

How did I read through a bunch of sections and only now inverse is used again, what's happening here?. So I asked myself a second time.

Huh... How?

It all stems back to linear mapping, the bastard that's laughing maniacally feeling superior as I use an ineffective method on my journey to learning this.

Consider this:

$$T_{(v){std}} = P T{(v){B}} \\ T{(v){std}} = \text{Linear transformation on standard basis} \\ P = \text{Change of Basis matrix} \\ T{(v)_{B}} = \text{Linear transformation on new basis} \\$$

P is what we defined as B beforehand.

That's easy, then just inverse it to get the answer right?

$$T_{(v){B}} = P^{-1} T{(v)_{std}}$$

That's right. P is the B that we used on the example.

But what about the linear transformation T, what's that?

$$T_{(v){std}} = A{std} \left[v_{std}\right] \\ A_{std} = \text{The matrix representation of T with respect to the standard basis} \\ \left[v_{std}\right] = \text{Vectors with respect to the standard basis} \\$$

And since the vectors any vector inside of T on the standard basis, we need to map the vector to the new basis as well.

How?

$$\left[v_{std}\right] = P \left[v_{b}\right] \\$$

Notice it's familiar with the T formula for changing basis right? that's because it's the same.

Add it to the formula

$$T_{(v){std}} = A{std} P \left[v_{b}\right]$$

Combine with the new basis linear transformation.

$$T_{(v){B}} = P^{-1} A{std} P \left[v_{b}\right]$$

Boom. This gets the formula.

Sidenote:

I absolutely have no clue why some are able to be subscript but some aren't. I think there's something wrong with the notation or maybe there's a parameter that's messing it up. But either way, just incase, here's a clarification.

$$Astd = A_{std} \\ T(v)std = T_{(v)_{std}}$$

Acknowledgement

I can't overstate this: I'm truly grateful for this book being open-sourced for everyone. Many people will be able to learn and understand machine learning on a fundamental level. Whether changing careers, demystifying AI, or just learning in general, this book offers immense value even for fledgling composer such as myself. So, Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong, thank you for this book.

Source:
Axler, Sheldon. 2015. Linear Algebra Done Right. Springer
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for Machine Learning. Cambridge: Cambridge University Press.
https://mml-book.com