Learning Advanced Mathematics

For me personally Math was one of those mysterious subjects I had to go through in school but never really understood, let alone appreciated. It was too abstract, involved lengthy computations, rode formula memorization with little to no explanation as to why it’s useful and how it’s applied in the real world. Frankly put Math was one of my weakest spots. My parents were surprised and shocked when I told them that I planned to study Computer Science, which is a branch of applied Mathematics. Throughout my life I had a love-hate relationship with Math. I still remember that feeling of relief when I passed my last Math exam in college.

During my career as a Software Engineer I was mostly Math absent. From time to time I consulted old Computer Science books to do some research on algorithms I then implemented. However those were usually the only touchpoints I had with Math.

Something changed over the last couple of years. While looking for the next personal challenges and goals to grow I figured that most of the really exciting achievements heavily utilize Math as a fundamental building block. That’s actually true for a lot of scientific fields including Econometrics, Data Science and Artifical Intelligence. It’s easy to follow the news and roughly understand how things might work but once you try to dig deeper and look under the hood it gets pretty hairy.

I found myself regularly lost somewhere in the dark alleys of Linear Algebra, Calculus and Statistics. Last year I finally stuck a fork in the road. I wanted to fundamentally change my understanding and decided to re-learn Math from scratch. After countless late nights, early mornings and weekends doing classes, exercises and proofs I’m finally at a pretty decent level of understanding advanced Mathematics. Right now I’m building upon this foundation to learn even more.

During this process I learned one important thing: Math is really amazing!

Math is the language of nature. Understanding it helps you understand how our world works!

With this blog post I’d like to share how I went from “What is a root of a polynomial again?” to “Generalized Autoregressive Conditional Heteroskedasticity” (at least to some extend). I’ll share the Curriculum I created and followed, the mistakes I made (spoiler: I made a lot) and the most useful resources I used throughout this journey.

Before we start I want to be honest with you: Math is a really involved discipline. There’s a lot out there…

And it can certainly be overwhelming. However if you’re really dedicated and want to put in those hours you’ll make it! If I can do it so can you!

Please keep in mind that this is the path which worked for me. This doesn’t necessarily mean that it will be as efficient for you. In my case I need to study, self-explain and practice, practice, practice to really understand a topic at hand. I know of people who can just sit in class, listen and ultimately get it. That’s definitely not how I operate.

Alright. Let’s get started!

The Curriculum

Math is one of those subjects where you’ll find a nearly endless stream of resources. Looking closer they all vary widely in terms of quality, density and understandability.

My first approach to ramp up my Math skills was to skim through an interesting research paper, write down all the Math I won’t understand and look those terms up to study them in greater detail. This was fundamentally wrong on many levels. After some trial and error I took a step back and did a lot of research to figure out which topics I should study to support my goal and how those topics are related to one another.

The Curriculum I finally put together is a good foundation if you want to jump into other “Hard Sciences”. My personal goal was to learn the knowledge I need to take a really deep dive into Artificial Intelligence. To be more specific I’m really excited about Deep Learning and the next steps in the direction of Machine intelligence.

Every topic which is covered in this Curriculum uses 3 basic pillars to build a solid Mathematical foundation:

Intuition

Videos, interactive visualizations and other helpful resources which outline how the Math came to be and how it works on an intuitive level.

Deep Dive

A good enough “deep dive” to get familiar with the foundational concepts while avoiding confusion due to overuse of theorems, proofs, lemmas, etc.

Practicality

Practice, practice, practice. Resources such as books with lots of exercises to solidify the knowledge.

Algebra

Algebra is the first topic which should be studied extensively.

Having a really good understanding of Algebra makes everything a whole lot easier! Calculus comes down to 90% Algebra most of the time. If you know how to solve Algebra problems you won’t have a hard time in Calculus either.

Most of you might remember a phrase similar to

“Solve this equation for x”

That’s what Algebra is about. In an Algebra class you’ll learn about the following topics:

• Solving equations
• Solving inequalities
• Polynomials
• Factoring
• Functions
• Graphing
• Symmetry
• Fractions
• Radicals
• Exponents
• Logarithms
• Linear systems of equations
• Nonlinear systems of equations

As stated above it’s of uber importance that you really hone your Algebra skills. I’m repeating myself but Algebra is one of the main building blocks for advanced Mathematics.

Resources

• Paul’s Online Notes - Algebra
• No Bullshit Guide to Mathematics
• Schaum’s Outlines - College Algebra

Trigonometry

In Trigonometry you’ll study the relationship of lengths and angles of triangles.

You’ll learn about the unit circle and it’s relation to sin and cos, cones and their relation to circles, ellipses, parabolas and hyperbolas, Pythagoras’ Theorem and more. Trigonometry is interesting in itself since it can be immediately applied to real life problems.

Here’s a list of topics you’ll usually learn in a Trigonometry class:

• Pythagoras’ Theorem
• Sin and cos
• The unit circle
• Trigonometric identities
• Radians vs. Degree

Generally speaking this course is rather short. Nevertheless it’s a good preparation class for Calculus.

Resources

• Paul’s Online Notes - Algebra Trig Review
• No Bullshit Guide to Mathematics
• Schaum’s Outlines - Trigonometry

Calculus

The study of continuous change is one of the main focus areas in Calculus.

This might sound rather abstract and the intuition behind it is really paradox if you think about it (see “Essence of Calculus” below). However you might remember that you dealt with Derivatives, Limits and area calculations for functions.

There are usually 3 different Calculus classes (namely Calculus I, II and II) one can take. Those 3 classes range from easy topics such as “Derivatives” and “Limits” to advanced topics such as “Triple Integrals in Spherical Coordinates”. I’d suggest to definitely take the first class (Calculus I) and continue with the second one (Calculus II) if time permits. If you’re in a hurry taking Calculus I is usually sufficient.

In Calculus I you’ll learn about:

• Limits
• Continuity
• L’Hospitals Rule
• Derivatives
• Power, Product, Quotient, Chain rule
• Higher Order Derivatives
• Min / Max Values
• Concavity
• Integrals
• Substitution Rule

Calculus is an important topic since it’s heavily used in optimization problems to find local minima. The “Gradient Descent” algorithm uses techniques from Calculus such as Derivatives and is leveraged in modern (Deep) Neural Networks to adjust the weights of Neurons during Backpropagation.

Resources

• 3Blue1Brown - Essence of Calculus
• The Learning Machine - Calculus
• Paul’s Online Notes - Calculus I
• No Bullshit Guide to Math and Physics
• Schaum’s Outlines - Calculus

Linear Algebra

Linear Algebra is one of the most, if not the most important topic when learning Math for Data Science, Artificial Intelligence and Deep Learning.

Linear Algebra is pretty much omnipresent in modern computing since it lets you efficiently do calculations on multi-dimensional data. During childhood you probably spent quite some time in in front of your computer screen while wading through virtual worlds. Photorealistic 3D renderings are possible thanks to Math and more specifically Linear Algebra.

Linear Algebra courses usually cover:

• Systems of Equations
• Vectors
• Matrices
• Inverse Matrices
• Identity Matrix
• Matrix Arithmetic
• Determinants
• Dot & Cross Product
• Vector Spaces
• Basis and Dimension
• Linear Transformation
• Eigenvectors & Eigenvalues

As already stated above, Linear Algebra is one of the most important topics in modern computing. Lots of problems such as image recognition can be broken down into calculations on multi-dimensional data.

You might have heard about the Machine Learning framework TensorFlow which was developed and made publicly available by Google. Well, a Tensor is just fancy word for a higher-dimensional way to organize information. Hence a Scalar is a Tensor of rank 0, a Vector is a Tensor of rank 1, a N x N Matrix is a Tensor of rank 2, etc.

Another interesting fact is that Deep Neural Networks are usually trained on GPUs (Graphic Processing Unit) or TPUs (Tensor Processing Unit). The simple reason is that GPUs and TPUs are way better at processing Linear Algebra computations compared to CPUs since (at least GPUs) were invented as a dedicated hardware unit to do exactly that when rendering computer graphics.

Aside: Here’s the original paper by Andrew Ng et al. where GPUs were first explored to carry out Deep Learning computations.

Resources

• 3Blue1Brown - Essence of Linear Algebra
• The Learning Machine - Linear Algebra
• Paul’s Online Notes - Linear Algebra
• No Bullshit Guide to Linear Algebra
• Schaum’s Outlines - Linear Algebra

Statistics & Probabilities

The last topic which should be covered in this Curriculum is Statistics & Probabilities.

While both topics are sometimes taught separately it makes sense to learn them in conjunction since statistics and probabilities share a deep underlying relationship.

A typical Statistics & Probabilities class covers:

• Charting and plotting
• Probability
• Conditional Probability
• Bayes Rule
• Probability Distributions
• Average
• Variance
• Binomial Distribution
• Central Limit Theorem
• Normal Distribution
• Confidence Intervals
• Hypothesis Test
• Regression
• Correlation

In Data Science one usually has to deal with statistical analysis to see if the computations actually made sense. Furthermore it’s helpful to compute and visualize correlations between data and certain events. Bayes Rule is another important tool which helps us update our belief about our view of “the world” when more evidence is available. The realms of Machine Learning and Deep Learning usually deal with lots of uncertainty. Having a good toolbox to deal with this makes our life a whole lot easier.

A pretty popular example of applied statistics is the Monte Carlo Tree Search algorithm. This heuristic algorithm was used in DeepMinds AI breakthrough “AlphaGo” to determine which moves it should consider while playing the Go boardgame.

Feel free to read through the official paper for more information about the underlying technologies. Trust me, it’s amazing to read and understand how Math played such a huge role to build such a powerful contestant.

Resources

• University of Louisville - Probability and Mathematical Statistics
• Jarkko Isotalo - Basics of Statistics
• Joseph Watkins - An Introduction to the Science of Statistics
• JBStatistics - Basics of Probability
• Brown University - Seeing Theory
• Schaum’s Outlines - Probability and Statistics

Mistakes

As I already stated above it’s been quite a journey and I made lots of mistakes along the way.

In this section I’d like to share some of those mistakes so that you don’t have to go through this yourself.

The first mistake I made was jumping straight into Math without having a clear plan / Curriculum and more importantly goal. I dived right into certain topics I picked up while reading research papers and quickly figured that some of them were too advanced since I understood only little (if anything at all). My approach was to back off and start somewhere else. “Trial an error” so to say. This was obviously very costly in terms of time and resources.

The solution here was to actually have a clear goal (learning Math to understand the underlying principles of Artificial Intelligence) and take the time to research a lot to come up with a sound Curriculum and start from there. Having that sorted out I only had to follow the path and knew that I was good.

During this aforementioned trial and error phase I made the mistake of taking way too many MOOCs. Don’t get me wrong, MOOCs are great! It has never been possible before to take an MIT course from your couch. In my case exactly that was the problem. Most of the time I was passively watching the course content nodding along. After a couple of “completed courses” and the feeling of knowing the ins and outs I jumped into more sophisticated problems to figure that I developed a pretty shallow knowledge.

Doing a retrospective on the “completed courses” I saw that my learning style isn’t really tailored to MOOCs. I decided to switch my focus to the good old physical textbooks. I especially focused on textbooks with good didactics, lots of examples and exercises with solutions (the Schaum’s Outlines series is golden here). Switching from passively consuming to actively participating in the form of working through numerous exercises was really the breakthrough for me. It ensured that I left my comfort zone, went deep into the trenches and really battle-tested my knowledge about the topic at hand.

The other upside of using textbooks is that it made it possible to learn in a distraction free environment. No computer, no notifications, no distractions. Just me, a black coffee and my Math textbook!

Another, final tip I’d like to share is that you should really keep track of your feeling and engagement while studying. Do you feel fired up? Are you excited? Or are you just consuming and your thoughts are constantly wandering off because you don’t really care that much? If that’s the case then it’s usually time to move on. Don’t try to push through. There’s nothing worse than completing a course just for the sake of completing it. If it doesn’t feel right or isn’t working for you it’s important to let go and move on. There’s enough material and maybe the next one suits your needs.

Conclusion

In this blog post I’ve shared my journey from being someone who acknowledged Math as something one should’ve heard about to someone who learned to love Math and its applications to solve complex problems. In order to really understand and learn more about Artificial Intelligence and Deep Learning I created a Curriculum which does not only cover the underlying Math concepts of such disciplines but will serve the student well when learning more about other “Hard Sciences” such as Computer Science in general, Physics, Meteorology, Biology, etc.

I’m still early in my Math journey and there’s an infinite amount of exciting stuff to learn. With the given Curriculum I feel confident that I’ve gained a solid foundation to pick up more complex topics I’ll encounter while learning about Artificial Intelligence, Deep Learning and Advanced Mathematics in general.