Supervised learning is the process where we give a computer an input and its expected output, forcing the computer to figure out how to map that input to that output. The model handles this using a vast number of parameters. Through supervised learning, these parameters are continuously adjusted after failing billion of times, eventually learning how to reach the correct answers.
1. The core equation: y = f[x, φ]
This formula means that from the input x, we get the predicted output y. Here, φ represents the parameters (weights and biases) that the model to find so that, through mathematical operations, it can accurately predict y from x.
Depending on what y is, supervised learning is divided into two main types of problems:
- Regression: Used when the output y is a continuous number. For example, predicting a house price or tomorrow's temperature.
- Classification: Used when the output y is a discrete label/category. For example, identifying whether an image is a "cat" or "dog", or detecting if an email is "spam" or "not spam".
This process works similarly to matrix transformations. The input shape, combined with specific parameter and weights, goes through calculations to yield the final output shape.
2. How the Model Self-Corrects: Derivatives & Gradient Descent
The process behind adjusting these parameters is driven by derivatives and gradient descent.
With the initial parameters chosen randomly, the loss is usually very high. The model must find a way to reduce this error little by little, and the key methods for this is taking derivatives. A derivative is basically a gradient (the slope of the error hill). By calculating the derivative with respect to each parameter, the model knows exactly which direction to move. This continuous adjustment forces the loss to decrease step by step, until it reach the flat bottom where it cannot be reduced anymore.
3. Evaluating Performance: Loss & Testing
Loss is the metric used to evaluate how accurate the model's prediction are. The loss value will be very highly (poor) if the model makes an incorrect prediction with a high confident, or if it makes a correct prediction but with very low confident. The ultimate goal of training is to reduce this loss.
To evaluate the model properly and prevent it from just "memorizing" answers, we don't just use one dataset. Instead, the data is usually split into three distinct parts:
- Training Set: The data the model uses to learn and adjust its parameters through gradient descent (like doing homework).
- Validation Set: The data used to test the model during training. It helps us tune the model's structure and check if it is starting to overfit (like a practice quiz before the real exam).
- Testing Set: The final exam. This data is kept completely hidden until training is finished to see how the model performs in the real world.
During this evaluation process, we often encounter two major issues:
- Underfitting: The model learns nothing and performs poorly on both training and testing data.
- Overfitting: The model just memorizes the training data, making it unable to predict anything new.
Top comments (0)