Introduction
Sometimes, in data science, there is a need to predict the future results based on some historical data. The regression is a machine learning method used for finding relations between variables and making predictions on the numeric values like prices, sales or demands.
But complex models are prone to the effect of overfitting, which means that they study the training data too accurately and have poor results on the testing ones. That is why regularization is necessary to avoid this problem and make models simpler and better-generalized.
What Is Regression?
The regression is the supervised machine learning method which predicts the numeric value depending on one or several input variables.
For instance, the real estate company may use regression to find the price of houses according to:
- Size of the house.
- Number of bedrooms.
- Location.
- Age of the house.
Types of Regression
Linear Regression
Linear regression is the simplest regression method which supposes linear dependence between the input variables and target value.
Example:
from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(X_train, y_train)
Multiple Linear Regression
The regression method which uses several predictors for its estimations, for instance, predicting the salary according to the education level, experience and job role of the employee.
Polynomial Regression
The regression method used in cases where the dependencies between the variables are more complicated and do not depend on the straight line.
The Problem of Overfitting
If the machine learning model has extremely good performance on training data, but poor performance on testing ones – then it probably suffers from overfitting.
It means that the model studied some noises or unnecessary dependencies while studying training data.
To solve this problem, regularization is needed.
What is Regularization?
Regularization is a method where a penalty term is added to the regression model to avoid over-fitting or complexity in the model. It makes sure that the model doesn't become dependent on specific features.
Types of Regularization are:
Ridge Regression (L2)
Ridge shrinks the coefficients without excluding the features.
from sklearn.linear_model import Ridge
model = Ridge(alpha=1.0)
** Lasso Regression (L1)**
Lasso shrinks some of the coefficients to make them zero, which means some features would be excluded.
from sklearn.linear_model import Lasso
model = Lasso(alpha=0.1)
Elastic Net
It combines Ridge and Lasso together to get both the benefits of both.
| Regression | Regularization |
|---|---|
| Used for prediction and understanding relationships | Used to improve model performance |
| Minimizes prediction errors | Reduces model complexity |
| Can overfit with many features | Helps prevent overfitting |
Regression is one of the most commonly used models in data science to predict numerical results. There are two important concepts of regression, namely regularization and regression.
Regularization helps in creating simpler models that are more robust in the case of real-world applications.
Top comments (0)