Simple Linear Regression

Linear Regression helps us to predict value of dependent variable(y) based on given independent variables(x1,x2,........).

We prefer Linear Regression Algorithm when there is a Linear Relationship between independent and dependent variable.

If there is a linear relation between dependent variable(y) and independent variables then, equation for linear regression will be,

y=m1.x1 + m2.x2 + m3.x3 + -------- 
                       ---+mn.xn+c

If only one independent variable is given then equation will be,

              y=mx+c 

So, when dependent variable is dependent linearly only on one independent variable then we will use Simple Linear Regression.

Given 2 features YearsExperience and Salary and we have to predict the Salary using YearsExperience Feature.

Lets first visualize our data,

As we can see that there is a linear Relation between both the features.

Now we have to find the best fit line that is, the Regression Line.
So, to get the best fit line, we have to find best suitable 'm' and 'c' values.
‘m’ is the values of slope and ‘c’ is the intercept.

The line will be best fit if it has minimum error difference between predicted and actual value.
For e.g.
Mean Squared Error (M.S.E)=
1/n(Σ(predicted_y - actual_y)**2)
So our main aim is to minimize the value of Error in order to get best possible fit that is, our regression line.

Implementation of Simple Linear Regression using different techniques like Least Square Method , Sklearn Library and Gradient Descent is given here