DEV Community: mac172

Simple maths behind linear regression model

mac172 — Wed, 30 Mar 2022 09:21:11 +0000

This blog is a little bit long as I want to explain whole linear regression model with error/cost function and gradient descent algorithm

AI this term is making huge interest in peoples mind. But do you know what it is really?, to tell you it is nothing but mathematical equations playing around with large, really large, really really large datasets. Mathematics work is to describe the real world with logic that people want it, it's nothing special and that's why it is created in first place.

Back to topic, machine learning (subfield of AI) is taking most of the concept from mathematics and one of them is "linear regression".

'Linear regression is a linear model which describes linear combination between input variables to output variables'

Linear Regression

Regression means continuous, so output we get from it is also continuous. It is in supervised machine learning type, containing input with output associated with it like

{\lbrace (X_1,y_1),(X_2,y_2),...,(X_n,y_n) \rbrace}

or if you are fan of math's

in less messy way.

Main concept in linear regression comes from equation of line
y = mx + b
where, m = slope of line generally determine by

{y_2 - y_1 \above{2pt} x_2 - x_1}

and b = y-intercept

That's it the core idea in linear regression. There are lot of horrible mathematical notation used for writing clear and less messy equations, but their function are easy to understand so don't fear.

First let's start with linear regression equation, don't worry I will clear it's function that easy to understand.

Equation

h_\theta(x) = \sum_{i=0}^n {\theta_i x_i}

Now let's break down it little bit. first

h_\theta(x) = output

and

x_0 = 1

this two things are clear to us now remainings,

x = numbers of features/inputs

\theta = parameters

if you know little bit about neural network, then you heard about term called "weights". Weights are nothing but vector of numbers which help input value to match it's output value approximately.

for example 2y = 6, now what value of 'y' make this equation balance? Of course 3, that's how weights are work, it calculated again and again until best matching value obtained.

and lastly

n = number of samples or examples

Now we get output from our Linear Regression equation but what next? The output is not what we expected. How we will get our expected output?

To do this we have another two concepts called "Cost Function" and "Gradient Descent". Let's talk about cost function first.

Cost Function

Cost function is method of calculating error done by model when predicting output.

It's simply difference between predicted output and actual output.

One of them Mean Square Error

J(\theta) = {\sum_{i=0}^n {(h(x^i) - y^i)^N} \above{1pt}2}

and to take minimum value

min(J(\theta))

Here, $x^i$ and $y^i$ are $i^{th}$ value.

Gradient Descent

It is an optimization algorithm usually used to reduce cost function. Core concept is based on differentiation, it is used for finding minima by moving in negative direction.

Before moving to it, first see term called learning rate.

Learning Rate $(α)(\alpha)$

Learning rate are steps taken by gradient descent algorithm for how much area will calculate at a time.

A high learning rate maybe cover large area in less time but it can be overshoot model learning capabilities, means it learn same feature again and again and behave worse in real world situation.

Low learning rate might be useful but it takes lot of time to reach minima.

Let's move to gradient descent algorithm.

\theta_j = \theta_j - \alpha \sum_{i=0}^m {(h(x^i) - y^i) . x_j}

Above equation is the gradient descent equation. Note that we minus the differential value from our parameter (weights), why we do that? The answer is, suppose if we add them instead of minus then algorithm go in upward or postive direction which search for maxima instead of minima and we get huge error in our model and that is bad thing. Now let's break down it little bit.

We know $θj\theta_j$ as $j^{th}$ parameter.

$α\alpha$ is learning rate, how much step we want to take at a time.

$h(x^i) - y^i$ difference between $i^{th}$ predicted output and $i^{th}$ actual output.

$x_j$ is $j^{th}$ input value.

This algorithm act on all input datasets, means the time consuming for training is directly proportional to input size of datasets. If datasets is lage, time required is also large. For this we have it's another brother called "Stochastic gradient descent".

\theta_j = \theta_j + \alpha \sum_{i=0}^m {(h(x^i) - y^i) . x_j}

This equation will iterates over definite number of samples, i.e. we can specify how much input data we want to train.

pseudo code for stochastic gradient descent.

Understanding Bell Inequality of Quantum Computing

mac172 — Sun, 27 Mar 2022 16:27:26 +0000

Quantum Mechanics/Physics is rising and intersting field, right?. When learning it you come across topic called Bell Inequality. This is little guide for it.

It is a test which depends on Quantum Mechanics to describe real world experiment.

So let's know what is it and how do we calculate it.

Bell Inequality

John Stewart Bell is a Irish physicist. He devised an amazing test which distinguish between Bohr and Einstein models. Most amazing feature of this test is not only it is philosophies but also testable. So, let's take a look at it.

Before we start learning Bell Inequality, we have to learn it's basic ideas on which it depends. I hope you have basic knowledge about matrices and it's operations.

Basic

Column vector represent as "ket" $∣ψ⟩\ket{\psi}$

Row vector represent as "bra"
$⟨ψ∣\bra{\psi}$

You can look at it like this

Hadamard gate

Also known as H gate. It is single qubit operation that map qubit into equal superposition.

Simply it convert basis state into equally superposition states like this

\ket{0} = {\ket{0} + \ket{1} \above{2pt} \sqrt{2}}

\ket{1} = {\ket{0} - \ket{1} \above{2pt} \sqrt{2}}

Matrix for this is

Simply every element is 1 with scalar multiple of $\above{1pt} \sqrt{2}}$

example let's calculate hadamard matrix for $∣0⟩\ket{0}$

At top we write matrix for $∣0⟩\ket{0}$ , take look at it because it contains 1 at top and 0 and bottom describing it is at $∣0⟩\ket{0}$ state. Side of it is Hadamard matrix (H). Below is simply multiplication of 2x2 matrix and 2x1 matrix. After calculating we get 2x1 matrix containing 1 at each position indicating it has both $∣0⟩∣1⟩\ket{0} \ket{1}$ states
and at the end we can write it in simple form

\ket{0} = {\ket{0} + \ket{1} \above{2pt} \sqrt{2}}

Symbol

CNOT Gate

It is two qubit operation. So it take two qubit as input, label it as first is "control" qubit and second is "target" qubit. Main point to remember is,

If control qubit is 1 then it perform Pauli-X gate on target qubit means convert 0 to 1 or vice-versa.
If control qubit is 0, then target qubit remain unchanged.

You can consider this as classical NOT gate.
Example

\ket{00} = \ket{00}

\ket{01} = \ket{01}

\ket{10} = \ket{11}

\ket{11} = \ket{10}

Symbol

That's it we need to understand Bell Inequality, although you read EPR and Bell's story as additional resources to your knowledge.

Bell Inequality

Before getting output from Bell circuit (similar to classical logic circuit) it first undergo Hadamard transformation and then CNOT gate act on it and give us output.

Process

Now we have hadamard gates and CNOT gate with us the circuit of Bell state is

So first hadamard gate act on it like

\ket{0} = {\ket{0} + \ket{1} \above{2pt} \sqrt{2}}

And after converting $∣0⟩\ket{0}$ to it's equally superposition states we apply CNOT gate. As CNOT gate require two qubits as input.

\ket{00} = {(\ket{0} + \ket{1})\ket{0} \above{2pt} \sqrt{2}}

\ket{00} = {\ket{00} + \ket{10} \above{2pt} \sqrt{2}}

We know CNOT gate will change target qubit if control qubit is 1 else it remains unchanged.

\ket{00} = {\ket{00} + \ket{11} \above{2pt} \sqrt{2}}

Similarly we get another 3 states.

for $∣01⟩\ket{01}$

\ket{01} = {\ket{01} + \ket{10} \above{2pt} \sqrt{2}}

for $∣10⟩\ket{10}$

\ket{10} = {\ket{00} - \ket{11} \above{2pt} \sqrt{2}}

for $∣11⟩\ket{11}$

\ket{11} = {\ket{01} - \ket{10} \above{2pt} \sqrt{2}}

And above are Bell's states/pairs