What is a neural network?

In previous blog posts, we have discussed Machine Learning and Deep Learning. We spoke about how they are not very different. They only differ in the algorithms and models used to solve problems.If you have heard the term ‘Deep Learning’, you would also come across ‘Neural Network’. Woah sounds ominous. In this blog post, let’s unpack the term ‘Neural Network’ and take a peek inside the closet. Is it really a monster hiding inside? Or a harmless cat?

Neural Networks in Human Brains
The term ‘neural network’ comes from the human brain. The human brain contains billions of neurons which pass signals. A huge network of these neurons passing signals is called a neural network.

The neurons used in deep learning are inspired by the human brain. You can consider them to be a simple form of the same concept.

Neural Networks and Deep Learning
As we have seen in previous posts, Machine Learning models are function approximators. They attempt to learn the relationships between inputs and outputs and create an approximation of the function between them.

As an example consider the below table

X. Y.

1 3

2 6

3 9

Here Y = 3X represents the function or relationship between the input X and output Y. In deep learning, neural networks attempt to do the same thing.

In this figure, we see how a single neuron acts as a function between inputs and outputs. In deep learning, we create a network of these neurons all working together to do the same. This increases the capability of the neural network to learn more complex relationships.

How does a neural network graph make predictions?

When it comes down to the fundamentals, neural network predictions are just maths, not magic. So let’s walk through a simple prediction. Some concepts are simplified for understanding.

The input to a neural network is a vector. A vector is just a series of numerical inputs. Therefore any data (such as text, images, audio etc) you want the neural network to process must first be converted to numerical vector form before the neural network can process it.

Let’s say the input is [x1, x2, x3….]

Each layer in the neural network has a weight matrix associated with it. [w1, w2, w3…..]

As the input passes through each layer, it is multiplied with the respective weight matrix and in the process it is transformed into something different.

After all the layers are processed in this way, we are left with the output vector [y1, y2, y3…..]

This is the output of the neural network.

If the task is to predict whether the input image is a cat or a dog, the output vector could look like this,

[ycat, ydog]

[1, 0] is a cat

[0, 1] is a dog

How does a neural network model learn?
We saw how neural networks make predictions, but how do they learn the weights for each layer in the first place?

Neural networks learn their weights using a process called ‘Training’.

Training happens when we feed a lot of data where the inputs and outputs are known, also called ‘Training Data’.

The network is initialized with random weights
It is then asked to make predictions on training data
Wherever it makes wrong predictions, The weights are changed slightly in the direction of correct predictions. This step is also called ‘neural network backpropagation’
This process is repeated over and over again till the weights hardly change, also called ‘Convergence’
The network is then asked to predict on new data which it has never seen before (Test Data). The accuracy of predictions is measured.
Conclusion
A Neural network is an advanced algorithm which is capable of learning complex relationships in the data and make accurate predictions. I like to compare them with ‘Play Doh’. A clay which children play with which can be moulded and stretched to be anything you want them to be.

If you have any feedback or you wish to know more on the AI and ML, you can also read our articles here. Or else just drop us an email at [e-mail] or visit our website www.asksid.ai to start a conversation!