loading...
Cover image for Creating of neural network using JavaScript in 7 minutes!

Creating of neural network using JavaScript in 7 minutes!

liashchynskyi profile image Petro Liashchynskyi ・4 min read

Hey, what's up 😁 Today, i'm gonna tell you how to build a simple neural network with JavaScript by your own with no AI frameworks. Let's go!

For good understanding you need to know these things:

  • OOP, JS, ES6;
  • basic math;
  • basic linear algebra.

Simple theory

A neural network is a collection of neurons with synapses connected them. A neuron can be represented as a function that receive some input values and produced some output as a result.

Simple neuron

Every single synapse has its own weight. So, the main elements of a neural net are neurons connected into layers in specific way.

Every single neural net has at least an input layer, at least one hidden and an output layer. When each neuron in each layer is connected to all neurons in the next layer then it's called multilayer perceptron (MLP). If neural net has more than one of hidden layer then it's called Deep Neural Network (DNN).

DNN

The picture represents DNN of type 6–4–3–1 means 6 neurons in the input layer, 4 in the first hidden, 3 in the second one and 1 in the output layer.


Forward propagation

A neuron can have one or more inputs that can be an outputs of other neurons.

Synapses

  • X1 and X2 - input data;
  • w1, w2 - weights;
  • f(x1, x2) - activation function;
  • Y - output value.

So, we can describe all the stuff above by mathematical formula:

Neuron input

The formula describes neuron input value. In this formula: n - number of inputs, x - input value, w - weight, b - bias (w e won't use that feature yet, but only one thing you should know about that now - it always equals to 1).

As you can see, we need to multiply each input value by its weight and summarize products. We have sum of the products of multiplying x by w. The next step is passing the output value net through activation function. The same operation needs to be applied to each neuron in our neural net.

Finally, you know what the forward propagation is.


Backward propagation (or backpropagation or just backprop)

Backprop is one of the powerful algorithms first introduced in 1970. [Read more about how it works.]

Backprop consists of several steps you need apply to each neuron in your neural net.

  • First of all, you need to calculate error of the output layer of neural net.

error formula

target  -  true value, output  -  real output from neural net.

  • Second step is about calculating delta error value.

delta error

f'  -  derivative of activation function.

  • Calculating an error of hidden layer neurons.

hidden neuron error

synapse  -  weight of a neuron that's connected between hidden and output layer.

Then we calculate delta again, but now for hidden layer neurons.

hidden delta

output  -  output value of a neuron in a hidden layer.

  • It's time to update the weights.

weights update

lrate  -  learning rate.

Buddies, we just used the simplest backprop algorithm and gradient descent😯. If you wanna dive deeper then watch this video.

And that's it. We’re done with all math. Just code it!!!


Practice

So, we’ll create MLP for solving XOR problem (really, man? 😯).

From the simplest things to the hardest, bro. All in good time.

Input, Output for XOR.

XOR

We’ll use Node.js platform and math.js library (which is similar to numpy in Python). Run these commands in your terminal:

mkdir mlp && cd mlp 
npm init 
npm install babel-cli babel-preset-env mathjs

Let’s create a file called activations.js which will contain our activation functions definition. In our example we’ll use classical sigmoid function (oldschool, bro).

Then let’s create nn.js file that contains NeuralNetwork class implementation.

It seems that something is missing.. ohh, exactly! we need to add trainable ability to our network.

And just add predict method for producing result.

Finally, let’s create index.js file where all the stuff we created above will joined.

Predictions from our neural net:

Predictions

Conclusions

As you can see, the error of the network is going to zero with each next epoch. But you know what? I’ll tell you a secret — it won’t reach zero, bro. That thing can take very long time to be done. It won’t happens. Never.

Finally, we see results that are very close to input data. The simplest neural net, but it works!

Source code is available on my GitHub.

Original article posted by me in my native language.

Discussion

pic
Editor guide
Collapse
alin_air profile image
Alin Andrei

I haven't understood most of it. I'll be back as soon as I get it folks

Collapse
liashchynskyi profile image
Petro Liashchynskyi Author

In the next article I would try to thoroughly explain how neural network works (with math examples), if you want 😉 And you will figure it out 😁

Collapse
jenc profile image
jen chan

Same boat but still fascinating

Collapse
liashchynskyi profile image
Petro Liashchynskyi Author

Hey, check out my new article about math principles of neural networks 😀dev.to/liashchynskyi/how-neural-ne...

Collapse
alin_air profile image
Alin Andrei

Hey, your post widened my perception. JavaScript or not, neural networks are pretty cool. I am willing to research the subject

Thanks!

Collapse
jenc profile image
jen chan

Post almost unrelated, but I was reading a tut about tensorflow.js and this video about tensors is veyr mathy but also very good youtube.com/watch?v=f5liqUk0ZTw

Collapse
edo9k profile image
Eduardo França

Hi, I put the code on a RunKit notebook, for anyone who wants to see it running without having to install anything on their computer: runkit.com/edo9k/tiny-rnn

For some reason I kept getting an error in the train function, the error was output_layer is not a function, so I just wrapped the output_layer variable in an arrow function. Now it's running.

Great article, thank you.

Collapse
omairseoexpert profile image
Umair Akram

Its a magic to use this Java script and make this great neural network. I am really very impress after read this helpful knowledge. The professional Web Development Companies in Saudi Arabia from adwebstudio.com/web-design-company...

Collapse
nuwannnz profile image
Nuwan Karunarathna

Wow, nice article and awesome explanation!

Collapse
thegeoffstevens profile image
Geoff Stevens

This is awesome! Very easy to follow. It's always fun to try to do things without frameworks.

Collapse
ondrejs profile image
Ondrej

You got deserved star on GitHub. Very good article, nice demonstration, thank you!

Collapse
liashchynskyi profile image
Petro Liashchynskyi Author

ohh, thank you so much!

Collapse
jenc profile image
jen chan

Understood everything til backprop but if this is the simplest I’m glad for the explanation! Thanks bro 🤪

Collapse
charlie117 profile image
Charlie Fuentes

Amazing article and really well explained. Congratulations 🎊

And thanks for this!

Collapse
liashchynskyi profile image
Collapse
havenhaven profile image
havenhaven

And I wanna play it cool happy wheels

Collapse
omairseoexpert profile image
Umair Akram

Its very helpful post of the 7 minutes JavaScript work. I am really very thankful this. Your post give me the great knowledge. Keep it up! The top Web Developer NYC from digitalsaad.us/

Collapse
rileywilliam profile image
WilliamRiley

Exclusive guidance post that tells developers how to build a simple neural network with JavaScript without AI frameworks and in short time with easy steps.
William,
eliteassignment.co.uk

Collapse
pubgapk2 profile image
pubgapk

Thanks

Collapse
creshbunny profile image
creshbunny

I have read your article; it is very instructive and valuable to me. I admire the valuable information you offer in your articles. Thanks for posting it. Hotmail

Collapse
muhammadkashif773351 profile image
Muhammad-Kashif-773351

This is a great blog posting and very useful.SEO Expert Dubai