DEV Community: LeviAckerman3855

Dann.js - Teaching a Neural Network how to predict San Francisco house prices

LeviAckerman3855 — Sat, 14 Aug 2021 13:04:03 +0000

Welcome back to our tutorial series!
Remember in the previous tutorial, we taught a network how to do XOR operations? In this tutorial we will teach it to predict the house prices of San Francisco.

For predicting the house prices of San Francisco, we need these parameters:

longitude
latitude
housing_median_age
total_rooms
total_bedrooms
population
households
median_income
ocean_proximity

We will be providing these to the network as a dictionary in JavaScript, so the network can easily understand it. A sample house price for a time period would be:

{"inputs":[0.1769999999999996,0.5663157894736844,0.7843137254901961,0.08932295640673484,0.06621146285950755,0.05960555695694012,0.082223318533136,0.5396689655172414,0.75],"target":[0.9022663824066705]

Here we are giving the network the nine parameters and giving a sample target which is the median house price for that time period.

You can access the used dataset here.

What you will need

Node.js
A good computer with more than 2 GB of RAM and a good CPU

Getting started!

Setting up the basic components

Environment

First, we need to set up our environment.
First we need to install Dann.js so we can use it in our programs. So run this code in your terminal after switching to the folder:

npm install dannjs

Main file

As you have learnt in the previous tutorial, we start using Dann.js in our JavaScript program by typing:

const Dannjs = require('dannjs');
const Dann = Dannjs.dann;

Now lets initialize the network by typing Dann with the arguments 9,1. Why are we using 9 input neurons and 1 output neuron? Because for the housing status, we are inputting 9 parameters. The output is only one value, so we are requiring only 1 output neuron. So we are assigning only the amount of needed neurons, which are (9,1).

const nn = Dann(9,1);

Setting up the dataset

Download the dataset from github here. Save them in the project directory as dataset.js. We will be using this in our main file.

Import the dataset to the main file:

const dataset = require("./dataset").houses

Setting up the hidden layers

I have found that 3 hidden layers work well. You can experiment with other values:

nn.addHiddenLayer(8,'leakyReLU');
nn.addHiddenLayer(8,'leakyReLU');
nn.addHiddenLayer(6,'tanH');

We are setting the loss function, to mae. MAE loss function is defined as The sum of absolute differences between our target and predicted variables. You can read more about it (here)[https://heartbeat.fritz.ai/5-regression-loss-functions-all-machine-learners-should-know-4fb140e9d4b0].

nn.outputActivation('sigmoid');
nn.setLossFunction('mae');

Now we finally weight the hidden layers:

nn.makeWeights();

Training the model

We are training the model using the dataset using traditional way, aka backPropagation. In this methods, we do a n number of times training through feeding the data manually to the network using the .backPropagate method.

let trainIndex = 0;
function train(epoch = 1) {

  // save dataset length
  let len = houses.data.length;

  // Train multiple epochs
  for (let e = 0; e < epoch; e++) {

    // Log epochs
    if (logs == true) {
      console.log("Epoch " + nn.epoch);
    }

    // Train 1 epoch
    let sum = 0;
    for (let i = 0; i < len; i++) {
      nn.backpropagate(
        houses.data[i].inputs,
        houses.data[i].target
      );
      sum += nn.loss;
    }

    // Save average epoch loss
    losses.push(sum/len);

    // Save epoch's accuracy with testing dataset
    let result = test(logs);
    accuracies.push(result);

    // Increment the Dann model's epoch
    nn.epoch++;
  }
}
train(1000);

Here we are moving away from the out-of-box training and packing the training in a callable function, and calling it.

We are also saving the returned data as we need it to view long series.

Finishing up

Now you can test the model by using .feedForward method.
Happy neural-networking!

Dann.js - Making a Neural Network solve XOR problems!

LeviAckerman3855 — Fri, 13 Aug 2021 06:36:31 +0000

As you must have read from the previous tutorial, Dann.js is a NPM module for Node.js which allows you to build a neural network easily. You can read the previous tutorial here.

In this tutorial, we will build a new neural network which will solve XOR problems.

XOR is a logical operation that is true if and only if its arguments differ (one is true, the other is false).

So basically, it is a OR gate with a condition that it will be true only when the second bit is inverse of the first. You can read more about XOR here.

A XOR has a this truth table (A table which summarizes which condition produces which output):

1st Bit         2nd Bit         Output
0               0               0
0               1               1
1               0               0
1               1               0

What you will need

Node.js
A computer with more than 2GB of ram and a good CPU

Getting started

Setup

Install Dann.js into your environment if you haven't already by doing:

npm i dannjs

As you have learnt in the previous tutorial, we start using Dann.js in our JavaScript program by typing:

const Dannjs = require('dannjs');
const Dann = Dannjs.dann;

For training the network to do XOR operations, we import the xor dataset from Dann.js:

const xor = Dannjs.xor;

Now lets initialize the network by typing Dann with the arguments 2,1. Why are we using 2 input neurons and 1 output neuron? Because a XOR operation requires 2 input bits and outputs a single bit. So we are assigning only the amount of needed bits, which is (2,1).

const xorDann = new Dann(2,1);

Setting up the calculations

Setting up the hidden layers

As you must have read in the previous tutorial, a hidden layer is essentially a neuron layer that can perform calculations. The name 'hidden' comes from the way that you don't have to see the values of every neuron, in contrast to the input/output layers. You can learn more about hidden layers & the basics surrounding it here.

Here we are setting up a hidden layer using the .addHiddenLayer method, which takes a argument of number of neurons assigned to the layer, and we are taking the neuron count to be 12. You can change the number way as you want, but I have found this to be the most stable.

xorDann.addHiddenLayer(12);

Now, to set up the hidden layers, we do

xorDann.makeWeights();

Testing the network

Testing is essential in anything, isn't it? Here we will test our network to see the stats of it, and detect any errors.

Since we have not trained it, we will simply being logging the details.

xorDann.log();

In my case, it outputs:

Dann NeuralNetwork:
  Layers:
    Input Layer:   2
    hidden Layer: 12  (sigmoid)
    output Layer: 1  (sigmoid)
  Other Values:
    Learning rate: 0.001
    Loss Function: mse
    Current Epoch: 0
    Latest Loss: 0

If it outputs the same thing, go on.
Comment out the line by prepending // in front of it.

Training the network

Our network doesn't know anything at all. Throw it a value and it will give a random bit.
But how do we train it?
We don't have a dataset!

You remember the Dannjs.xor we imported? it is a XOR dataset which has the training values to train the network about XOR.

So we should set a for loop to train, i.e, backPropagate the data to the network.

for (data of xor){
    xorDann.backpropagate(data.input, data.output);
}

Running the network now, feeding it 0,0 should output 0, should not it?
Lets feed it:

xorDann.feedForward([0,0],{log:true});

Let us run the network and see what happens:

Dann NeuralNetwork:
  Layers:
    Input Layer:   2
    hidden Layer: 12  (sigmoid)
    output Layer: 1  (sigmoid)
  Other Values:
    Learning rate: 0.001
    Loss Function: mse
    Current Epoch: 0
    Latest Loss: 0
Prediction:
[0.416897070979890]

The output can be different on yours. You will say, 0.4 is not even close to 0! Yes. You are right. We have trained this network only one time, and like a newborn child it will make mistakes. But why not train it, say 100000 times??
Lets train it:

for(let i = 0; i < 100000; i++){
for (data of xor){
    xorDann.backpropagate(data.input, data.output);
}}

Now let's run the network:

xorDann.feedForward([0,0],{log:true});

And in my case, the new output is:

Dann NeuralNetwork:
  Layers:
    Input Layer:   2
    hidden Layer: 12  (sigmoid)
    output Layer: 1  (sigmoid)
  Other Values:
    Learning rate: 0.001
    Loss Function: mse
    Current Epoch: 0
    Latest Loss: 0
Prediction:
[0.0224234234324]

After running it about 10 times, the output became:

Dann NeuralNetwork:
  Layers:
    Input Layer:   2
    hidden Layer: 12  (sigmoid)
    output Layer: 1  (sigmoid)
  Other Values:
    Learning rate: 0.001
    Loss Function: mse
    Current Epoch: 0
    Latest Loss: 0
Prediction:
[0.0044234234324]

Pretty close, right?

Finishing up

You can experiment with your own gate, and for your reference the XOR class is actually:

[
    { input: [ 1, 0 ], output: [1] },
    { input: [ 0, 1 ], output: [1] },
    { input: [ 1, 1 ], output: [0] },
    { input: [ 0, 0 ], output: [0] }
]

You can modify this gate and make your own gate!

The whole code used in this tutorial is:

const Dannjs = require('dannjs');
const Dann = Dannjs.dann;
const xor = Dannjs.xor; //datasource you can change it
const xorDann = new Dann(2,1);

xorDann.addHiddenLayer(12);
xorDann.makeWeights();

for(let i = 0; i < 100000; i++){
for (data of xor){
    xorDann.backpropagate(data.input, data.output);
}}

// running it

xorDann.feedForward([0,0],{log:true});

You can experiment with different values and see what you get!

Dann.js - Creating a neural network which learns and counts!

LeviAckerman3855 — Thu, 12 Aug 2021 07:21:26 +0000

Ever wanted to learn how huge neural networks work, and create one? Dann.js makes is as easy as ever!

Dann.js is a neural network library for JavaScript, which makes creating neural networks super simplified. It acts as a playground for experimentation with deep neural networks. It is also fast to implement, which makes it a great choice for various applications.

In this tutorial, we will learn how to set up Dann.js and teach a neural network how to count!

What you will need

Node.js
A computer with more than 2GB of ram and a good CPU

Getting started

Setup

To get Dann.js setup, we first install it as a NPM package.

So to install Dann.js, we do

npm i dannjs

This installs Dann.js as a package for Nodejs, so we can use it in our JavaScript.

Let's get started!

So open up your favorite editor (I personally recommend Atom), and create a new project. In the main JavaScript file, which is usually main.js or index.js (If there isn't one create it), and import Dann.js using require statements as follows:

const Dannjs = require('dannjs');
const Dann = Dannjs.dann;

This imports the dannjs module we installed into the constant Dannjs so we can refer to it later. This also initializes the Dann, the basic class of the Neural Network.

Creating the network

Now we have the Dannjs module imported, we should create our own neural network using instances of the Dann class.

Since in this project we are going to make a network which counts numbers in binary, we are going to create a basic network called CountingDan. You can name it whatever you want.

The basic syntax for creating a Neural Network instance is Dann(inputneurons,outputneurons). To start basic, we are giving our neural network 4 input neurons and 4 output neurons.

const CountingDan = Dann(4,4);

I've tracked down that 16 (4*4) neurons functioned good enough. You could try different things with this worth later.

Finishing the creation

We're not done at this point! We need to add some 'hidden' layers. A hidden layer is essentially a neuron layer that can perform calculations. The name 'hidden' comes from the way that you don't have to see the values of every neuron, in contrast to the input/output layers. You can learn more about hidden layers & the basics surrounding it here. We are also going to set the activation function to leakyReLU

countingDan.addHiddenLayer(16,'leakyReLU');
countingDan.lr = 0.01;

Finishing the creation of the counting network

Technically, we have finished the creation of the network countingDan. You can still experiment more, but this should be enough for our current project.

We should test out model by using the .log() method, which essentially displays information about our network, and by feeding the network some data to process:

countingDan.log();
countingDan.feedForward([0,0,1,0],{log:true});

The .feedForward() method takes a array, which it feeds to the input neurons of the network. Remember, we specified 4 input neurons for our network? So we are passing 4 binary bits in an array, one for each neuron. The log parameter specifies that it should tell every processing and log the output.

In my case, it outputted this:

Dann NeuralNetwork:
  Layers:
    Input Layer:   4
    hidden Layer: 16  (leakyReLU)
    output Layer: 4  (sigmoid)
  Other Values:
    Learning rate: 0.01
    Loss Function: mse
    Latest Loss: 0

Prediction:  [0.5398676080698,0.6730957170697,0.6748749672290,0.6377636387674]

It can be different in your case, as we never trained the model and it just gives some random results, as you can expect from a newborn baby!

Training the model

Setting up the dataset

To train the model, we’re going to need a dataset. Here is a lightweight JavaScript dataset for 4-bits binary counting. It basically looks like this:

const dataset4bit = [

    //...
    {
        input:[1,0,1,0],
        target:[1,0,1,1]
    },
    //  {
    //      input:[1,0,1,1],
    //      target:[1,1,0,0]
    //  },
    {
        input:[1,1,0,0],
        target:[1,1,0,1]
    },

    //...
];

You can download the dataset from this link

We can see that this dataset contains one number x in 4-bit binary, as the input value and the number x+1 in 4-bit binary as the target value. I commented out the element [1,0,1,1] so we can have a test sample the neural network has never seen. To access the data, we can copy the code included in the GitHub gist above and save it in binaryDataset.js in the same directory as our project. We can then require the file as a module:

const dataset = require('./binaryDataset.js').dataset;

Here we are importing the object dataset from the file binaryDataset.js.

We can now simply access the object data as:

dataset[i].input
dataset[i].target

as attributes.

Training the model

Now that we have access to the dataset let’s apply it by calling .backpropagate() method for each data point in our dataset array. This will tune the weights of the model according to the data you give it.

for (data of dataset4bit) {
    countingDan.backpropagate(data.input,data.target);
}

As we have the main dataset object defined as dataset4bit, we access it that way, and train the model for every bit of the dataset by calling .backpropagate() once for every bit.

Sadly, one epoch (One training pass) is not enough for any dataset to completely train, as you would expect from a child.

So we should train it multiple times, say 100000 times?
This will allow the network to train from every bit:

const epoch = 100000;
for (let e=0; e < epoch;e++) {
    for (data of dataset) {
        countingDan.backpropagate(data.input,data.target);
    }
}

Running the network

Now we have trained it enough, we should run the model!

countingNN.feedForward([1,0,1,1],{log:true});

This outputs:

Prediction:  [0.999884854,0.9699951248,0.020084607062,0.008207215405]

Which seems good enough, as it is pretty close to [1,1,0,0], which we wanted as an answer.

Finishing up

You can experiment with many datasets, and change the values as you like. The whole code used in this tutorial is:

const Dannjs = require('dannjs');
const Dann = Dannjs.dann;
const dataset = require('./binaryDataset.js').dataset;

const countingDan = new Dann(4,4);
countingDan.addHiddenLayer(16,'leakyReLU');

countingDan.lr = 0.01;

countingDan.feedForward([1,0,1,1],{log:true});
const epoch = 100000;
for (let e=0; e < epoch;e++) {
    for (data of dataset) {
        countingDan.backpropagate(data.input,data.target);
    }
}
countingDan.feedForward([1,0,1,1],{log:true});