DEV Community: Pranev

Machine Learning in Deno: Single Layer Perceptrons

Pranev — Sun, 25 Feb 2024 12:03:29 +0000

Ever wished you could solve your regression and classification problems in JavaScript/TypeScript without having to rely on Neural Network libraries like Tensorflow.js?

Presenting La Classy, a single-layer perceptron library for Deno.

Classy provides a bunch of solvers (gradient descent, OLS, etc.), loss functions, activation functions, learning rate schedulers, and optimizers for gradient descent such as Adam and RMSProp. You can try various combinations of these depending on the problem.

Here's a quick example of how to classify the popular Iris dataset using Classy in Deno:

Import required libraries

First, we import the following libraries. You'll see use for them in later sections:

deno.land

import { parse } from "https://deno.land/std@0.217.0/csv/mod.ts";
import {
  ClassificationReport,
  Matrix,
  useSplit,
  CategoricalEncoder,
} from "https://deno.land/x/vectorizer@v0.5.0";
import {
  SagSolver,
  softmaxActivation,
  rmsPropOptimizer,
  crossEntropy,
} from "https://deno.land/x/classylala@v1.1.0/mod.ts";

You can also load them from JSR.

JSR

import { parse } from "jsr:@std/csv";
import {
  ClassificationReport,
  Matrix,
  useSplit,
  CategoricalEncoder,
} from "jsr:@lala/appraisal@0.5.0";
import {
  SagSolver,
  softmaxActivation,
  rmsPropOptimizer,
  crossEntropy,
} from "jsr:@lala/classy@1.1.0";

Load the Dataset

First we load our dataset. You can download the iris.csv file from here.

const data = parse(Deno.readTextFileSync("iris.csv"));

We then skip the first row of our dataset (the header).

data.shift();

Prepare input variables

Now that we have our data ready, we can separate the predictor and target variables from our dataset.

The first four columns of our dataset are the predictor variables and the fifth column indicates the iris species, our target variable.

const x = data.map((fl, i) => fl.slice(0, 4).map(Number));

Classy requires inputs to follow the following type:

type MaybeMatrix = {
    data: Float64Array,
    shape: [number, number]
}

However, our data is currently in the form of a 2D array. We can use the Matrix class from @lala/appraisal since it satisfies the type.

const X = new Matrix(x, {shape: [data.length], dType: "f64"})

In the meantime, let's also prepare the classes. We take just the fourth column from our dataset.

const y_pre = data.map((fl) => fl[4]);

Our y_pre will now be a 1D array with strings that denote the classes of each sample. We need to perform one-hot encoding on this data to use it for machine learning.

For one-hot encoding, we can use CategoricalEncoder from @lala/appraisal.

const encoder = new CategoricalEncoder()
encoder.fit(y_pre)

const y = encoder.transform(y_pre, "f64")

Now that we have both the predictors and the targets, let's take a small portion of the dataset out for testing accuracy.

const [[x_train, y_train], [x_test, y_test]] = useSplit(
  { ratio: [7, 3], shuffle: true },
  X,
  y
);
x_train.slice(0, 10)

We can now begin training our model.

Initialize the Model

In this example, we are using Stochastic Average Gradients with RMSProp as our optimizer.

const solver = new SagSolver({
  loss: crossEntropy(),
  activation: softmaxActivation(),
  optimizer: rmsPropOptimizer(4, 3),
});

crossEntropy is the standard loss function for multiclass problems.
softmax is an activation function used for multiclass problems.

Try using hinge as the loss function with linearActivation later.

With our solver ready, we can begin training the model.

Train the model

We can train our model using the train() method.

solver.train(x_train, y_train, {
  learning_rate: 0.01,
  epochs: 300,
  n_batches: 20,
  patience: 10
});

Let's break down each argument:

The first two arguments are the training input and output variables.
The learning_rate hyperparameter determines how large each step should be when updating parameters during training. If you are not using adamOptimizer or rmsPropOptimizer, set this to a very small value.
The epochs determine how many iterations the model should train for. This is the maximum number of iterations even if it doesn't converge.
n_batches is the number of minibatches. A large number will result in longer epochs.
patience determines how many epochs the model should continue learning if the loss starts increasing. If there is no improvement in loss within 10 epochs, the model will roll back 10 epochs and stop training.

Testing Metrics

Now that our model is trained, we can test its accuracy on the data we kept aside for testing.

const res = solver.predict(x_test)

We now have a Matrix with joint probabilities of classes for each sample. We can use the CategoricalEncoder again to convert them into one-hot representations.

// this method mutates the variable
CategoricalEncoder.fromSoftmax(res)

We can now convert the categorical variables into class labels.

// Predicted Values
const y_pred = encoder.untransform(res)
// Actual Values
const y_act = encoder.untransform(y_test)

The ClassificationReport class accepts predicted and actual values as parameters to create a classification report and provide metrics.

const report = new ClassificationReport(y_act, y_pred)

console.log(report)

At the time of writing this article, I gave the code a run and this was my classification report.

Class   Precision   F1Score Recall  Support
Iris-setosa 1   1   1   15
Iris-versicolor 1   1   1   13
Iris-virginica  0.85    0.92    1   17
Accuracy            0.98    45

You can try different combinations of loss functions and activation functions, try different hyperparameters, and try to solve different supervised learning problems using Classy.

Check out more examples at deno-ml!

I will make another article with a regression problem later. Have a great day and good luck on making your machine learn!

Extending the JavaScript Map

Pranev — Tue, 11 Apr 2023 09:14:03 +0000

This is meant to provide an understanding of how stuff work. This is not the "best" method to do any of the mentioned stuff.

A map is an abstract data type composed of key-value pairs. JavaScript comes with the built-in Map object that allows creating a map easily.

 const myMap = new Map<string, string>();
 map.set("foo", "bar");
 console.log(map.get("foo"));

Pretty easy, isn't it?

However, the methods available on a Map are very limited, namely:

Map.prototype.clear()
Map.prototype.delete()
Map.prototype.entries()
Map.prototype.forEach()
Map.prototype.get()
Map.prototype.has()
Map.prototype.keys()
Map.prototype.set()
Map.prototype.values()

In this article, you'll be guided to extend the Map object and add more methods for ease of use. The following are the list of methods we will be implementing:

find()
filter()
map()
reduce()

Now let's move on to implementing them.

Extending a Class

Classes in JavaScript can be extended using the extends keyword.

 class ExtendedMap extends Map {
    constructor() {}
 }

Since we are extending an existing class, we should follow up with a super() call.

 class ExtendedMap extends Map {
    constructor() {
        super();
    }
 }

Cool. Next comes adding custom methods.

Implementing Methods

 class ExtendedMap extends Map {
    constructor() {
        super();
    }
    foo() {
        console.log("bar")
    }
 }

You should now be able to access the extended map.

const myMap = new ExtendedMap();
myMap.foo();

bar

Map.find()

The find() method performs a linear search on the collection and returns the first element that passes our condition.

For this, we first convert our Map into an iterator. To do so, we use Map.prototype.entries(). We can then iterate over the result using a for..of loop.

 find() {
    for (const [k, v] of this.entries()) {

    }
 }

Our find() method needs one parameter, the test function. The function should return a truthy value in order to return an element.

 find(fn) {
    for (const [k, v] of this.entries()) {

    }
 }

Next is of course, running our test function on each entry in the map.

 find(fn) {
    for (const [k, v] of this.entries()) {
        if(fn(v, k)) return v;
    }
 }

There we have, our simple find() method. However, this method will return NOTHING if it didn't find a match. In that case, we can return a null
or undefined.

 find(fn) {
    for (const [k, v] of this.entries()) {
        if(fn(v, k)) return v;
    }
    return undefined;
 }

Map.filter()

The filter() method performs a linear search on the collection and returns all elements that pass our condition.

The working is the same as our find() method.

 filter(fn) {
    for (const [k, v] of this.entries()) {

    }
 }

The only change is that we are returning an array of elements now.

 filter(fn) {
    const res = [];
    for (const [k, v] of this.entries()) {
        if(fn(v, k)) res.push(v);
    }
    return res;
 }

Unlike find(), method will return an empty array even if it didn't find a match. We don't need to explicitly return a fallback.

Map.map()

map() is a bit different. It is a method which runs a function on every element in the collection and returns an array of results.

Starting from where we left our filter,

 map(fn) {
    const res = [];
    for (const [k, v] of this.entries()) {
        if(fn(v, k)) res.push(v);
    }
    return res;
 }

All we need to do is, remove the condition and push the function result instead.

 map(fn) {
    const res = [];
    for (const [k, v] of this.entries()) {
        res.push(fn(v, k));
    }
    return res;
 }

Easy, is it not? Now comes the final part of this guide.

Map.reduce()

The reduce() method accepts a reducer callback as a parameter and executes the callback on each element, with the previous iteration's result as a parameter. Like a chain.

Let's continue from where we left our map().

 reduce(fn) {
    const res = [];
    for (const [k, v] of this.entries()) {
        res.push(fn(v, k));
    }
    return res;
 }

We will need two parameters, one being the callback and the other being the initial value of our result.

 reduce(fn, first) {
    let res = first;
    for (const [k, v] of this.entries()) {
        res = fn(res, [k, v]);
    }
    return res;
 }

And that's it! We have implemented reduce() for our extended Map.

Our result

Check out the full source code with even more methods at retraigo/bettermap.

 class ExtendedMap extends Map {
    constructor() {
        super();
    }
    find(fn) {
        for (const [k, v] of this.entries()) {
            if(fn(v, k)) return v;
        }
        return undefined;
    }
    filter(fn) {
        const res = [];
        for (const [k, v] of this.entries()) {
            if(fn(v, k)) res.push(v);
        }
        return res;
    }
    map(fn) {
        const res = [];
        for (const [k, v] of this.entries()) {
            res.push(fn(v, k));
        }
        return res;
    }
    reduce(fn, first) {
        const res = first;
        for (const [k, v] of this.entries()) {
            res = fn(res, [k, v]);
        }
        return res;
    }
 }