DEV Community: Alessandro Pio Ardizio

Poolifier as Node.js thread/cluster pool

Alessandro Pio Ardizio — Tue, 02 Mar 2021 07:29:07 +0000

Back in the days there was no chance to have a custom thread pool in a Node.js application.
From Node.js version 10 ( experimental ) a great module was provided, the worker_threads module.
Worker_threads module enables the use of threads that execute JavaScript in parallel.

The Beginning

Since that I am familiar with Java programming language, I immediately started to look for thread pool implementations just like are present in java.
There was only one small and not exhaustive thread pool at that time, so I started to implement poolifier.
I choosed to implement two thread pools, a fixed-size thread pool and a dynamic-size thread pool so that users could tune and choose which one fits better for their applications.

Growth of the poolifier project

Poolifier initially got some downloads and interest but when I really was not sharing to much about it, I found out that the number of downloads increased a lot, few months later users started to ask for new features and small fixes.

Community

Some contributions started to come into the project, in particular two people have being ( and still now are ) much much active to improve the project adding new features and enhancements.
By the way, special thanks to Shinigami92 and Jerome Benoit to help on the project so much !!!

Poolifier v2

All the commitment on the project resulted in a new fresh version with enhancements and new features.
New features
-Cluster pools( pools of processes) are now available
-New worker selection strategies available

Enhancements

Performance enhancements
Project re-written in typescript
Enhancement on internal code and variable names
Add static code analysis with Sonar, and changed coveralls in favor of Sonar, this also covers security
Production code minification for production to increase performances
Added unit test cases
Bug fixing

Benchmarks

While poolifier was growing, other people ( including some Node.js gurus ) implemented new thread pools, so currently there are more and more thread pool implementations.
Due to that we choosed to measure the performance on a CPU intensive task for each of the most used thread pools and those are our results.
We chose to use hyperfine because it allows to run isolated Node.js processes so that each pool does not impact each other.
External pools ( we choosed the most popular that we found ) with which we compared the poolifier results:

Those are our results running for each pool a CPU intensive task 100k times :

More info on how to run this benchmark on your own are available here.

Conclusion

So based on the benchmarks poolifier seems to be the fastest thread pool and well suited to run Node.js CPU intensive tasks and now it comes with fresh new features to try and an active community.
We will measure poolifier versus other pools with more and more task types ( that could fit better on processes pools instead of thread pools )going forward !!!
What you wait? Try poolifier! Leave a star, consider to contribute, ask a question !!!

Thanks to read this post, if you liked it please share.

Node js | Thread pool implementations

Alessandro Pio Ardizio — Thu, 23 Jan 2020 09:33:27 +0000

I develop two thread pool on top on worker_threads that you can use with an high level API https://github.com/pioardi/poolifier .

This project contain two thread pool implementations :

The first implementation is a static thread pool , with a defined number of threads that are started at creation time and will be reused.

The second implementation is a dynamic thread pool with a number of threads started at creation time ( these threads will be always active and reused) and other threads created when the load will increase ( with an upper limit, these threads will be reused when active ), the new created threads will be stopped after a configurable period of inactivity.

I need your feedbacks and I would like your contribution on my project :)

Nodejs | Thread pool implementations

Alessandro Pio Ardizio — Wed, 22 Jan 2020 23:04:17 +0000

Node Thread Pool ⏫ 🔛

Nodejs introduced worker_threads module to allow a better use of nodejs for CPU intensive tasks, but there is no official thread pool that allow to reuse threads.

So I develop two thread pool on top on worker_threads that you can use with an high level API https://github.com/pioardi/poolifier .

This project contain two thread pool implementations :

The first implementation is a static thread pool , with a defined number of threads that are started at creation time and will be reused.

The second implementation is a dynamic thread pool with a number of threads started at creation time ( these threads will be always active and reused) and other threads created when the load will increase ( with an upper limit, these threads will be reused when active ), the new created threads will be stopped after a configurable period of inactivity.

You have to implement your worker extending the ThreadWorker class

Installation

npm install poolifier --save

Usage

You can implement a worker in a simple way , extending the class ThreadWorker :

'use strict'
const { ThreadWorker } = require('poolifier')

function yourFunction (data) {
  // this will be executed in the worker thread,
  // the data will be received by using the execute method
  return { ok: 1 }
}

class MyWorker extends ThreadWorker {
  constructor () {
    super(yourFunction, { maxInactiveTime: 1000 * 60})
  }
}
module.exports = new MyWorker()

Instantiate your pool based on your needed :

'use strict'
const { FixedThreadPool, DynamicThreadPool } = require('poolifier')

// a fixed thread pool
const pool = new FixedThreadPool(15,
  './yourWorker.js',
  { errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker is online') })

// or a dynamic thread pool
const pool = new DynamicThreadPool(10, 100,
  './yourWorker.js',
  { errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker is online') })

pool.emitter.on('FullPool', () => console.log('Pool is full'))

// the execute method signature is the same for both implementations,
// so you can easy switch from one to another
pool.execute({}).then(res => {
  console.log(res)
}).catch ....

Considerations

Performance is one of the main target of these thread pool implementations, we want to have a strong focus on this.

We already have a bench folder where you can find some comparisons.
To choose your pool consider that with a FixedThreadPool or a DynamicThreadPool ( in this case is important the min parameter passed to the constructor) your application memory footprint will increase .

Increasing the memory footprint, your application will be ready to accept more CPU bound tasks, but during idle time your application will consume more memory.

One good choose from my point of view is to profile your application using Fixed/Dynamic thread pool , and to see your application metrics when you increase/decrease the num of threads.

For example you could keep the memory footprint low choosing a DynamicThreadPool with 5 threads, and allow to create new threads until 50/100 when needed, this is the advantage to use the DynamicThreadPool.

But in general , always profile your application

Contribute

Pull requests, contributors and feedbacks are welcome , join the project :)

Open source project feedbacks

Alessandro Pio Ardizio — Sat, 04 Jan 2020 19:01:07 +0000

Could you please give me some feedback on this project https://github.com/pioardi/ring-election ?

I would like people to build something great on top of this !

How to build high available/fault-tolerant services in node.js

Alessandro Pio Ardizio — Fri, 03 Jan 2020 10:41:30 +0000

How to build high available/fault-tolerant services in node.js

During my job for an important client , I was thinking about high availability and recovery NFRs , our tech stack included cassandra and kafka , two distributed systems of which I studied internal behavior.
Kafka used Zookeeper to keep track of assigned partitions to each consumer , Cassandra had a gossip algorithm between nodes and divides data in partition ranges .
So I was starting to think if there was any library ( not an external service like zookeeper ) that had an algorithm with gossip implemented so that people could build some new distributed systems more easily.
That library does not exist , and then I created ring-election .

You can integrate ring-election into your node process and you will have some important NFRs already constructed !!!

What the ring-election driver offers you ?

A default partitioner that for an object returns the partition to which it is assigned.
Mechanism of leader election
Failure detection between nodes.
Assignment and rebalancing of partitions between nodes
Automatic re-election of the leader
Listen for new assigned/revoked partitions

What problems can you solve with this driver ?

Scalability
High availability
Concurrency between nodes in a cluster
Automatic failover

How it works under the hood

Terminology

Leader , the node that will handle the cluster and will not have assigned partitions
Follower , a node that will have assigned partitions and will work on them
Heartbeat , a message sent periodically from the followers to leader node to keep track that is alive.
Heartcheck , a process that run on the leader and go to check the last heartbeat received by each follower
Priority , is assigned to each follower based on the time that they joined the cluster , when a node die the priority is decreased by one . If the leader die the node with lower priority will become the leader
Node id , each follower node has an assigned id that is unique into the cluster

Start up phase described

Detect follower failures ( Heartbeat/Heartcheck )

Leader Failure

How to integrate it ?

Join https://github.com/pioardi/ring-election to have more info .
If you want to suggest new features or you want help to integrate ring-election open an issue on github and I will be happy to help you.