DEV Community

Adam Crockett 🌀
Adam Crockett 🌀

Posted on

Perf question: How many processes in Node js?

I have this idea for a CMS (this is a technical exercise only). Traditional CMS's have modules / plugins which deliver a separated piece of functionality extending the base CMS's capabilities.

This got me thinking, what if I spawn a process per plugin in my node based prototype. I could use sigstop and sigcont to pause and resume processes on demand. Infact I have wrote a small API for this and it works! Fantastic, but what if in (my wildest dreams) my CMS turned into the next WordPress or Drupal? A user might add around 100 plugins I suppose. Could the average host handle all this?

What are the limits?

I could assume that some plugins are dependent on another main plugin and so these could run in the same process, that would reduce the amount of processes.

What do you think?

If I'm looking for benchmarks for node's nodeyness what would I Google?

Top comments (7)

Collapse
 
rhymes profile image
rhymes

You should probably measure it yourself, now that you have a prototype, benchmark it. Spawn 100 processes, measure how much more memory each process occupies (don't think startup is worth measuring, unless you notice it's remarkably slow).

Where are these processes/plugins going to run? On the server right?

If I understood correctly: is the use case a self hosted installation of a CMS that a group of users are going to use and each of them will install N plugins?

Collapse
 
adam_cyclones profile image
Adam Crockett 🌀

100% correct it would be self hosted with multiple users. I would need to devise the kind of load that a plugin would put on the system, but I can't think of anything realistic. This would be hosted on maybe a compute platform.

Collapse
 
rhymes profile image
rhymes

Let's say you have a 1000 users with 100 plugins each, 100_000 processes is a lot of processes and a lot of RAM. Or is 100 plugin/processes shared by users?

Even if most of the time they would probably be idle, it's still a lot of RAM.

Thread Thread
 
adam_cyclones profile image
Adam Crockett 🌀 • Edited

Definitely shared :). I could look at suspending to disk, docker is doing it in experimental builds.

Checkpointing via c library I think if I remember correctly.

Collapse
 
nektro profile image
Meghan (she/her)

This is a bad idea and an anti pattern in node world. Similar to goroutines in Go, extra processes in Node should be short lived and mostly handled in the background for extraneous tasks. It would be much better to have the plugins register themselves to an exposed init function, and go from there. Node is event based, so while I'm all ears to your approach, 95% of Node programs, even big ones, don't need to make any extra processes, manually that is.

Collapse
 
adam_cyclones profile image
Adam Crockett 🌀

Since we are talking hypothetically anyway, I appreciate your ears haha, here is what I'm trying to achieve.

Goals:

  • Application must feel seamless during enable disable functionality
  • Application must be secure meeting owasp top 10
  • Pause and resume execution of JavaScript. (or emulate it)
  • Hotswap and never shutdown, application, emulating a PHP based pluggable CMS.
  • Use plugins on demand as required and intelligently measure and hybernate plugins which are not in use.

I'm agile, I'm not doing it all at once, just laying some foundations.

By default all my plugin processes are suspended at startup, they will be short lived switching States as required.

Your refering to workers and fork when you say "not manually"?

Is my idea overkill?

How then could I pause plugins from running. Could a generator function toggle on and off.

Ps, I know very little about go.

Collapse
 
adam_cyclones profile image
Adam Crockett 🌀

I'm going to rethink this a bit. Thanks everyone!