Is the pain worth the gain? Writing webapps in C (Benchmarks vs Flask and Nodejs)

I wanted to do a performance comparison between Facil(a web framework for C) against some more conventional tools, such as Flask(Gunicorn server) and Nodejs. Let's dive in!

How the benchmarking was done:

• A simple webapp was made using each framework. These webapps embed the client's IP address in a page template and then serve it to the client.
• The 3 webapps were tested under light, medium and heavy load for 5 minutes each, running on a 1GB single core virtual machine(on Digital Ocean)
• They were evaluated on response time, CPU and memory consumption.
• In all of the plots below, the average value by Facil is shown as a dotted blue line for easier comparison.

TLDR;

• If you are developing a high load web application(over 1000 requests per second), writing it using Facil could definitely pay its dividends over time when it comes to resource consumption, response time(latency) and the variance of these two factors(they tend to stay more predictable, even under a sudden surge in load)
• For light load applications, Facil might be overkill(did not provide significant benefits) unless you are developing for an embedded platform or you prefer to develop in C compared to other languages

Heavy load (~1100 requests per second)

Response Time

Facil has come out on top by far when it comes to latency under heavy load. Facil managed to:

• Have less of a latency spike (it responded better to a sudden surge in load at the beginning of the test)
• Have much less variance in response time (more reliable results)
• Have a lesser average response time under heavy load

Resource Consumption

When it comes to resource consumption, Facil also managed to:

• Have lesser resource consumption in both ram and CPU, especially compared to Flask
• Have a lesser variance in resource consumption compared to other frameworks

Medium load (~400 requests/sec)

Response Time

Under medium load, Facil was:

• still leading ahead of Flask in terms of response time, but Nodejs is catching up(Facil was arguably a tiny bit better though).
• Facil still had a better response to sudden load in the beginning of the test than Nodejs

In terms of resource consumption, under medium load, Facil:

• Is still significantly leading in terms of RAM consumption ahead of both Flask and Nodejs
• Had less of a CPU spike under sudden load
• Had less variance in CPU consumption compared to the other two candidates

Light load (~100 requests/sec)

Under light load, Facil:

• Had seemingly no advantage in response time over the other frameworks

• Facil had some minimal advantage in RAM consumption compared to other frameworks

Final notes

Obviously this was just a simple test, but it has shown that if you are expecting large loads, want to keep hosting as cheap as possible by minimizing resource consumption, or doing development for embedded platforms, a non-conventional framework like Facil might be worth looking into.

You can find the source code for the 3 webapps under:

wickdChromosome / simple_webapp_comparison

Repo for article comparing writing webapps in C, Flask, Nodejs

Comments

[Jean-Michel 🕵🏻‍♂️ Fayard]

Except that:

• we have no idea whether those benchmarks would be representative of our actual bottlenecks - if anything, i would assume they aren't
• security issues that comes with writing in C probably overcome any benefit
• the cost of hosting is almost always trivial compared to the salary of good developers, so you are probably optimizing the wrong thing if switching to a lower level makes your team 2% or more less productive & happy

For example hosting Ruby is definitely more expensive than most other languages, it still does not matter at all in the grand scheme of things

==> m.signalvnoise.com/only-15-of-the-...

[Bence Kotis]

Of course, I never intended this to be a be all end all benchmark(as I said in the final notes, it is rather simplistic), I just figured these results might be of some interest to other people.
Those are interesting points, especially about the costs of hosting with Ruby. Thanks for reading!

[Jean-Michel 🕵🏻‍♂️ Fayard]

Fair enough.
I think benchmarks should be used by library and framework authors to improve their software.
As app developers, we should mostly care only about how good the API & documentation & community is.

[Paweł Kowalski]

There are also hobby projects where all those things dont matter much ;)

[Jean-Michel 🕵🏻‍♂️ Fayard]

How do you manage to have heavy loads for your hobby projects, impressive 😛?

[Paweł Kowalski]

Hobby projects dont have heavy loads, i would say ;) They are just about having fun.

[Valts Liepiņš]

This is an intersting benchmark, although why did you limit the compiled solutions to only C?

I'd be very curious to see higher level compiled languages in these charts, like Rust and Haskell. They would give you the performance of C, but language features of languages such ash Python, thus reducing the pain associated with C programs.

[Bence Kotis]

That's a great idea for a next post - thanks for reading! :)

[Shaun Smiley]

Agreed. I'd be particularly interested in facil vs rust, as they are more comparable (i.e. produce gc free, optimized binaries).

[Ben Halpern]

I love these graphics.

[Bence Kotis]

Thanks! The individual frames were made using matplotlib, then I generated the gifs using gifski(gif.ski/)

[Juan Carlos]

Very interesting post!. 😃

Nim lang is missing here, write as easy as Python, run fast as C.

[Bence Kotis]

Thanks for reading! :D Haven't used Nim lang before, but looking at some benchmarks it looks like a neat language

[Rolf Streefkerk]

This comparison would be interesting to run again against; Netty and FastHTTP, these are known to be among the top performers.

github.com/valyala/fasthttp
netty.io

[Bence Kotis]

Great suggestion, I'll have to make some more in-depth benchmarks and give these frameworks a go. Thanks for reading!

[opensas]

Great article, it might be interesting to add other options that claim to be as performant as C but more friendlier and closer to dynamic languages (like javascript and python)
In that sense, I think a comparison with Crystal language would be great.

[Bence Kotis]

Great suggestion, thanks!

[Raymond]

I've also heard a similar story from discord where they switched from Go to Rust for a specific service
blog.discord.com/why-discord-is-sw...
garbage collection seems to be a problem for high load applications as it causes spikes

[Bence Kotis]

Thats an interesting article, thanks!

[Zizaco]

You should add Rust to the comparison

[Bence Kotis]

I'll have to do a more detailed benchmark with all the new languages suggested - thanks for reading!

[Rommel Paras]

I'd love to see some comparison with Go as a web server. I've been pondering how it would operate vs Python FastApi

[Bence Kotis]

Thanks for the suggestion!

[Viet Vu]

Worth it? Yes. But not on a bigger perspective. A C/C++ developer is expensive.

[lakhara]

Hey Bence, very interesting article! I was wondering what tools have you used for benchmarking. Can you help me with that ?

[Bence Kotis]

Hey Lakhara, thanks for reading! I used ApacheBench to create the load and measure the latency, and used top in batch mode to measure the CPU and RAM%.

[Abdisalan]

Now the question is... What do you consider pain?

[Jonathan Boudreau]

I think for your benchmark to get a bit closer to a real world example, it would need a database call at the very least. When it comes to webapps, this is usually the bottleneck - not the application.

[Bence Kotis]

Great suggestion, thanks!