I wanted to do a performance comparison between Facil(a web framework for C) against some more conventional tools, such as Flask(Gunicorn server) and Nodejs. Let's dive in!
How the benchmarking was done:
- A simple webapp was made using each framework. These webapps embed the client's IP address in a page template and then serve it to the client.
- The 3 webapps were tested under light, medium and heavy load for 5 minutes each, running on a 1GB single core virtual machine(on Digital Ocean)
- They were evaluated on response time, CPU and memory consumption.
- In all of the plots below, the average value by Facil is shown as a dotted blue line for easier comparison.
TLDR;
- If you are developing a high load web application(over 1000 requests per second), writing it using Facil could definitely pay its dividends over time when it comes to resource consumption, response time(latency) and the variance of these two factors(they tend to stay more predictable, even under a sudden surge in load)
- For light load applications, Facil might be overkill(did not provide significant benefits) unless you are developing for an embedded platform or you prefer to develop in C compared to other languages
Heavy load (~1100 requests per second)
Response Time
Facil has come out on top by far when it comes to latency under heavy load. Facil managed to:
- Have less of a latency spike (it responded better to a sudden surge in load at the beginning of the test)
- Have much less variance in response time (more reliable results)
- Have a lesser average response time under heavy load
Resource Consumption
When it comes to resource consumption, Facil also managed to:
- Have lesser resource consumption in both ram and CPU, especially compared to Flask
- Have a lesser variance in resource consumption compared to other frameworks
Medium load (~400 requests/sec)
Response Time
Under medium load, Facil was:
- still leading ahead of Flask in terms of response time, but Nodejs is catching up(Facil was arguably a tiny bit better though).
- Facil still had a better response to sudden load in the beginning of the test than Nodejs
In terms of resource consumption, under medium load, Facil:
- Is still significantly leading in terms of RAM consumption ahead of both Flask and Nodejs
- Had less of a CPU spike under sudden load
- Had less variance in CPU consumption compared to the other two candidates
Light load (~100 requests/sec)
Under light load, Facil:
- Had seemingly no advantage in response time over the other frameworks
- Facil had some minimal advantage in RAM consumption compared to other frameworks
Final notes
Obviously this was just a simple test, but it has shown that if you are expecting large loads, want to keep hosting as cheap as possible by minimizing resource consumption, or doing development for embedded platforms, a non-conventional framework like Facil might be worth looking into.
You can find the source code for the 3 webapps under:
Top comments (29)
Except that:
For example hosting Ruby is definitely more expensive than most other languages, it still does not matter at all in the grand scheme of things
==> m.signalvnoise.com/only-15-of-the-...
Of course, I never intended this to be a be all end all benchmark(as I said in the final notes, it is rather simplistic), I just figured these results might be of some interest to other people.
Those are interesting points, especially about the costs of hosting with Ruby. Thanks for reading!
Fair enough.
I think benchmarks should be used by library and framework authors to improve their software.
As app developers, we should mostly care only about how good the API & documentation & community is.
There are also hobby projects where all those things dont matter much ;)
How do you manage to have heavy loads for your hobby projects, impressive π?
Hobby projects dont have heavy loads, i would say ;) They are just about having fun.
This is an intersting benchmark, although why did you limit the compiled solutions to only C?
I'd be very curious to see higher level compiled languages in these charts, like Rust and Haskell. They would give you the performance of C, but language features of languages such ash Python, thus reducing the pain associated with C programs.
That's a great idea for a next post - thanks for reading! :)
Agreed. I'd be particularly interested in facil vs rust, as they are more comparable (i.e. produce gc free, optimized binaries).
I love these graphics.
Thanks! The individual frames were made using matplotlib, then I generated the gifs using gifski(gif.ski/)
Very interesting post!. π
Nim lang is missing here, write as easy as Python, run fast as C.
Thanks for reading! :D Haven't used Nim lang before, but looking at some benchmarks it looks like a neat language
This comparison would be interesting to run again against; Netty and FastHTTP, these are known to be among the top performers.
github.com/valyala/fasthttp
netty.io
Great suggestion, I'll have to make some more in-depth benchmarks and give these frameworks a go. Thanks for reading!
Great article, it might be interesting to add other options that claim to be as performant as C but more friendlier and closer to dynamic languages (like javascript and python)
In that sense, I think a comparison with Crystal language would be great.
Great suggestion, thanks!
I've also heard a similar story from discord where they switched from Go to Rust for a specific service
blog.discord.com/why-discord-is-sw...
garbage collection seems to be a problem for high load applications as it causes spikes
Thats an interesting article, thanks!
You should add Rust to the comparison
I'll have to do a more detailed benchmark with all the new languages suggested - thanks for reading!
I'd love to see some comparison with Go as a web server. I've been pondering how it would operate vs Python FastApi
Thanks for the suggestion!
Worth it? Yes. But not on a bigger perspective. A C/C++ developer is expensive.