Serve your data blazingly fast with Sqlite3 and Javascript

Aurélien Bottazini — Sun, 16 Oct 2022 18:03:05 +0000

Serve your data blazingly fast with Sqlite3 and Javascript

I love SQLite3. It is my de-facto database for personal projects. I have seen people import millions of rows per minute. I was stunned to see it used as an NOSQL Database. I adore that you can host SQLite databases everywhere. SQLite3 versatilety is amazing.

I wanted to explore the performance of SQLite3 when it is used with a web server. Indeed with 2022 Europe energy crisis It is vital to find cost efficient solutions.

I experimented with various programming languages: Go, Ruby, JavaScript, Rust.

My goal: get a feel for what is achievable concurrency wise.

I balanced speed with code complexity. I did not want to spend too much time on each implementation. The SQLite queries I used are simple. One insert statement, one query statement. This is not a test to verify SQLite performance. I know SQLite is fast. This is a test to see the blocking impact of SQLite queries on web servers.

The JavaScript results amazed me.

node:

With an sqlite insert: 16k requests per second
With an sqlite select: 31k requests per second

rust:

With an sqlite insert: 14k requests per second
With an sqlite select: 61k requests per second

rust is slower with inserts but twice as fast with queries.

bun.sh (a new javascript runtime):

With an sqlite insert: 21k requests per second
With an sqlite select: 75k requests per second

What? bun.sh is twice as fast on inserts and better on selects.
We live in a time when anyone can serve tens of thousands of requests per second from one server (the one I used is 3 years old).
You don't need a fancy database.
An sqlite3 database hosted on the same host is more than enough.

I remember hearing John Carmack say

The engineering under JavaScript is really pretty phenomenal

bun.sh is a new example of phenomenal engineering for JavaScript.
When you combine all the blazingly fast JavaScript optimisations with Sqlite3 you get performance that is on par with compiled languages.

I imagine I could get rust to be faster.
But that's going to require more work to figure out the implementation details.
bun.sh + SQLite3 are more than enough for all my performance needs.

References

John Carmack on Javascript: https://www.youtube.com/watch?v=rczu8kc8JZA
bun.sh
blazingly fast like ThePrimeagen
My test code: https://github.com/aurelienbottazini/sqlite-performance-test
Insert millions of rows per minute
SQLite as a NOSQL Database
Host SQLite on Github pages

DEV Community: Aurélien Bottazini

Serve your data blazingly fast with Sqlite3 and Javascript

Serve your data blazingly fast with Sqlite3 and Javascript

References