DEV Community

Cover image for Your SQLite Queries Deserve Their Own Workers
Athreya aka Maneshwar
Athreya aka Maneshwar

Posted on

Your SQLite Queries Deserve Their Own Workers

Hello, I'm Maneshwar. I'm working on FreeDevTools online currently building **one place for all dev tools, cheat codes, and TLDRs* — a free, open-source hub where developers can quickly find and use tools without any hassle of searching all over the internet.*

SQLite is insanely fast and better-sqlite3 is one of the fastest Node.js bindings for it.

But there’s a catch:

better-sqlite3 is synchronous.
Meaning:
If your query is slow, your entire Node.js event loop freezes until the query finishes.

Most apps never notice this because SQLite is quick.

But if you're building an API, SSR site, search system, analytics dashboard, or anything with heavy SELECT / INSERT queries…

You can hit performance issues.

This is where Worker Threads come in.

This post explains:

  1. What problem worker threads solve
  2. How they work with better-sqlite3
  3. How to set up a thread pool
  4. Full working code
  5. When it’s worth using them

Let’s get into it.

Why better-sqlite3 blocks the event loop

better-sqlite3 runs queries like this:

const result = db.prepare("SELECT * FROM heavy_table").all();
Enter fullscreen mode Exit fullscreen mode

This line blocks everything else until it finishes.

If it takes 200ms, your server can’t respond to anything for 200ms.

With 5 concurrent users, you're dead.

Node is single-threaded by default — one slow query can freeze your whole app.

The solution? Can Be Worker Threads

Worker threads are real OS threads.
They run JavaScript in parallel with your main thread.

Think of them like:

  • Your main thread = server receptionist
  • Workers = kitchen cooks

The receptionist takes the order (SQL query), sends it to a cook (worker), and stays free to take more orders.

Workers run the blocking DB queries without freezing the server.

_- visual selection (3)

File: worker.js

This is the code that actually runs the SQL.

// worker.js
const { parentPort } = require('worker_threads');
const db = require('better-sqlite3')('foobar.db');

// Listen for work from the main thread
parentPort.on('message', ({ sql, parameters }) => {
  try {
    const result = db.prepare(sql).all(...parameters);
    parentPort.postMessage({ ok: true, result });
  } catch (err) {
    parentPort.postMessage({ ok: false, error: err.message });
  }
});
Enter fullscreen mode Exit fullscreen mode

The worker:

  • Opens its own SQLite connection
  • Runs the synchronous query
  • Sends results back

That’s it. Perfectly safe for reads and writes.

File: master.js — Thread Pool + Queue

This is the main logic your app will use.

// master.js
const { Worker } = require('worker_threads');
const os = require('os');

// Job queue
const queue = [];

// Public API
exports.asyncQuery = (sql, ...parameters) => {
  return new Promise((resolve, reject) => {
    queue.push({ resolve, reject, message: { sql, parameters } });
    drainQueue();
  });
};

let workers = [];

// Let each worker take a job if available
function drainQueue() {
  for (const worker of workers) {
    worker.takeWork();
  }
}

// Create worker pool = number of CPU cores
new Array(os.availableParallelism()).fill(null).forEach(function spawn() {
  const worker = new Worker('./worker.js');

  let job = null;
  let crashedError = null;

  function takeWork() {
    if (!job && queue.length) {
      job = queue.shift();
      worker.postMessage(job.message);
    }
  }

  worker
    .on('online', () => {
      workers.push({ takeWork });
      takeWork();
    })
    .on('message', ({ ok, result, error }) => {
      if (ok) job.resolve(result);
      else job.reject(error);
      job = null;
      takeWork();
    })
    .on('error', (err) => {
      crashedError = err;
    })
    .on('exit', (code) => {
      // Remove worker from pool
      workers = workers.filter(w => w.takeWork !== takeWork);

      // If a job was running, fail it
      if (job) job.reject(crashedError || new Error('Worker died'));

      // Respawn if crash
      if (code !== 0) {
        console.error(`Worker exited with ${code}, respawning…`);
        spawn();
      }
    });
});
Enter fullscreen mode Exit fullscreen mode

You now have a fully functional async SQLite engine, powered by worker threads.

This feels async, but behind the scenes, it uses a worker thread for the heavy lifting.

⚡Performance Benefits

  1. No more “event loop freeze”: Your server stays responsive.
  2. Queries run in parallel: If you have 8 CPU cores → you get 8 parallel SQLite queries.
  3. Writes are safe: Only one writer at a time (SQLite rule), but workers queue automatically.
  4. Massive throughput improvement: For heavy apps, throughput may jump from: 80 req/sec → 500+ req/sec (because the event loop is free)

Should you use worker threads?

Use them if:

✔ You run SQLite inside a server / API
✔ You have expensive SELECT queries
✔ You have 4+ CPU cores
✔ You’re running SSR with Astro/Next/Express and SQLite queries block HTML rendering

Don't bother if:

✖ Your queries are tiny
✖ You only run SQLite from a CLI script
✖ You never have concurrent users

Final Thoughts

better-sqlite3 is amazing, but it’s synchronous.
Worker threads let you keep the speed without freezing your server.

  • They’re safe
  • They’re fast
  • They scale with your CPU cores
  • They make heavy read/write workloads smooth

Once you add a worker pool, SQLite suddenly feels like using Postgres with connection pooling, except super lightweight and local.

FreeDevTools

👉 Check out: FreeDevTools

Any feedback or contributors are welcome!

It’s online, open-source, and ready for anyone to use.

⭐ Star it on GitHub: freedevtools

Top comments (0)