I wanted a SQLite job queue that felt like a library

I run several small projects on one small server. For an application that needs to send emails or run a few background tasks, adding Redis means another service to configure, deploy, and keep running.

I wanted to install a package, give it a SQLite connection, and start a worker.

That became walq, a TypeScript job queue backed by SQLite.

A small API

import Database from 'better-sqlite3'
import { Queue } from '@walq/core'
import { betterSqlite3 } from '@walq/better-sqlite3'

const db = new Database('app.db')

const queue = new Queue<{ userId: number }>('emails', {
  storage: betterSqlite3(db),
})

queue.process(async ({ userId }) => {
  await sendWelcomeEmail(userId)
})

await queue.add({ userId: 42 })

The queue can share the application's SQLite database or use its own file. In either case, there is no separate queue server to deploy. Integration tests can use a temporary database and run the real worker.

More than a tasks table

Storing a task is straightforward. Making sure it survives a crashed worker, and keeping completed tasks from filling the database, takes more work.

These are some of the details walq handles:

Situation	What walq does
A worker crashes	Makes its tasks available again after their leases expire
A handler takes a long time	Renews its lease automatically
A handler fails	Supports configurable retries and backoff
Completed tasks accumulate	Removes them automatically in bounded batches
You need to investigate failures	Keeps the latest 100 failed tasks by default

Lease recovery means a task can run more than once. A worker might send an email and crash before recording completion, so handlers need to account for repeated execution.

Options where you need them

Priority, deduplication, and grouping are task options:

await queue.add(
  { userId: 42 },
  {
    priority: 10,
    dedupe: 'welcome:42',
    group: 'customer:7',
  },
)

Priority lets urgent work move ahead. A dedupe key prevents another task with the same key from being added to that queue until the existing task is removed.

Groups are useful when many independent customers share the same worker. If one customer queues 10,000 tasks, walq rotates between available groups using round-robin selection instead of letting that backlog monopolize the worker. By default, only one task per group can be active at a time.

Walq also supports delayed tasks, schedules, and batch processing.

Keeping SQLite responsive

Synchronous SQLite calls can block the application's event loop, even when the queue uses its own database file.

Walq splits claims into bounded transactions and yields to the event loop between them. Smaller transactions alone are not enough: if they run back-to-back, the application still waits for the entire sequence.

In a benchmark with 256 queues and 131,072 ready tasks:

Implementation	Longest blocking interval, p95	Tasks claimed/sec
Prototype: one large transaction	86 ms	53,100
Prototype: smaller transactions, no yielding	90 ms	51,500
Current adapter: smaller transactions with yielding	20 ms	50,600

The current adapter retained roughly 95% of claim throughput while giving the application more frequent opportunities to run other work.

The 20 ms figure is the p95 duration of an individual blocking interval, not the duration of the whole claim call. The complete claim call still took about 89 ms p95.

These results compare a prototype with the current adapter, whose scheduling logic also differs. They measure claims and event-loop turns, rather than handler throughput or HTTP latency.

Apple M1 Pro, Node 24, SQLite WAL/FULL, no competing writer. Figures are medians across five runs.

Cleanup follows the same principle: delete a bounded batch, then return control to the event loop.

Try it

If your application and workers run on one server, walq lets you add background processing with a package and a SQLite file.

SQLite still has a single writer. For workers spread across multiple machines, a separate queue service may be a better fit.

npm install @walq/core @walq/better-sqlite3 better-sqlite3

Start with the usage examples on GitHub.