DEV Community: Fabien B.

Software doesn't age. It stops adapting.

Fabien B. — Sun, 26 Jul 2026 15:08:29 +0000

A deep-dive into the mechanisms that make software progressively harder to evolve.

Introduction

An application written ten years ago still works.

Users rely on it every day. The servers run. The tests pass. On the surface, nothing looks wrong.

And yet, no one dares touch it anymore.

Adding a feature that seemed simple takes several weeks. Fixing one bug reveals two more. Every change requires understanding mechanisms that no one on the team truly masters anymore. Developers end up working around the problem instead of modifying the code, afraid of breaking a balance that has become fragile.

The software still works, but it seems to have lost something essential: its ability to evolve.

We often talk about "legacy," technical debt, or obsolescence. These terms describe the symptoms well, but they don't answer a more fundamental question:

Why can a piece of software that never changes become increasingly difficult to evolve?

After all, the source code is the same as yesterday. The algorithms haven't changed overnight. The files are still there, line for line.

What has changed, on the other hand, is everything around this software.

Business needs evolve. Technologies advance. Teams turn over. Tools change. Knowledge gets lost. User expectations rise. The ecosystem keeps moving forward while the application stays still.

This might be the real paradox of software: it's not the code that ages, it's its fit with its environment that progressively degrades.

This distinction matters, because it changes how we approach modernization. If we think software ages simply because it's old, the only answer seems to be a rewrite. But if we understand that its real problem is an inability to adapt, a different question emerges:

How do we design software that can keep evolving, even as everything around it changes?

That's the idea I want to explore in this article.

The code doesn't age. Its environment does.

Unlike a car that wears out from use, or food that degrades over time, source code is remarkably stable.

An application left untouched in a drawer for ten years will be exactly the same when you reopen it. The files won't have changed a single line. The algorithms will still produce the same results. The original compiler will probably still be able to produce the same binary.

In that sense, code doesn't age.

And yet, we've all opened an old project with that strange feeling that it belongs to another era. Not because it no longer works, but because everything around it has changed.

Operating systems evolve. Browsers change. Frameworks disappear. Libraries get security patches. Languages add new concepts. Business needs shift. Teams turn over. Engineering practices move forward.

The software, meanwhile, stays frozen.

It's precisely this gap that creates aging.

Software doesn't become harder to maintain because it's old. It becomes harder to maintain because the gap between its original choices and its current environment keeps growing.

This gap can take several forms. Some are purely technical. Others are organizational or human. Others still are directly tied to the business the software is meant to serve.

Understanding these different forms of aging matters. Without it, we risk treating the symptoms — upgrading a framework, rewriting a technical layer, or switching languages — without ever addressing the real cause of the problem.

In my view, software can age in six different ways.

The six forces that push software toward obsolescence

We often talk about software aging as if it were a single, inevitable phenomenon — as if software became "old" simply because time passes.

I don't think that's the case.

Time isn't the real culprit. It's only a revealer.

What ages software is the many forces acting on it over the years. Each one creates a gap between the assumptions made at design time and today's reality.

The bigger this gap grows, the harder the software becomes to understand, maintain, and evolve.

These forces aren't all technical. Some come from the business, others from the team, others still from the organization itself. This is actually what makes modernization so complex: upgrading a framework doesn't solve a business problem, and a new architecture doesn't replace lost knowledge.

In my view, there are six major forces that progressively push software toward obsolescence.

1. Technical pressure

Languages evolve.

Frameworks change.

Libraries patch security flaws.

Operating systems, browsers, and platforms keep evolving.

Software that is never maintained inevitably drifts away from its ecosystem.

2. Business pressure

Software was designed to meet a company's needs... at a given point in time.

But a company evolves.

Its processes change, so do its products, and so do its customers' expectations.

When the software's business model stops evolving at the same pace as the company's, every new feature becomes an exercise in contortion.

3. Architectural pressure

An architecture is always an answer to a context.

An application designed for five developers may no longer fit a team of fifty.

Conversely, some applications stay surprisingly simple even though they inherited an architecture built for needs that never actually materialized.

A good architecture isn't universal; it's suited to a given situation. When that situation changes, the architecture can become a constraint.

4. Human pressure

Teams change.

Developers leave.

Tacit knowledge disappears.

New hires never lived through the decisions that shaped the software.

A system that no one truly understands always ends up hard to evolve, even if its code is high quality.

5. Organizational pressure

The company grows.

Responsibilities shift.

Teams reorganize.

New regulatory constraints appear.

The software now has to meet needs that didn't exist when it was designed.

An architecture that worked perfectly for a small team can become a bottleneck in a much larger organization.

6. The pressure of ideas

This one is probably the quietest, but also one of the most important.

The way we build software keeps changing.

New practices emerge. Some disappear. Others become standards.

Developers joining a project today no longer share the same references as those who designed it fifteen years ago.

It's not just the technology that changes: it's also how we think about software.

A project built on concepts that have become rare or forgotten becomes progressively harder to understand — not because it's bad, but because the collective culture has moved on.

These six forces are always at work.

None of them alone is responsible for software aging.

It's their accumulation that, year after year, gradually pulls an application away from its environment.

And that's precisely why modernization can't be reduced to a simple technical upgrade. It's fundamentally about closing that gap so the software regains its ability to evolve.

Aging or technical debt?

At this point, one question often comes up:

Isn't this simply technical debt?

I don't think so.

Technical debt is an extremely useful concept, but it's often used to describe every problem found in an existing application. Over time, it becomes a universal explanation that no longer really helps us understand what's happening.

To me, software aging is a broader phenomenon.

Technical debt is one of its possible consequences, but it's neither its only cause nor its only manifestation.

Take an example.

An application built fifteen years ago may have been designed with great care. Clear code, well organized, tests in place, technical choices perfectly suited to the time.

Yet fifteen years later, it may run on an abandoned framework, serve a business model that no longer matches the company's activity, or rely on concepts that new developers no longer encounter anywhere else.

This application has aged.

But it would be hard to say it carries technical debt in the traditional sense.

Conversely, an application built only a year ago can already carry considerable technical debt.

Premature abstractions.

Circular dependencies.

Useless layers.

An architecture copied without reflection.

Contradictory conventions.

The software is recent.

Yet every new feature is already harder to integrate.

It hasn't had time to age.

It has, however, accumulated debt.

That's why I prefer to keep these two notions separate.

Aging describes the gap that progressively widens between software and its environment.

Technical debt refers to technical choices whose cost has been deferred to the future.

The two phenomena often reinforce each other.

Significant technical debt accelerates aging, because it makes updates, refactoring, and changes more costly.

Conversely, software that ages without maintenance almost always ends up accumulating new debt: workarounds, temporary fixes, code duplication, or successive patches.

But conflating the two often leads to bad decisions.

If we think everything is technical debt, the answer is usually to "clean up the code."

Yet sometimes the real problem isn't the code itself — it's that the application no longer fits the context it operates in.

In that case, modernization isn't just about improving code quality.

It's mainly about closing the gap between the software and the world around it, so it regains its ability to evolve.

Aging or maturity?

At this point, one might think that old software is necessarily software nearing the end of its life.

That would be a mistake.

The age of a piece of software ultimately tells us very little about its health.

Some decades-old projects remain references. They keep evolving, keep adding features, adapt to new use cases, and remain pleasant to maintain.

Conversely, some applications built only a few years ago are already considered "legacy" systems.

The difference isn't their age.

It's their ability to evolve.

I like to draw a distinction between software that ages and software that matures.

Software that ages accumulates gaps with its environment. Every change becomes harder than the last. Yesterday's decisions progressively become constraints. The team spends more and more energy working around the system's limitations instead of addressing business needs.

Mature software, on the other hand, keeps evolving alongside its environment.

Its architecture isn't frozen. It's regularly adapted. Dependencies are kept up to date. Concepts get questioned when the context changes. The code is simplified as much as it's enriched.

In other words, mature software isn't software that no longer changes.

It's software that has learned how to change.

This distinction matters, because it changes how we evaluate a system.

Age isn't a relevant indicator.

What matters is how fast the software can evolve without degrading.

Can you add a feature without triggering a cascade of regressions?

Can you upgrade a dependency without freezing the team for several weeks?

Do new developers quickly understand how the system works?

Do the architectural choices still match today's needs?

These questions say far more about a system's health than its creation date.

That's probably why some projects go through several decades without ever becoming a burden, while others already feel like they belong to the past after just a few years.

Maturity isn't the absence of change.

It's the ability to absorb change without losing coherence.

How to slow down software aging

If software aging is inevitable, that doesn't mean we're condemned to just endure it.

Just as it's impossible to stop a building from wearing down, it's impossible to freeze software in some ideal state. What is entirely possible, however, is to slow its aging and preserve its ability to evolve.

This takes less in the way of technical recipes than discipline and self-questioning.

Accept that software is never finished

One of the most common mistakes is treating a production release as the end of the project.

In reality, it's only the beginning.

Once people start using the software, they discover new needs. They suggest improvements. They question certain decisions. Sometimes they use the application in ways no one ever imagined.

Sponsors often follow the same path. A first version answers the initial need, but once the first benefits show up, new ideas naturally appear.

This is especially visible when software automates part of the work. The time saved lets teams think about other improvements, imagine new processes or new services. Effective software often creates its own follow-on demand.

On top of that, there's market change, regulatory constraints, organizational shifts, or new user expectations.

Software that delivers value is software people will always ask more of.

In the end, good software is a bit like a competent employee: the more effective it is, the more responsibility it gets handed.

Designing software isn't about building a finished product. It's about building a system able to absorb changes you don't know about yet.

Prepare the ground before adding a new feature

When a new need arises, it's tempting to reach for the fastest possible solution.

After all, you just have to "add a feature."

But that view is often misleading.

Every new feature shifts the system's balance. It interacts with existing behaviors, introduces new rules, and increases the number of cases to account for.

Stack enough of these changes without ever revisiting what already exists, and the software ends up looking like a game of Jenga. Each new piece seems to hold, but every addition makes the whole thing a little more fragile.

Sometimes it's worth spending a bit of time preparing the ground before starting development.

That doesn't mean rewriting the application or launching a massive refactoring effort.

It means adapting the foundation so the new feature naturally fits into the existing architecture.

Good development isn't just about making a new feature work. It's also about preserving the coherence of the whole.

Preserve simplicity

Every new feature increases a piece of software's complexity.

So the question isn't whether to add complexity, but how to keep it from piling up faster than the value it produces.

Every extra layer is one more contract to maintain.

Every dependency becomes an added responsibility.

Every abstraction will eventually need to be understood, explained, and probably adapted.

Simplicity isn't a pursuit of elegance.

It's a strategy.

A simple system is easier to understand, easier to change, and easier to reorganize when the context shifts.

Conversely, an unnecessarily complex system progressively becomes rigid. Developers hesitate to touch the code, changes take longer, and every change feels riskier than the last.

Preserving simplicity isn't about writing as little code as possible.

It's about keeping only the complexity that's actually necessary.

Regularly question past decisions

The best technical decisions are always made within a given context.

An architecture suited to a small team won't necessarily still make sense once the project has dozens of developers.

A data model suited to a few thousand users may no longer serve the needs of several million.

Business needs change.

Technologies evolve.

Regulatory constraints appear.

The company itself transforms.

Given all that, it would be surprising if every decision made five or ten years ago were still the best one today.

Questioning a past decision doesn't mean it was wrong.

It simply means the context has changed.

A good architecture isn't one that resists change.

It's one that accepts being questioned when change becomes necessary.

Remove as much as you add

Development usually means adding.

Adding a feature.

Adding a business rule.

Adding a class.

Adding a service.

Yet the best software also evolves because it knows how to remove things.

New features sometimes make parts of the system unnecessary. Two mechanisms end up doing the same thing. An old business rule no longer has a reason to exist. An abstraction created a few years ago no longer serves any purpose today.

If nothing is ever removed, the software keeps growing without ever getting simpler.

Execution paths multiply.

Exceptions pile up.

Duplicates appear.

The system becomes harder to understand — not because it does more, but because it keeps every trace of its history.

Every new change should be an opportunity to ask not just what needs to be added, but also what can disappear.

Removing code is often one of the best ways to slow down an application's aging.

Document the decisions, not the code

We often hear that good code should be clear enough to not need documentation.

I agree — up to a point.

Well-written code usually explains how the software works.

It rarely explains why it works that way.

Why this architecture?

Why this naming convention?

Why this business vocabulary?

Why this constraint?

Why this dependency rather than another?

These decisions aren't visible in the code.

Yet they deserve to be preserved, because they're what let future developers understand the choices that were made.

Good documentation doesn't try to describe every method or every class.

It captures the project's most stable elements: business concepts, team conventions, architectural decisions, and their rationale.

And above all, it has to live alongside the project.

Documentation versioned with the code stands a real chance of evolving with it.

A hundred-and-twenty-page Word document buried on SharePoint, on the other hand, almost always ends up as an archive no one ever opens again.

Slowing software aging isn't about preventing change.

It's exactly the opposite.

You have to accept that change is permanent, and design the software so it can absorb that change without losing coherence.

Sometimes that means investing a bit more time today to save a lot of time tomorrow.

Because the real cost of a feature isn't the cost of building it.

It's the cost it will impose on every future change.

Conclusion

We often talk about "old" software, "legacy" systems, or technical debt, as if time were the main culprit behind their condition.

Yet time does nothing on its own.

It doesn't change the code. It doesn't add complexity. It doesn't create dependencies. It doesn't erase knowledge.

It's the changes happening all around it that progressively create a gap between the software and its environment.

It's this gap that makes changes more expensive, decisions harder, and teams more hesitant.

Software that keeps adapting, on the other hand, can stay relevant for decades. Its architecture evolves, its code gets simpler, its concepts get questioned when necessary. It doesn't try to resist change; it learns to live with it.

This might be the real role of software engineering.

We're not building software meant to stay still.

We're building software meant to evolve.

And if I had to sum up this article in a single sentence, it would be this one:

Software doesn't age. It stops adapting.

Everything else — architecture, refactoring, simplicity, documentation, even technical debt — ultimately serves a single purpose:

Preserving, year after year, a piece of software's ability to change.

Build an Electron app with local data storage — no SQLite bindings, no server

Fabien B. — Wed, 10 Jun 2026 22:36:47 +0000

If you've shipped an Electron app that persists structured data, you've probably hit the wall at some point. You need something more than a flat JSON file, but a full database server is absurd for a desktop app. So you reach for better-sqlite3 — and then spend the next hour fighting native bindings.

The Electron storage problem

Electron bundles its own version of Node.js. That version almost never matches the one on your machine. Native Node modules — the ones that compile C++ bindings — need to be recompiled specifically for the Electron runtime, using electron-rebuild or equivalent tooling.

In practice this means:

Every Electron update potentially breaks your native dependencies
CI pipelines need platform-specific build steps for Windows, macOS and Linux
Code signing and notarization on macOS gets complicated by native binaries
End users on some machines encounter install failures you can't reproduce locally

better-sqlite3 is an excellent library. But it is a native module, and native modules in Electron are a tax you pay forever.

The alternatives people reach for have their own problems. lowdb and lokijs keep everything in memory and flush to disk — fine for small datasets, but you lose durability on crash and memory usage grows with your data. nedb hasn't been maintained in years. A remote database is obviously wrong for a local desktop app.

What you actually need

For most Electron apps, the storage requirements are modest but specific:

Structured documents, not just key-value pairs
Queries that go beyond "give me everything"
Writes that survive a crash
Zero native binaries — pure JavaScript, works everywhere Electron runs
No server process running alongside your app

That's exactly the gap pocket-db is designed to fill. It's a single-file embedded document store — the SQLite model applied to JSON documents, with a MongoDB-style API. Pure TypeScript, shipped as both ESM and CommonJS. No native bindings. No optional dependencies.

Building the example app

Let's build a small note-taking app: create, list, search and delete notes. Simple enough to follow in full, realistic enough to show the pattern.

Setup

We'll use electron-vite, which scaffolds the src/main / src/preload / src/renderer layout used below, with TypeScript and React ready to go:

npm create @quick-start/electron@latest my-notes -- --template react-ts
cd my-notes
npm install @axfab/pocket-db

No electron-rebuild. No postinstall scripts. That's the point.

Initialising the database

In Electron, database access belongs in the main process. You have access to the filesystem there, and it keeps your renderer process clean. Use app.getPath('userData') — Electron's standard location for app data, persisted across updates.

One thing to know: Electron creates the userData directory lazily, so on a first launch it may not exist yet — and open() does not create parent directories. Create it before opening:

// src/main/db.ts
import { app } from 'electron'
import { open } from '@axfab/pocket-db'
import { mkdirSync } from 'node:fs'
import path from 'node:path'

const dataDir = app.getPath('userData')
mkdirSync(dataDir, { recursive: true }) // may not exist on first launch

export const db = open({ path: path.join(dataDir, 'notes.pdb') })
export const notes = db.collection('notes')

One file, one call. The database is created if it doesn't exist. No migrations, no schema definition.

Exposing database operations via IPC

Electron's security model means the renderer can't touch Node APIs directly. You expose operations through IPC handlers in the main process and call them via contextBridge in the preload script.

Search deserves a moment of care: the search term comes from user input, and $regex compiles it as a regular expression. A term containing ( or [ would throw, and a raw pattern is case-sensitive. Escape the input and pass $options: 'i':

// src/main/index.ts
import { app, BrowserWindow, ipcMain } from 'electron'
import { db, notes } from './db'

/** Escapes regex metacharacters so user input is matched literally. */
const escapeRegex = (term: string) =>
  term.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')

ipcMain.handle('notes:create', (_event, title: string, body: string) => {
  return notes.insertOne({ title, body, createdAt: Date.now() })
})

ipcMain.handle('notes:list', () => {
  return notes.find().sort({ createdAt: -1 }).toArray()
})

ipcMain.handle('notes:search', (_event, term: string) => {
  const pattern = escapeRegex(term)
  return notes.find({
    $or: [
      { title: { $regex: pattern, $options: 'i' } },
      { body:  { $regex: pattern, $options: 'i' } }
    ]
  }).toArray()
})

ipcMain.handle('notes:delete', (_event, id: string) => {
  return notes.deleteOne(id)
})

app.on('before-quit', () => db.close())

// src/preload/index.ts
// Note: keep the preload compiled to CommonJS (electron-vite's default) —
// sandboxed renderers don't load ESM preload scripts.
import { contextBridge, ipcRenderer } from 'electron'

contextBridge.exposeInMainWorld('notes', {
  create:  (title: string, body: string) => ipcRenderer.invoke('notes:create', title, body),
  list:    ()           => ipcRenderer.invoke('notes:list'),
  search:  (term: string) => ipcRenderer.invoke('notes:search', term),
  delete:  (id: string) => ipcRenderer.invoke('notes:delete', id),
})

Using it in the renderer

From the renderer, the API is just async function calls:

// src/renderer/src/App.tsx
const handleCreate = async () => {
  await window.notes.create(title, body)
  setNotes(await window.notes.list())
}

const handleSearch = async (term: string) => {
  const results = await window.notes.search(term)
  setNotes(results)
}

const handleDelete = async (id: string) => {
  await window.notes.delete(id)
  setNotes(await window.notes.list())
}

No database logic in the renderer. No connection strings. No async initialisation to wait for. The main process owns the database, the renderer just calls handlers.

Where indexes help — and where they don't

Two things to know about pocket-db's query planner in V1: $regex is never index-assisted (it's always evaluated against the documents), and $or queries bypass index planning entirely. So the regex search above is a full collection scan — perfectly fine for a notes app, but don't expect an index to speed it up.

Indexes shine on equality and range lookups. Say notes belong to notebooks:

// run once at startup, safe to call repeatedly
notes.createIndex('notebook', { type: 'string' })

Now a query like notes.find({ notebook: 'work' }) reads only the matching documents instead of scanning the collection — the planner picks the index up automatically, no change needed in your query code. The same goes for a number index on createdAt if you query by date range.

Periodic compaction

The append-only log accumulates dead records as notes are edited and deleted. Schedule compaction when the app is idle:

app.on('browser-window-blur', () => {
  db.compact()
})

compact() is fast and synchronous — it rewrites the file in a single forward pass. Calling it when the window loses focus means it runs during natural pauses without affecting responsiveness.

Packaging and distribution

This is where the native module problem usually surfaces. With pocket-db, there's nothing to rebuild. The electron-vite scaffold already wires up electron-builder; its config just works as-is:

# electron-builder.yml (generated by the scaffold)
mac:
  target: dmg
win:
  target: nsis
linux:
  target: AppImage

No electron-rebuild in your build pipeline. No platform-specific post-install hooks. The same config builds cleanly on all three platforms.

If you're using GitHub Actions to build releases:

# .github/workflows/release.yml
- name: Build
  run: npm run build && npx electron-builder
  # No electron-rebuild step needed

One fewer thing to maintain.

What about data migration?

pocket-db is schemaless — documents in the same collection can have different shapes. That means additive changes (adding a new field) require nothing: new documents have the field, old ones don't, and $exists lets you query for either.

For breaking changes, a one-time migration on startup is straightforward. Keep a version marker in a metadata collection — and remember that on a fresh database the marker doesn't exist yet:

const schema = db.collection('_meta')
const marker = schema.findOne({ key: 'version' })
const version = typeof marker?.value === 'number' ? marker.value : 0

if (version < 2) {
  // run migration
  notes.updateMany({}, { $set: { archived: false } })

  if (marker) {
    schema.replaceOne(marker._id as string, { key: 'version', value: 2 })
  } else {
    schema.insertOne({ key: 'version', value: 2 })
  }
}

Note the replaceOne(marker._id, …) call: the first argument is the document id. (Passing a full document as the only argument is a separate overload that replaces that document by itself — not what you want here.)

No migration framework needed for most desktop app scenarios.

The trade-offs to know

pocket-db is designed for single-process use. An Electron app fits that model perfectly — the main process is the only writer, and that's exactly the intended use case.

It is not the right choice if your Electron app spawns multiple Node worker processes that all need to write to the same database simultaneously. It is also not the right choice if you're storing hundreds of thousands of documents and need complex aggregation — at that scale, an actual database server starts making sense even in a desktop context.

For the common case — a desktop app storing user data, preferences, history, cached API responses, local task lists — it fits well.

Getting started

npm install @axfab/pocket-db

Full documentation at pocket-db.axfab.net. Source and issues at github.com/axfab/pocket-db. The complete example app from this article is in the repository of exemple, tests included.

If you've been putting up with native binding headaches in Electron, give it a try and let me know how it goes.

pocket-db is MIT licensed.

pocket-db vs lowdb vs LokiJS: an honest embedded database benchmark

Fabien B. — Sun, 31 May 2026 18:02:52 +0000

If you've ever built a CLI tool, an Electron app, or a local-first prototype in Node.js, you've faced the same question: where do I store structured data without spinning up a server?

The usual suspects are well known — lowdb, LokiJS, nedb, better-sqlite3. I recently built pocket-db, a new embedded document store for Node.js.

I wanted a database solution that offered me:

a single-file database
no native compilation
Mongo-like API
easy embedding in CLI or Electron apps

I wanted to know honestly where it stands. So I ran a benchmark across 10 common operations on a collection of 1,000 documents.
Here's what I found — not shying away from the unflattering parts.

The contenders

pocket-db — append-only single-file document store, MongoDB-like API, indexes in memory
sqlite (in-memory) — better-sqlite3 with an in-memory database, the fastest SQLite configuration
sqlite (file) — better-sqlite3 persisted to disk, the most common SQLite setup
json-file — full JSON file read/write on every operation
lowdb — JSON-based, keeps everything in memory and flushes to disk on write
lokijs — in-memory document store, optional persistence

All tests run on Apple M-series hardware. Results are in ops/sec — so higher is better.

The results

Operation                  pocket-db  sqlite (mem)  sqlite (file)  json-file      lowdb     lokijs
─────────────────────────────────────────────────────────────────────────────────────────────────
insertOne                    198,177     226,278 *         4,421      3,893      2,256      1,004
insertMany (100)               2,345       2,963 *           402        788        628        264
findById                     142,776   1,064,774         248,942  12,532,585 * 321,548  4,061,606
findAll                           96         361             361    115,774    870,822 *     1,179
findByName (scan)                 97         536             524     19,579     24,235 *     8,222
findByRole (index)               277         884             887     18,527     21,796 *     3,215
updateOne                     97,889     423,072 *         2,675        784        566        188
deleteOne                    454,402     465,026 *         5,551        924        663        220
countAll                      12,038   2,233,389         285,285  42,553,191 * 34,914,251 37,348,273
sortByScore (desc)                91         293             293      4,947      5,099 *     1,123
─────────────────────────────────────────────────────────────────────────────────────────────────
All values in ops/sec. * = fastest for this operation.

Before you ask: json-file wins read benchmarks because the entire dataset is already loaded in memory during the benchmark.

Why pocket-db writes so fast

The short answer: every write is a single sequential append to the end of the file. No B-tree rebalancing. No page allocation. No full-file reserialisation.

When you call insertOne, updateOne or deleteOne, pocket-db appends one record to the log and updates the in-memory index pointer. That's it. An update doesn't touch the old record — it writes a replacement which is enough to marks the previous offset as dead. A delete appends a tombstone. This is why deleteOne reaches 454,000 ops/sec, only marginally behind SQLite in-memory.

There's a second reason the numbers look good today: no fsync wait. Writes land in the OS page cache and return immediately. This is fast, but it's a trade-off — a hard crash before the OS flushes to disk can lose the last few writes. Pocket-db is perfectly capable to recover from incomplete operations, so consistency is never compromised. For the target use cases (CLI tools, desktop apps, local prototypes), this is an acceptable trade-off. For anything requiring strict durability guarantees, it's something to be aware of.

Why reads are slower — and what's being done about it

This is the honest part of the benchmark.

pocket-db keeps only indexes in memory, not documents. Every findOne and every cursor step seeks to the document's byte offset in the file and reads from disk. Every result then goes through JSON parsing. This is the fundamental cost of the current design — and it shows clearly in the numbers.

In-memory stores like lowdb and LokiJS serve reads directly from JavaScript objects: no I/O, no parsing. That's why findById on a json-file store reaches 12 million ops/sec while pocket-db sits at 142,000.

Keeping indexes in memory was a necessary first compromise. Without them, reads would require a full file scan for every query — completely unusable in practice. Indexes bring findById to a competitive level for real-world access patterns where you're not reading the same hot document in a tight loop. But the gap on read-heavy workloads is real, and it needs to be closed.

The roadmap for the next version is to address this directly. The first candidate is a document cache: keep recently accessed documents in memory and avoid redundant disk reads for hot-document workloads. This alone should close most of the gap for typical usage patterns. The second candidate is a binary document format to replace JSON serialisation — parsing structured binary is significantly faster than JSON.parse for complex documents. A third option under consideration is memory-mapped I/O with pagination, which would allow the OS to manage the hot-document cache at a lower level.

The memory footprint is also something to watch. Indexes in memory work well at the scales pocket-db is designed for — thousands to low hundreds of thousands of documents. As collections grow, that in-memory index cost grows with them. Load testing at larger scales is to be considered before locked of the architecture.

Reading the numbers in context

The benchmark is deliberately narrow: 1,000 documents, ten operations, one process. That's a fair representation of the workloads pocket-db is built for — a CLI tool tracking state, an Electron app storing user data, a local server caching API responses.

At that scale, the practical difference between 97 and 870,000 read ops/sec is manageable. A CLI command doesn't loop through findByName a million times. But it does insert records frequently, update state on every run, and occasionally compact. For that profile, pocket-db's write throughput is genuinely competitive.

Where the numbers are a real concern is if you're considering pocket-db for a workload that's read-heavy by nature — serving requests in a tight loop, scanning large collections on every query. For those cases today, a pure in-memory store is the honest recommendation.

Where things stand

pocket-db is a young project. These benchmark results are encouraging for what it's designed to do: fast, durable writes in a single-file embedded store with a familiar API. The write story is solid. The read story needs work, and the work ahead is well defined.

The next performance chapter — document cache, binary format, larger-scale index testing — will determine whether pocket-db can compete across the full spectrum of embedded workloads. That's the honest assessment at this first version.

If you want to run the benchmarks yourself:

git clone https://github.com/axfab/pocket-db
npm install
npm run bench

And if a single-file, zero-dependency document store with a MongoDB-style API sounds like what your next project needs:

npm install @axfab/pocket-db

@axfab/pocket-db is MIT licensed. Contributions, issue reports, and benchmark challenges welcome.