DEV Community

sravan27
sravan27

Posted on

I found silent data-loss bugs in 5 production databases this month - here's the open-source checker

#webdev #javascript #security #database

Most database bugs throw errors. The dangerous ones don't — they quietly return the wrong rows. No exception, no log line, just a query that silently drops or over‑matches data. That's the worst kind of bug, because nothing tells you it happened.

This month I went hunting for exactly that failure mode in JavaScript databases — the client‑side, embedded, and sync databases that re‑implement SQL‑ish operators (LIKE, case‑insensitive match, range comparison) in JS. I found it, and got fixes merged or under review, in five production databases:

Database Bug Status
PowerSync LIKE / range semantics merged (#644)
Rocicorp's Zero range / comparison merged (#6083, #6088)
InstantDB $like / $ilike newline merged (#2714)
ElectricSQL LIKE newline + escaped wildcards PR #4437
Dexie case‑fold drops rows PR #2306

Then I packaged the audit into an open‑source tool so you can run it on your database: silentdrop — npm i silentdrop.

A concrete one: Dexie silently drops rows

Dexie is the dominant IndexedDB wrapper (~2M downloads/week). Its equalsIgnoreCase walks the index assuming case conversion is length‑preserving. It isn't — German ß → SS, ligatures fi → FI, Turkish İ. So:

await db.items.where('name').equalsIgnoreCase('straße').toArray()
// expected: ['straße', 'STRAßE', 'Straße']
// actual:   ['straße', 'Straße']   ← 'STRAßE' silently dropped
Enter fullscreen mode Exit fullscreen mode

No error. A row that matches by the database's own case‑insensitive contract simply isn't returned. (Reported as Dexie #2306.)

Why it happens

These engines compile LIKE to a RegExp, or compare strings with JS operators, and the gaps from real SQL semantics are invisible:

  • LIKE and newlines — in SQL, % matches any sequence including \n. A RegExp without the dotAll flag silently misses rows containing newlines.
  • LIKE metacharacters — LIKE 'a.b' must match the literal a.b, not axb. Translate to RegExp without escaping and you over‑match — a correctness and injection risk.
  • Case folds that change length — the Dexie one above.
  • Non‑BMP ordering — SQL/Postgres orders text by code point; naive JS comparison orders by UTF‑16 code unit, so an emoji (U+1F600) sorts below U+F000 and a range query silently drops it.

The checker

silentdrop runs these cases against your database's operators and reports the divergences. You wire a tiny adapter:

import { check, report } from "silentdrop";

const adapter = {
  async reset()         { /* clear the store */ },
  async seed(values)    { /* insert string values */ },
  async like(pattern)   { /* run a LIKE query, return matches */ },
  async ilike(needle)   { /* case-insensitive equality */ },
  async gt(bound)       { /* values > bound */ },
};

report(await check(adapter));
Enter fullscreen mode Exit fullscreen mode

Run it against Dexie and it flags the case‑fold drop live; run it against AlaSQL and it passes the LIKE tests but flags the code‑point ordering divergence. A complete, runnable Dexie example is in the repo.

Why you should care

If you store names, addresses, search terms — anything with international characters — and you rely on case‑insensitive lookup or range queries for correctness (uniqueness checks, "is this taken?", access checks, "everything ≥ X"), you may be silently losing rows in production today. The fix is usually small. Finding it is the hard part — that's what the tool is for.

MIT‑licensed, zero runtime dependencies: https://github.com/sravan27/silentdrop. PRs adding adapters for more databases are very welcome.

If your sync/database layer is correctness‑critical and you'd rather have the whole operator surface hardened by hand — the same pass behind the five databases above — I take that on as a fixed 48‑hour sprint; details are in the repo README.

Top comments (0)