DEV Community

Cover image for I Used Sentry to Expose a Silent Data-Loss Bug with Zero Errors
Mir Shah
Mir Shah Subscriber

Posted on

I Used Sentry to Expose a Silent Data-Loss Bug with Zero Errors

#devchallenge #bugsmash #ai #development

Summer Bug Smash: Clear the Lineup 🐛🛹

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.

40 successful saves. 39 missing workspaces.

The application did not crash. No exception was thrown. Every promise resolved.

And that was exactly why this bug was dangerous.

The front-page lesson: A green “success” proves that an operation finished. It does not prove that its result is still there.

THE FOUR-MINUTE FIELD REPORT

IN THIS EDITION
The project  ·  The incident  ·  The patch  ·  The safeguards  ·  The signal

SECTION I · THE PROJECT DESK

Project Overview

Last month, I built Aether Canvas during OpenAI Build Week.

It is a local-first Electron application where spatially grouping ordinary files creates the mini-app you need. Put a flight confirmation, hotel booking, budget, packing list, and city guide together, for example, and Aether turns the cluster into a living trip workspace with routes, spending, tasks, and places—all traceable to the source files.

The idea is simple:

Space is the prompt.

But the build was anything but simple. It was a one-week hackathon, and those seven days had to cover validating the idea, designing the architecture, building the interface, connecting the AI workflow, testing the main experience, and preparing the final demo and presentation.

We shipped a working product. We did not have enough time for deep concurrency and persistence stress testing.

WHAT WE SHIPPED

A real local-first Electron product: spatial files, AI-generated workspaces, autosave, persistence, and a polished demo path. 		WHAT THE CLOCK HID

The main experience worked, but the one-week schedule left little room for simultaneous-operation and shutdown stress tests.

I also chose not to modify the original submitted repository before the winner announcement. I wanted the artifact being judged to remain exactly as submitted. Instead, I created a separate Bug Smash repository, with the submission preserved at commit 3163641a and tagged openai-hackathon-submission.

That gave me something unusually valuable for debugging: an untouched before state.

SECTION II · INCIDENT REPORT

Bug Fix or Performance Improvement

The bug hid behind successful writes

Aether stores each workspace in its own JSON file and keeps a separate index containing the workspaces visible in the sidebar.

The original implementation already used atomic temporary-file replacement and a write queue. At first glance, that looked safe. After tracing the workspace IPC calls, autosave path, rename path, and persistence service, I found the boundary was in the wrong place.

✓ WHAT WAS SAFE

Each individual JSON replacement was atomic. A partial write would not leave behind a half-written index. 		✕ WHAT WAS NOT SAFE

The read and modification happened before the queued write, so two complete operations could still overwrite one another.

The queue protected an individual JSON write:

const operation = writeQueue.then(async () => {
  await fs.writeFile(temporaryPath, contents, 'utf8');
  await fs.rename(temporaryPath, targetPath);
});
Enter fullscreen mode Exit fullscreen mode

But updating the workspace index is not one write. It is a complete read → modify → write transaction:

Create A: read index [] ── add A ── write [A]
Create B: read index [] ── add B ── write [B]
                                      ▲
                     both writes succeed; A disappears from the index
Enter fullscreen mode Exit fullscreen mode

Two operations could read the same old index before either write entered the queue. Both would make a perfectly valid update. Both writes would complete successfully. The last valid-but-stale snapshot would win.

No malformed JSON. No rejected promise. No crash for error monitoring to catch.

Just a workspace file that still existed on disk but was no longer reachable from the application.

Making a timing bug repeatable

I did not want to hammer the UI with clicks until I got lucky. I built deterministic before/after harnesses that run the same workloads against two real implementations:

🔴 BEFORE

Loads the authentic workspace service directly from submitted commit 3163641a with git show—not from a hand-written broken copy. 		🟢 AFTER

Loads the repaired service from the Bug Smash branch and subjects it to the exact same operations and assertions.

The workload creates 40 workspaces simultaneously, then runs 20 autosave-versus-rename races. Each stage uses an isolated temporary Electron profile, opens the real desktop UI with the resulting data, and deletes that data after the window closes. My actual Aether spaces are never touched.

npm run bugsmash:before -- --sentry
npm run bugsmash:after -- --sentry
Enter fullscreen mode Exit fullscreen mode

The harness also refuses to present an inconclusive result: the before stage must reproduce the legacy failure, and the after stage must preserve every mutation.

Identical workload Original submission Repaired version
Workspace files written 40 / 40 40 / 40
Workspaces reachable in the index 1 / 40 40 / 40
Orphaned workspace files 39 0
Autosave/rename trials that lost a mutation 20 / 20 0 / 20

The number that changed the investigation

40 files existed on disk. Only one existed to the user.

SECTION III · THE PATCH DESK

Code

CASE FILE EVIDENCE
Merge request Review Merge Request !1 and the complete diff
Repair commit 57fc12a — transaction-safe persistence
Public repository Aether Canvas Bug Smash

The heart of the repair is intentionally small. A failure-safe exclusive queue now surrounds the complete public operation:

const runExclusive = async <T>(operation: () => Promise<T>): Promise<T> => {
  pendingOperations += 1;
  const queued = operationQueue.catch(() => undefined).then(operation);
  operationQueue = queued.then(() => undefined, () => undefined);

  try {
    return await queued;
  } finally {
    pendingOperations -= 1;
  }
};
Enter fullscreen mode Exit fullscreen mode

Every workspace mutation now crosses that boundary as one unit:

return {
  list: () => runExclusive(readIndex),
  create: (name) => runExclusive(() => create(name)),
  load: (id) => runExclusive(() => readJson(filePath(id))),
  save: (workspace) => runExclusive(() => save(workspace)),
  rename: (id, name) => runExclusive(async () => {
    const workspace = await readJson(filePath(id));
    await save({ ...workspace, name: name.trim() || 'Untitled Space' });
  }),
};
Enter fullscreen mode Exit fullscreen mode

Now a create or rename finishes reading, modifying, and persisting its state before the next operation begins. A rejected operation is absorbed only for queue continuity and is still returned to its original caller, so one disk failure cannot permanently poison later saves.

Explore the complete before/after repository

SECTION IV · ENGINEERING FOLLOW-THROUGH

My Improvements

Fixing the index race exposed two nearby assumptions that also needed attention. I treated this as one persistence-integrity repair rather than stopping at the first passing test.

01 · TRANSACTION BOUNDARY

One long-lived store and one exclusive queue protect every complete persistence operation. 		02 · SAVE REVISIONS

A newer edit cannot be cleared by the completion of an older in-flight save.
03 · CLOSE HANDSHAKE

Electron waits for the renderer's latest snapshot before destroying the window. 		04 · REGRESSION PROOF

Six deterministic tests protect the failure modes instead of merely mirroring the implementation.

1. One store, one transaction queue

The Electron main process now keeps one long-lived workspace store across IPC handlers. Creating a new store per request would create multiple queues and quietly defeat serialization.

2. Revision-aware autosaving

The renderer previously used a boolean dirty flag. That is not enough when a new edit arrives while an older save is still running: completion of the older save can clear the flag belonging to the newer edit.

The new saver tracks revisions and drains until the saved revision catches the latest one:

while (this.latest && this.savedRevision < this.revision) {
  const snapshot = this.latest;
  const revision = this.revision;
  await save(snapshot);
  this.savedRevision = revision;
}
Enter fullscreen mode Exit fullscreen mode

If revision 2 arrives while revision 1 is being written, revision 1 cannot declare revision 2 safe. The loop writes again.

3. A real close handshake

An asynchronous beforeunload callback cannot force Electron to wait before destroying its renderer. The main process now intercepts the close request, asks the renderer to drain its latest snapshot, and closes the native window only after the renderer acknowledges completion.

4. Regression tests for the failure, not just the implementation

The six deterministic tests cover:

  • 40 concurrent creates preserving 40 unique index entries;
  • autosave racing rename while preserving both mutations;
  • queue recovery after a rejected operation;
  • a newer edit arriving during an active save;
  • retrying a failed snapshot write without losing dirty state; and
  • count-only integrity auditing for orphaned and missing files.

Strict TypeScript checks, the renderer/main/preload production builds, and Linux AppImage packaging also pass with the repair.

Reproduce the controlled comparison locally 
git clone https://gitlab.com/abbasmir12/aether-canva-bugsmash.git
cd aether-canva-bugsmash
git switch fix/workspace-transaction-race
npm install
npx vite build

# Run each stage separately so the real Electron state remains visible.
npm run bugsmash:before
npm run bugsmash:after

# Or print both results without opening two staged windows.
npm run bugsmash:demo
Enter fullscreen mode Exit fullscreen mode

The Sentry flag requires your own DSN. The reproduction itself does not.

SECTION V · OBSERVABILITY DESK

Best Use of Sentry

I am submitting this entry for Best Use of Sentry.

Sentry had a specific job here: make a silent logical invariant observable.

It would be inaccurate to say that Sentry magically discovered the source line or repaired the race. Code inspection and deterministic stress tests found the transaction-boundary bug. Sentry then gave me runtime evidence that the application could report successful operations while its persisted state was inconsistent—and confirmed that the repaired build remained consistent under the identical workload.

WHAT SENTRY DID

Recorded queue pressure, operation duration, and the count-only workspace integrity invariant in the running Electron application. 		WHAT SENTRY DID NOT DO

It did not invent the root cause or upload the user's workspace. Code inspection and tests located the faulty transaction boundary.

Operation tracing

Using the Sentry Electron SDK, every workspace IPC operation creates a custom transaction containing only:

  • the operation type, such as create, save, or rename;
  • queue depth; and
  • duration.

This made serialization pressure visible. For example, an authenticated workspace.create trace showed a queue depth of 10 and a root duration of approximately 175 ms. A successful span status alone, however, could not reveal the hidden loss. For that I needed a product-level invariant.

The integrity transaction

The opt-in workspace.integrity-audit compares the index with the workspace directory locally and reports only aggregate counts:

attributes: {
  'aether.workspace.files_count': audit.workspaceFiles,
  'aether.workspace.indexed_count': audit.indexedWorkspaces,
  'aether.workspace.orphaned_count': audit.orphanedFiles,
  'aether.workspace.missing_file_count': audit.missingFiles,
  'aether.workspace.integrity_consistent': audit.consistent,
}

span.setStatus(audit.consistent
  ? { code: 1, message: 'ok' }
  : { code: 2, message: 'data_loss' });
Enter fullscreen mode Exit fullscreen mode

The original run produced:

files_count          40
indexed_count         1
orphaned_count       39
integrity_consistent false
status               data_loss
Enter fullscreen mode Exit fullscreen mode

The repaired run produced:

files_count          40
indexed_count        40
orphaned_count        0
integrity_consistent true
status               ok
Enter fullscreen mode Exit fullscreen mode
🔴 BEFORE
SILENT DATA LOSS


40 files · 1 indexed
39 orphaned · data_loss 		│
│
│
│
│
│
│
│
│ 		🟢 AFTER
INTEGRITY RESTORED


40 files · 40 indexed
0 orphaned · ok

THE TRACE'S STRANGEST HEADLINE
Issues: 0  ·  Orphaned workspaces: 39

Traditional exception monitoring was telling the truth—nothing threw. The integrity transaction supplied the missing definition of correctness.

Observability without uploading the workspace

Aether is local-first, so the instrumentation had to respect that promise. The custom telemetry never includes workspace names, IDs, paths, file contents, canvas content, AI prompts, or AI responses. Transaction hooks remove user, request, extra, and breadcrumb payloads; UI-click breadcrumbs are disabled; raw IP storage is disabled in the Sentry project; and an advanced scrubbing rule removes user.geo.

This was also a useful lesson: observability is not only about collecting more information. Sometimes it is about identifying the smallest safe signal that proves the system is healthy.

FINAL EDITION · THE QUIETEST FIXES CAN PROTECT THE MOST IMPORTANT DATA

A deliberately boring result

Save a workspace, and it stays saved.

That is the point.

No flashy new feature, just 39 workspaces recovered in the controlled stress scenario, zero lost autosave mutations, and a save operation that finally means saved.

Top comments (0)