DEV Community: Amorizz

What the Sentry SDK drops after a 429 quota response

Amorizz — Thu, 12 Nov 2026 14:20:54 +0000

I'm building an error tracker, and I needed to know what an official Sentry SDK does when my ingest answers "you're over quota". I couldn't find anyone showing the wire, so I put a stub server in front of @sentry/node and logged every request it got.

The failure is silent. A Sentry issue from 2025 says users "struggle with “why isn’t it working”" until someone sends them to the Stats & Usage page. Nothing throws. Events just stop showing up.

Node v24.22.1, @sentry/node 11.11.0 (run-0-versions.txt), all local on 2026-11-09. The server is a 55-line stub I wrote, not Sentry. What Sentry's servers do comes from their public docs.

TL;DR

After one 429 with X-Sentry-Rate-Limits: 60:error:organization:usage_exceeded, I captured 10 errors and only 3 hit the wire: 2 accepted, 1 rejected. The other 7 never left the process. No queue, no retry, and one warn-level debug line.
The SDK backs off for exactly what the header says. With a 5s window it sent one probe every ~5s. A bare 429 with no headers blocked everything for 60s.
The SDK does report the loss in a client_report, but the report said 7 when 8 were lost, because it skips the event that got the 429. Under a bare 429 the report itself gets blocked: 11 local drops, none reported.

How the SDK handles a rate limit

It keeps a "disabled until" timestamp per data category, sets it from the response headers, and silently drops matching items before they hit the network until that time passes. There's no buffer of "send later" events.

That matches Sentry's rate limiting spec for SDKs: "Events, transactions, sessions, and/or any other supported payload type, depending on the communicated limits, are to be discarded during the period that the rate limit is in place." The billing side says the same in I've exhausted my quota, what now?: "When your quota is exceeded, events are dropped and cannot be queued or stored."

The parsing order in @sentry/core 11.11.0 (ratelimit.js). Rows 1, 3 and 4 are in my runs below; row 2 I only read in the source:

Response	What the SDK blocks	For how long
`X-Sentry-Rate-Limits: 60:error:...`	only the listed categories (`error`)	the first number, in seconds
`X-Sentry-Rate-Limits: 60::...` (empty category)	everything	the first number
`429` + only `Retry-After: 5`	everything	`Retry-After`
`429` with neither header	everything	60s

"Everything" includes the SDK's own client_report items. That matters later.

Step 1: a stub that plays "quota exceeded"

The stub accepts /api/<id>/envelope/, logs one JSON line per request, and answers 429 after CAP items of one category. Trimmed to the item loop and the cap decision:

// stub.js: fake Sentry ingest. Logs every envelope request to JSONL and can play "quota exceeded".
// Env: OUT (jsonl), CAT (error|transaction|span), CAP (items of CAT accepted before the cap),
//      HEADER (full|retry|none), WINDOW (seconds in the headers), RESET_MS (quota refills after this)
// ...
const typeToCat = { event: 'error', transaction: 'transaction', span: 'span', client_report: 'internal' };
// ...
    for (let i = 1; i < lines.length && lines[i]; i += 2) {
      const head = JSON.parse(lines[i]);
      items.push(head.type);
      if (head.type === 'client_report') report = JSON.parse(lines[i + 1]);
      if (head.type === 'event') items.message = JSON.parse(lines[i + 1]).exception?.values?.[0]?.value;
    }
    if (RESET_MS && cappedAt && Date.now() - cappedAt >= RESET_MS) { used = 0; cappedAt = 0; }
    const mine = items.filter(t => typeToCat[t] === CAT).length;
    let status = 200, headers = { 'content-type': 'application/json' };
    if (mine && used + mine > CAP) {
      status = 429;
      if (!cappedAt) cappedAt = Date.now();
      if (HEADER === 'full') {
        headers['x-sentry-rate-limits'] = `${WINDOW}:${CAT}:organization:usage_exceeded`;
        headers['retry-after'] = WINDOW;
      } else if (HEADER === 'retry') {
        headers['retry-after'] = WINDOW;
      }
    } else {
      used += mine;
    }

usage_exceeded is my own reason code; SDKs ignore it.

Step 2: an app that captures on a timer

The app captures boom #1, boom #2 and so on, then calls Sentry.close(3000) so pending sends and client reports flush. Trimmed to the init and the afterSendEvent hook, which prints what the transport got back for each error:

// app.js: capture N errors (and optionally a transaction each tick) and print what the SDK reports back.
// ...
Sentry.init({
  dsn: 'http://devkey@127.0.0.1:9999/1',
  debug: process.env.DEBUG === '1',
  tracesSampleRate: process.env.TX === '1' ? 1 : undefined,
  clientReportFlushInterval: Number(process.env.REPORT_MS || 60000),
});

// Detection hook: fires after every captured error, with the transport's response.
Sentry.getClient().on('afterSendEvent', (event, res) => {
  const msg = event.exception?.values?.[0]?.value;
  const verdict = res.statusCode === undefined ? 'NOT SENT' : res.statusCode;
  console.log(`${clock()} afterSendEvent ${msg} -> ${verdict}${res.headers?.['x-sentry-rate-limits'] ? ' limits=' + res.headers['x-sentry-rate-limits'] : ''}`);
});

Each scenario is one line in run-all.sh, which starts the stub and the app with the same env (after npm i --save-exact @sentry/node@11.11.0):

shot() { # $1 name, rest = env for both stub and app
  local name=$1; shift
  env OUT=stub-$name.jsonl "$@" node stub.js > /dev/null & local stub=$!; sleep 0.5
  { echo "## env: $*"; env "$@" node app.js 2>&1; sleep 0.3; echo "## stub saw:"; node summary.js stub-$name.jsonl; } > run-$name.txt
  kill $stub; wait $stub 2>/dev/null || true
}
# ...
shot 2-cap-60s       N=10 EVERY_MS=500 CAP=2 HEADER=full WINDOW=60

The baseline (run-1-baseline.txt, no cap) is boring in a good way: requests=5 error_events_on_wire=5 accepted=5.

What happens at the cap: 7 of 10 errors never leave the process

After the 429, the next errors are dropped locally. No wait, no retry, and the stub never sees them.

App side, from boom #2 to boom #5 (run-2-cap-60s.txt):

17:57:35.383 captured boom #2
17:57:35.387 afterSendEvent boom #2 -> 200
17:57:35.885 captured boom #3
17:57:35.888 afterSendEvent boom #3 -> 429 limits=60:error:organization:usage_exceeded
17:57:36.388 captured boom #4
17:57:36.388 afterSendEvent boom #4 -> NOT SENT
17:57:36.889 captured boom #5
17:57:36.890 afterSendEvent boom #5 -> NOT SENT

boom #6 to boom #10 look exactly like #4. The stub's view of the same run:

17:57:34.902 200 [event] boom #1
17:57:35.386 200 [event] boom #2
17:57:35.887 429 [event] boom #3 {"x-sentry-rate-limits":"60:error:organization:usage_exceeded","retry-after":"60"}
17:57:39.912 200 [client_report] discarded=[{"reason":"ratelimit_backoff","category":"error","quantity":7}]
requests=4 error_events_on_wire=3 accepted=2

10 captured. 2 stored. 1 rejected. 7 dropped in the SDK. boom #3 is never resent. And captureException kept handing back event ids the whole time (run-8-eventid.txt: boom #4 id=5bc619400e85411889c667014b50bbc1 for an event that never left).

With debug: true (run-2b-cap-debug.txt, 4 errors), these are the lines around the cap. The rest is 27 "Integration installed" lines and init noise:

Sentry Logger [warn]: Sentry responded with status code 429 to sent event.
17:57:41.978 afterSendEvent boom #3 -> 429 limits=60:error:organization:usage_exceeded
Sentry Logger [log]: Captured error event `boom #4`
17:57:42.477 captured boom #4
Sentry Logger [log]: Recording outcome: "ratelimit_backoff:error"
17:57:42.478 afterSendEvent boom #4 -> NOT SENT

One warn for the 429. Every drop after that is a log line. Keep only warnings and you see the first one and nothing else.

Category limits only block that category

A limit on span doesn't touch errors. The SDK keeps one timer per category, so errors kept flowing while spans were blocked. run-3-span-only.txt used CAT=span TX=1, so each tick sent one error and one span envelope:

17:57:45.100 200 [event] boom #3
17:57:45.600 429 [span] {"x-sentry-rate-limits":"60:span:organization:usage_exceeded","retry-after":"60"}
17:57:45.602 200 [event] boom #4
17:57:46.105 200 [event] boom #5
17:57:46.606 200 [event] boom #6
17:57:47.109 200 [client_report] discarded=[{"reason":"ratelimit_backoff","category":"span","quantity":3}]

Note the item type: with tracesSampleRate: 1, 11.11.0 sent span items, not transaction (stub-3-span-only.jsonl). If your proxy limits transaction, check what you actually receive.

How long it backs off, and when it tries again

It waits exactly the window it was given, then sends the next real event as a probe. If that one also gets a 429, the window starts over. With WINDOW=5 and one error per second (run-5a-window-5s.txt):

17:59:22.491 429 [event] boom #3 {"x-sentry-rate-limits":"5:error:organization:usage_exceeded","retry-after":"5"}
17:59:27.501 429 [event] boom #8 {"x-sentry-rate-limits":"5:error:organization:usage_exceeded","retry-after":"5"}
17:59:32.509 429 [event] boom #13 {"x-sentry-rate-limits":"5:error:organization:usage_exceeded","retry-after":"5"}

Under a still-exhausted quota you get one request per window, and that event is lost too. When the stub refilled the quota 6s after the cap (run-5b-refill.txt), events flowed again on the first probe:

17:59:42.635 429 [event] boom #8 {"x-sentry-rate-limits":"5:error:organization:usage_exceeded","retry-after":"5"}
17:59:47.644 200 [event] boom #13
17:59:48.645 200 [event] boom #14

boom #9 to #12 were dropped even though my stub would have taken them from about 17:59:43. The SDK can't know the quota is back until its window ends.

Without the rate-limit header, Retry-After: 5 gave the same 5s rhythm (run-4a-retry-only.txt). A bare 429 gave the spec's 60s default (run-4b-bare-429.txt):

17:58:11.283 429 [event] boom #3
17:59:11.330 429 [event] boom #15

The client report says 7. I lost 8

The SDK counts its drops and sends a client_report envelope. Raw line from stub-2-cap-60s.jsonl:

{"at":"17:57:39.912","ms":5717,"path":"/api/1/envelope/","items":["client_report"],"status":200,"client_report":{"timestamp":1794243459.91,"discarded_events":[{"reason":"ratelimit_backoff","category":"error","quantity":7}]}}

ratelimit_backoff comes from Sentry's client reports spec. Three things I only learned by running it:

It doesn't count the rejected event. quantity: 7, but 8 errors were lost: boom #3 got the 429 and boom #4 to #10 were dropped locally. If you build an ingest, count your own 429s as well.
It's on a timer. clientReportFlushInterval defaults to 60000 ms (from the SDK's type docs). Run 2 got its report at close(). With REPORT_MS=3000 (run-6-report-3s.txt) it came in batches: quantity 3, 3, then 1 at close. A process that dies hard before either never reports.
An all-category limit eats it. With a bare 429, run-4b-bare-429.txt shows no client_report at all for 11 local drops. The debug run shows why (run-4c-bare-debug.txt):

Sentry Logger [log]: Sending outcomes: [ { reason: 'ratelimit_backoff', category: 'error', quantity: 1 } ]
Sentry Logger [log]: Recording outcome: "ratelimit_backoff:internal"

The report is an internal item, and the spec says the "all" limit applies to internal too. So the report was blocked and counted as one more drop. run-4a-retry-only.txt only got its report because close() landed 5.007s after the last 429, just past the window.

How to detect it from your app and your ingest

In the app, the afterSendEvent hook from Step 2 is the detector. It fired for every captured error: statusCode: 429 for the rejected one, none for local drops. Swap my console.log for a counter or a console.warn.

I only tested rate limits. From the SDK source, a network error also comes back with no statusCode, so this lumps both together. Spans don't go through this hook.

If you run the ingest, sum client_report discards plus your own 429s per DSN. discards.js does that over the stub logs (run-7-detect.txt, first rows):

stub-2-cap-60s.jsonl       429s=1 reported_lost={"ratelimit_backoff:error":7}
stub-2b-cap-debug.jsonl    429s=1 reported_lost={"ratelimit_backoff:error":1}
stub-3-span-only.jsonl     429s=1 reported_lost={"ratelimit_backoff:span":3}
stub-4a-retry-only.jsonl   429s=2 reported_lost={"ratelimit_backoff:error":8}
stub-4b-bare-429.jsonl     429s=2 reported_lost={}

The last row is the trap. Two 429s, zero reported losses, 11 events actually dropped.

Failure checklist

Watching for exceptions. Nothing throws at the cap, and captureException still returns an event id for an event that never leaves the process.
Filtering out debug logs. One warn for the first 429, then only log-level "Recording outcome" lines.
Expecting a retry. The event that got the 429 is gone, and so is everything captured during the window (7 of 10 in run 2).
Trusting client reports for the full count. They skip the 429ed event, arrive on a 60s timer or at close(), and vanish under an all-category limit.
Sending a bare 429 from your own proxy or ingest. It blocks every category for 60s, client reports included. Send X-Sentry-Rate-Limits with a category if you only mean errors.
Limiting the wrong category in a proxy. 11.11.0 sent span items, not transaction. Check the item types on your wire.
Skipping Sentry.close() in short jobs. Pending client reports flush there. No close, no report.

Run it yourself

The listings above are trimmed. The full lab is run-all.sh (30 lines) plus stub.js, app.js, eventid.js, summary.js and discards.js: 11 scenarios in about 2.5 minutes, mostly the 60s bare 429. A second full run in a fresh copy gave the same counts.

I'm building Epure, a small self-hosted error tracker (errors only) that accepts events from Sentry SDKs and runs as an app plus Postgres in Docker Compose (v0.x, Apache 2.0). This lab is how I'm deciding what my own 429 should look like. Nothing above depends on it. The tap method on its own is in Log what your Sentry SDK actually sends, and the flood side is in Your sampleRate won't save you from an error flood.

If you've hit a quota cap in production, how did you first find out events were being dropped?

Verify a minified stack against its JavaScript source map

Amorizz — Sun, 08 Nov 2026 14:31:57 +0000

A few weeks ago I wrote that minified browser code gives you function names like t and n until you apply source maps, and then admitted I hadn't tested it. This weekend I tested it.

I didn't want another upload walkthrough. I wanted to know one thing: when a frame says a @ app.min.js:1:88, how do I prove the map I have turns it into the right function and the right line? And what does it look like when the map is wrong?

So I built a 15-line checkout module, minified it with esbuild, threw a known error through the official Sentry SDK into a local tap, and applied the map by hand with the source-map package. Then I broke it two ways on purpose.

Versions from run-0-versions.txt: Node v24.22.1, esbuild 0.28.2, source-map 0.8.0, @sentry/node 11.11.0. Everything ran locally on 2026-11-08. No tracker was involved, so this works the same if you run GlitchTip, self-hosted Sentry, or your own ingest.

TL;DR

With the matching map, the SDK frame a @ app.min.js:1:88 resolved to chargeCard at src/checkout.js:5, the exact throw line. The caller t resolved to processOrder at line 12.
A stale map gave no error and no warning. It mapped the crash to return { charged: total }; and the caller to module.exports = { processOrder };. A map stored under the wrong release wasn't found at all, and the event was stored anyway with o and u.
What fixed it for me is a check: throw a known error from the built bundle, apply the map you're about to upload, and fail if the crash frame isn't chargeCard on a line containing throw. Mine prints PASS or FAIL and exits 1 on a miss.

The mental model: a map is a position lookup with a two-part key

A source map answers one question: "this line and column in the bundle came from which file, line and column in the source?" Your tracker gets a position from the SDK, finds a map, and asks that question. It can only find the right map if the event's key matches how you stored it.

Here's what each side holds, from my lab files:

Piece	What it holds in this lab
SDK frame	`filename`, `function` (`a`), `lineno` (1), `colno` (88)
`.map` file	`version`, `sources`, `sourcesContent`, `mappings`, `names`
Lookup key	the event's `release` (`shop@1.0.0`) plus the frame's file name

Three things have to line up: the bundle that actually ran, the map built from that same bundle, and the key you stored the map under. Any one of them off and you get either minified frames or wrong frames. Wrong frames are the worse case, because they look fine.

Step 1: a bundle with two named functions

The source is two functions so a demangled frame is obvious at a glance:

// src/checkout.js: two named functions so a demangled frame is obvious.
function chargeCard(order) {
  const total = order.items.reduce((sum, item) => sum + item.price * item.qty, 0);
  if (order.card.balance < total) {
    throw new Error(`card declined: needed ${total}, balance ${order.card.balance}`);
  }
  return { charged: total };
}

function processOrder(order) {
  if (!order.items.length) return { charged: 0 };
  return chargeCard(order);
}

module.exports = { processOrder };

Install pinned and build:

npm i --save-exact esbuild@0.28.2 source-map@0.8.0 @sentry/node@11.11.0
npx esbuild src/checkout.js --bundle --minify --sourcemap --platform=node --outfile=dist/app.min.js

esbuild wrote dist/app.min.js (294b) and dist/app.min.js.map (1.0kb). The whole bundle is one line:

function a(e){let r=e.items.reduce((n,c)=>n+c.price*c.qty,0);if(e.card.balance<r)throw new Error(`card declined: needed ${r}, balance ${e.card.balance}`);return{charged:r}}function t(e){return e.items.length?a(e):{charged:0}}module.exports={processOrder:t};
//# sourceMappingURL=app.min.js.map

chargeCard became a. processOrder became t. Those are the names your tracker sees without a map.

Step 2: capture the frames the SDK actually sends

Don't trust err.stack for this. Read what the SDK puts in exception.values[0].stacktrace.frames, because that's what your tracker symbolicates.

I pointed Sentry.init at http://devkey@127.0.0.1:9999/1 and ran a tiny tap on that port. It unzips each envelope, keeps event items, and appends the release, the error message and the frames to a JSONL file:

// tap.js: fake DSN host. Saves each error event's frames to events.jsonl.
const http = require('http'), zlib = require('zlib'), fs = require('fs');
http.createServer((req, res) => {
  const chunks = [];
  req.on('data', c => chunks.push(c));
  req.on('end', () => {
    let buf = Buffer.concat(chunks);
    if (req.headers['content-encoding'] === 'gzip') buf = zlib.gunzipSync(buf);
    const lines = buf.toString().split('\n');
    for (let i = 1; i < lines.length && lines[i]; i += 2) {
      if (JSON.parse(lines[i]).type !== 'event') continue;
      const ev = JSON.parse(lines[i + 1]);
      fs.appendFileSync(process.env.OUT || 'events.jsonl', JSON.stringify({
        release: ev.release,
        value: ev.exception.values[0].value,
        frames: ev.exception.values[0].stacktrace.frames,
      }) + '\n');
    }
    res.end('{}');
  });
}).listen(9999, '127.0.0.1', () => console.log('tap on :9999'));

run.js inits the SDK with release: 'shop@1.0.0', requires the bundle, calls processOrder with a card balance of 50 and an 80 total, and calls Sentry.captureException in the catch. The in-app frames the SDK sent (run-3-sdk-frames.txt):

release: shop@1.0.0
{"filename":"dist/app.min.js","function":"a","lineno":1,"colno":88}
{"filename":"dist/app.min.js","function":"t","lineno":1,"colno":209}
{"filename":"run.js","function":"Object.?","lineno":11,"colno":3}

That's the "before" picture. Message card declined: needed 80, balance 50, function a, line 1.

Step 3: apply the map and print the original line

Applying a map is originalPositionFor with the frame's line and column. Printing the source line next to the result is what turns "it returned something" into proof you can eyeball.

The core of symbolicate.js:

const map = await new SourceMapConsumer(raw);
for (const f of ev.frames.filter(f => f.in_app && f.filename.endsWith('.min.js')).reverse()) {
  // SDK colno is 1-based, source-map wants 0-based columns
  const pos = map.originalPositionFor({ line: f.lineno, column: f.colno - 1 });
  const src = pos.source ? map.sourceContentFor(pos.source, true) : null;
  const text = src && pos.line ? src.split('\n')[pos.line - 1].trim() : '(no source)';
  // pos.name is the token at the crash column, not the enclosing function.
  // For the function, map the token right after "function <minName>(" in the bundle.
  const at = minLines[f.lineno - 1].indexOf(`function ${f.function}(`);
  const fn = at < 0 ? null : map.originalPositionFor({ line: f.lineno, column: at + 9 }).name;
  console.log(`${f.function} @ app.min.js:${f.lineno}:${f.colno}`);
  console.log(`  -> ${pos.source}:${pos.line}:${pos.column} token=${pos.name} function=${fn}`);
  console.log(`     ${text}`);
}

Output with the matching map (run-4-demangle.txt):

release shop@1.0.0, map dist/app.min.js.map
a @ app.min.js:1:88
  -> ../src/checkout.js:5:10 token=null function=chargeCard
     throw new Error(`card declined: needed ${total}, balance ${order.card.balance}`);
t @ app.min.js:1:209
  -> ../src/checkout.js:12:9 token=chargeCard function=processOrder
     return chargeCard(order);

That's the "after" picture, and it's right. The crash frame lands on the throw. The caller lands on return chargeCard(order); inside processOrder.

Look at the token= column, though. For frame t, the map's name at that position is chargeCard, because the column points at the call to chargeCard. If your symbolicator labels frames with pos.name, it calls the processOrder frame chargeCard. That's why my script does a second lookup on the function t( declaration to get the enclosing name. The fn lookup is a lab shortcut that only works for function x( declarations. Real symbolicators work out the enclosing scope properly, so it's worth checking that yours does.

Step 4: break it with a stale map

A stale map doesn't fail. It returns a confident answer for the wrong code.

I made v1.0.1: added an applyCoupon function above chargeCard and used it in the total. esbuild renamed everything. applyCoupon is now a, chargeCard is o, processOrder is u. The SDK sent o @ app.min.js:1:156 and u @ app.min.js:1:277 with release: shop@1.0.1.

Then I did what happens when a deploy ships the new bundle but the upload step reuses an old artifact. I applied the v1.0.0 map to the v1.0.1 frames (run-7-stale-map.txt):

release shop@1.0.1, map dist/app.min.js.map
o @ app.min.js:1:156
  -> ../src/checkout.js:7:2 token=null function=null
     return { charged: total };
u @ app.min.js:1:277
  -> ../src/checkout.js:15:32 token=null function=null
     module.exports = { processOrder };

No exception from source-map. No warning. The crash "happened" on a return statement, and the caller "is" the module.exports line. If your tracker shows only file and line, this looks like a real stack. You'd go read line 7 of checkout.js and find nothing wrong there.

With the right v1.0.1 map, the same frames resolve to ../src-v2/checkout.js:11:10 (the throw, now 6 lines lower) and processOrder at line 18 (run-7b-right-map.txt).

Step 5: miss the release key

A map stored under the wrong release isn't used at all, and the event still goes in.

resolve.js builds the key the way a tracker keys artifacts: artifacts/<release>/<file>.map. I only stored the v1.0.0 map. Output (run-8-release-lookup.txt):

a @ app.min.js:1:88 key=artifacts/shop@1.0.0/app.min.js.map FOUND
t @ app.min.js:1:209 key=artifacts/shop@1.0.0/app.min.js.map FOUND
event stored anyway: "card declined: needed 80, balance 50" (10 frames)
o @ app.min.js:1:156 key=artifacts/shop@1.0.1/app.min.js.map MISSING, frame stays minified
u @ app.min.js:1:277 key=artifacts/shop@1.0.1/app.min.js.map MISSING, frame stays minified
event stored anyway: "card declined: needed 80, balance 50" (10 frames)

This one at least looks broken. You see o and u and know the map is missing. Honestly I'd take this failure over the stale map every time.

Turn it into a check you run before upload

Throw a known error from the bundle you're about to deploy, symbolicate it with the map you're about to upload, and assert both the function and the line. With EXPECT=chargeCard set, symbolicate.js checks that the crash frame resolves to that function and that the source line contains throw (run-11-verify.txt):

PASS crash frame is chargeCard at a throw
exit 0
...
FAIL expected chargeCard at a throw, got null at "return { charged: total };"
exit 1

In a real app you don't throw from checkout in CI. Add a tiny canary export (a function that throws a fixed message) to the bundle, call it from a script after the build, and point the check at that. If the deploy job fails on exit 1, a stale map never reaches your tracker. The whole lab is in run-all.sh, a 51-line script of shell plus four small JS files (tap.js, run.js, symbolicate.js, resolve.js).

What about node --enable-source-maps?

For Node you can skip uploads and let Node apply the map at throw time. I tried it. It's good, but it isn't complete.

With node --enable-source-maps, the SDK sent this (run-10-sdk-enable-source-maps.txt):

{"filename":"src/checkout.js","function":"chargeCard","lineno":5,"colno":11}
{"filename":"src/checkout.js","function":"t","lineno":12,"colno":10}

File and line are right for both frames. The function name is right for the crash frame and still t for the caller. Plain err.stack printed at Object.t (.../src/checkout.js:12:10) too. Also note 5:11 here versus 5:10 from source-map. Same spot, one is 1-based and one is 0-based.

Browsers don't have this flag, so frontend bundles still need the upload path.

Failure checklist

Stale map. New bundle, old map. You get plausible wrong lines with no error (return { charged: total }; in my run). Only a known-throw check catches it.
Release string mismatch. The event says shop@1.0.1, the map is stored under shop@1.0.0. Frames stay o and u, and the event is stored anyway. Use the exact same string in Sentry.init and in the upload command.
File name mismatch. The file name is the other half of the key. My resolve.js uses the basename; real trackers match on the frame's URL or path, often with a ~/ prefix for "any host". I didn't run this case separately, but in resolve.js it fails the same way as the release miss.
Token name used as function name. pos.name at frame t was chargeCard, the callee. A symbolicator that trusts it mislabels the caller frame.
Column base. SDK frames and Node stacks are 1-based (5:11). source-map columns are 0-based (5:10). In my bundle, forgetting the - 1 landed on the same mapping (run-5-no-col-fix.txt is identical to run-4), so a check like mine wouldn't catch it here. It bites when the crash column sits right at a segment boundary.
Map served publicly with your source. esbuild puts sourcesContent in the map by default, so a .map next to app.min.js on your CDN ships your original code. With --sources-content=false the map dropped from 1024 to 493 bytes and still resolved chargeCard and processOrder, but printed (no source). That also means the throw line check needs the source from your repo instead.
--enable-source-maps is partial. It fixed file and line but left the caller as t.

Where this fits for me

I'm building Epure, a small self-hosted error tracker that matches maps by release and file name, so steps 4 and 5 are exactly the failures I care about. The source maps guide has the upload call and its limits. Nothing above depends on it. The check runs before any tracker sees the map.

Takeaway

A successful map upload only tells you a file arrived. A known throw that resolves to chargeCard on a throw line tells you it's the right file for the code that's running. That's one extra CI step, and it turns the stale map from a silent wrong answer into a failed deploy.

Do you scrub request bodies before they reach your error tracker?

Amorizz — Sun, 01 Nov 2026 14:22:28 +0000

Last week I put a fake DSN host behind a small Express app to log what the Sentry SDK actually sends. I went in for the byte counts. The thing I keep thinking about a week later is one line in a stored error event:

request.data = {"items":[1,2],"card_last4":"4242"}

That's the JSON body of a checkout request, saved inside the error. I never set sendDefaultPii. This was @sentry/node 11.9.0 with Express 4.22.3 and close to default config. The SDK did replace the session cookie with [Filtered] on its own. It left the body alone, and it attached the client IP as user.ip_address.

My test body was fake. A real one could be a password reset form, a signup with an email and phone number, or a webhook payload with a customer's address in it.

Why I can't shrug this off

On a SaaS plan, the vendor usually has server-side scrubbing and a retention policy, and you clicked through their DPA at some point. Self-hosted, it's my disk. The body sits in my Postgres, in last night's pg_dump, in the backup bucket, for as long as the retention I set (or forgot to set).

And the body is the part I almost never read when I debug. In that run the whole error event was 7,234 bytes. The exception (type, message, stack frames) was 4,026 of it and the modules list was 1,588. When I deleted the body, cookies and IP in beforeSend, the event went to 7,099 bytes. I dropped 135 bytes and nothing I'd look at for a 500.

beforeSend(event) {
  if (event.request) {
    delete event.request.data;
    delete event.request.cookies;
    delete event.request.headers?.cookie;
  }
  if (event.user) delete event.user.ip_address;
  return event;
}

Where I'm not sure

Deleting the whole body is the blunt option. Some bugs only make sense with the payload: a validation error on a field you didn't expect, or a client sending null where you assumed a string. A denylist (password, card*, token, email) keeps that context, but it only covers the fields you thought of. The day someone adds a tax_id field to a form, it lands in your error store until somebody notices.

Where I've landed for now: delete by default, and when I'm chasing one specific bug, log the part of the body I need at the call site, on purpose, then remove it.

I'm building Epure, a lightweight self-hosted error tracker, and this is the default I keep going back and forth on. Whatever a tool does out of the box is what most teams end up running in production, so I'd rather hear from people with real traffic before I pick.

What does your beforeSend do with event.request.data right now: nothing, a key denylist, or delete the whole body?

Log what your Sentry SDK actually sends before you leave SaaS

Amorizz — Sun, 25 Oct 2026 13:28:29 +0000

I was about to tell someone "just swap the DSN" when I realized I couldn't say what the DSN actually receives. So I put a tap behind the DSN of a small Express app and sent it 100 requests. The SDK shipped 546 KB, and the one error event in there was 7.6 KB of it.

Last week I pointed the official SDK at a 36-line homemade ingest to see what breaks. This week is the other half of that question: if I point my DSN at something that only does errors, what do I lose?

This post is the tap, the numbers, and the init I'd use if I only wanted errors. It works the same whether you're moving to self-hosted Sentry, GlitchTip, something you wrote, or just trying to cut your SaaS bill without moving at all.

Everything below ran on 2026-10-25 with Node 20.19.2, @sentry/node 11.9.0 and Express 4.22.3. SDK defaults change between majors, so run the tap against your own version before you trust my table.

TL;DR

The official SDK sends several kinds of data to the DSN, not just errors. In my "everything on" setup, 100 requests produced 305 spans, 100 log records, 90 metric records, 2 cron check-ins and 1 error event. Spans were 81% of the bytes.
An errors-only init (no tracesSampleRate line, drop logs and metrics in beforeSendLog / beforeSendMetric) took the same traffic from 546,514 bytes on the wire to 9,111. About 60x less.
Three things surprised me: tracesSampleRate: 0 still counted 512 spans, 10 failed checkouts showed up as 1 event, and every error carried the raw request body.

The mental model: the DSN gets envelopes, envelopes hold typed items

Every SDK request to your DSN is an "envelope": one header line, then pairs of lines (item header, item payload). The item header has a type. That type is what decides which product feature the data feeds on the SaaS side.

Item type	What it feeds in SaaS Sentry	If your new host only does errors
`event`	Issues: the errors themselves	Kept. This is the part every errors-only tool does
`span`	Tracing and performance views	Gone. Send them nowhere, or to a tracing backend
`log`	Logs	Gone, unless your target has a logs product
`trace_metric`	Metrics	Gone
`check_in`	Crons monitors ("the nightly job didn't run")	Check your target. If it ignores them, you lose missed-job alerts
`sessions`	Release health, crash-free rates	Check your target
`client_report`	Counts of what the SDK dropped and why	Worth reading even if nothing shows it

So "what do I keep" has a mechanical answer. Run your app against something that logs item types, and look at the column on the right.

The tap: 30 lines that log every item type

This is a fake DSN host. It accepts anything, unzips gzip bodies, and writes one JSON line per item with its type, its size and a few keys. It stores nothing else.

// tap.js: a fake DSN host. Logs one line per envelope item: type, size, a few keys.
const http = require('http'), zlib = require('zlib'), fs = require('fs');
const OUT = process.env.OUT || 'tap.jsonl';
let envelopes = 0;

http.createServer((req, res) => {
  const chunks = [];
  req.on('data', c => chunks.push(c));
  req.on('end', () => {
    let buf = Buffer.concat(chunks);
    const wire = buf.length, env = ++envelopes;
    if (req.headers['content-encoding'] === 'gzip') buf = zlib.gunzipSync(buf);
    const lines = buf.toString().split('\n');
    // envelope = header line, then pairs of (item header, item payload)
    for (let i = 1; i < lines.length && lines[i]; i += 2) {
      const head = JSON.parse(lines[i]);
      const body = lines[i + 1] || '';
      let payload = {}; try { payload = JSON.parse(body); } catch {}
      if (head.type === 'event') fs.appendFileSync('events-full.jsonl', body + '\n');
      fs.appendFileSync(OUT, JSON.stringify({
        type: head.type, bytes: Buffer.byteLength(body), env, wire,
        items: payload.items ? payload.items.length : undefined,
        keys: Object.keys(payload).sort(),
        reason: payload.discarded_events,
      }) + '\n');
    }
    res.end('{}');
  });
}).listen(9999, '127.0.0.1', () => console.log('tap on :9999'));

It also keeps full error events in events-full.jsonl, because I wanted to see what's inside them. More on that below.

A 17-line summary.js groups those lines by type, adds up bytes, sums the wire size per envelope and prints any client report reasons. Nothing clever: count, sum, sort.

To point any app at it, change the DSN host and keep the key and project id shape:

SENTRY_DSN=http://devkey@127.0.0.1:9999/1

The key doesn't matter, the tap never checks it. Don't run this on anything but localhost.

The app: a small Express API with everything turned on

The app has two routes. /products writes a log line and bumps a counter. /checkout throws a TypeError when the client forgets the coupon. There's also a nightly job wrapped in a cron monitor. The Sentry config is picked from a profile so I could rerun the same traffic with different inits:

const Sentry = require('@sentry/node');
const configs = {
  saas: { tracesSampleRate: 1.0 },
  errors: {
    // errors-only: no tracesSampleRate at all, drop logs and metrics before they leave
    beforeSendLog: () => null,
    beforeSendMetric: () => null,
  },
};
Sentry.init({ dsn: 'http://devkey@127.0.0.1:9999/1', release: 'shop@1.4.2', environment: 'lab',
  ...configs[process.env.PROFILE || 'saas'] });

const express = require('express');
const app = express();
app.use(express.json());

app.get('/products', (req, res) => {
  Sentry.logger.info('listed products', { count: 3 });
  Sentry.metrics.count('products.listed', 1);
  res.json([{ id: 1 }, { id: 2 }, { id: 3 }]);
});
app.post('/checkout', (req, res) => {
  Sentry.logger.info('checkout started', { items: req.body.items.length });
  const code = req.body.coupon.code; // TypeError when the client forgets the coupon
  res.json({ ok: true, code });
});

Sentry.setupExpressErrorHandler(app);
app.use((err, req, res, next) => res.status(500).json({ error: 'internal' }));

app.post('/admin/nightly', async (req, res) => {
  await Sentry.withMonitor('nightly-invoices', async () => { /* work */ },
    { schedule: { type: 'crontab', value: '0 3 * * *' } });
  res.json({ ok: true });
});

The load comes from a separate process: 100 sequential requests, every 10th one a checkout with a cookie and a JSON body that includes card_last4, then one call to the nightly job. Then the app flushes and exits. Running it in a separate process matters. My first version fired the requests from inside the app process, and the client's own fetch breadcrumbs piled up into every error event, which made the events look much bigger than they are.

npm i @sentry/node@11.9.0 express@4.22.3
node tap.js &
PROFILE=saas node app.js &
node load.js
node summary.js tap.jsonl

Result 1: the "everything on" init sent 546 KB, 1.2% of it errors

With tracesSampleRate: 1.0 and the default integrations, here's what the tap saw:

statuses: {"200":90,"500":10}
type            items     bytes   share
span               61    503499   80.9% (305 records inside)
log                 1     62418   10.0% (100 records inside)
trace_metric        1     47980    7.7% (90 records inside)
event               1      7600    1.2%
check_in            2       661    0.1%
client_report       1       110    0.0%
total              67    622268
envelopes: 67, bytes on the wire (gzip where the SDK used it): 546514
client_report: [{"reason":"event_processor","category":"error","quantity":9}]

Two things I didn't expect. First, I never turned logs or metrics on. Calling Sentry.logger.info and Sentry.metrics.count was enough for 11.9.0 to batch and send them. Second, spans dominate everything else, at roughly 1.6 KB each, about 3 per request.

If you leave SaaS for an errors-only host and keep this init, all of that still leaves your servers. Where it ends up depends on the host. The best case is that it's thrown away on arrival. Either way you pay for the CPU and bandwidth to build and send it.

Result 2: the errors-only init sent 9.1 KB

Same traffic, same app, with the errors profile:

type            items     bytes   share
event               1      7234   88.2%
check_in            2       590    7.2%
client_report       1       236    2.9%
sessions            1       145    1.8%
total               5      8205
envelopes: 5, bytes on the wire (gzip where the SDK used it): 9111
client_report: [{"reason":"before_send","category":"log_item","quantity":100},{"reason":"before_send","category":"trace_metric","quantity":90},{"reason":"event_processor","category":"error","quantity":9}]

546,514 bytes down to 9,111. The init change is three things: delete the tracesSampleRate line, and return null from beforeSendLog and beforeSendMetric. The client report confirms it, with 100 log items and 90 metrics dropped by before_send, on purpose.

Notice what's still there: the two check_in items and a sessions item. Those aren't errors. If the host you're moving to doesn't handle cron check-ins, your "nightly job didn't run" alert dies quietly on migration day. That's the row I'd check first. (The saas run didn't send a sessions item at all and the errors runs did. I didn't dig into why, so I'm only reporting it.)

Gotcha: `tracesSampleRate: 0` is not the same as deleting it

I ran a third profile, errors-only plus tracesSampleRate: 0. No spans hit the wire, so it looks the same. The client report says otherwise:

client_report: [{"reason":"sample_rate","category":"span","quantity":512},{"reason":"before_send","category":"log_item","quantity":100},...]

With the rate at 0, tracing is still configured. The SDK created spans, sampled every one of them out, and counted 512 as dropped by sample_rate for the same traffic. The wire total went from 9,111 to 9,816 bytes, mostly because that report is bigger. Small on the wire, but the span instrumentation still runs on every request. If you want tracing off, delete the line instead of setting it to 0.

Gotcha: 10 failed checkouts, 1 event

All three runs above stored 1 error event for 10 failing requests. The client report explains it: event_processor, error, quantity 9.

That's the Dedupe integration. It drops an error when it looks the same as the previous one the SDK captured (same message and stack). My errors came every 10th request with nothing else failing in between, so 9 of 10 were dropped before they left the process. Removing it:

integrations: all => all.filter(i => i.name !== 'Dedupe')

gave 10 events and 72,289 bytes, about 7.2 KB per event.

Why this matters when you switch: people compare "errors per day" between the old and new tracker to check the migration worked. Both numbers are after the SDK's dedupe and sampling. If the counts don't match, check the client reports before you blame the new host. I keep Dedupe on, by the way. 10 identical events in a row aren't 10 bugs.

Gotcha: every error carried the request body

This is the one I'd fix even if I never migrated. Here's part of the stored error event from the errors-only run:

request.data = {"items":[1,2],"card_last4":"4242"}
request.cookies = {'session': '[Filtered]'}
user = {'ip_address': '::ffff:127.0.0.1'}

The SDK filtered the cookie value on its own. It did not filter the JSON body, and it attached the client IP. I never set sendDefaultPii. With SaaS that data sits with your vendor. Self-hosted, it sits on your disk, in your backups, for as long as your retention says. Decide on purpose:

beforeSend(event) {
  if (event.request) {
    delete event.request.cookies;
    delete event.request.data;
    delete event.request.headers?.cookie;
  }
  if (event.user) delete event.user.ip_address;
  return event;
}

With that, the event went from 7,234 to 7,099 bytes. The stored event had only headers, method and url under request, and an empty user. Most of an error event is the exception (4,026 bytes of JSON here) and the modules list (1,588), not the request, so scrubbing costs you almost nothing when you debug.

What you keep, what you give up

From this run, the honest subset when you go errors-only:

You keep: errors with stack traces, release and environment tags, request method and URL, breadcrumbs, and the SDK's own drop accounting in client reports if your host reads them.
You give up: tracing, logs and metrics as products. If you used any of them, plan a replacement before the switch, not after.
Check before switching: cron check-ins and release health sessions. They still get sent with an errors-only init.
Not covered here: browser Session Replay, profiling and mobile SDKs. I only tested @sentry/node. Run the same tap against your browser bundle before you trust this list for the frontend.

Failure checklist

You left tracesSampleRate: 0 in. Nothing on the wire, but tracing still runs. Delete the line.
Logs and metrics leave without an opt-in. In 11.9.0, calling Sentry.logger or Sentry.metrics was enough. Drop them in beforeSendLog / beforeSendMetric, or remove the calls.
Cron monitors break silently. withMonitor keeps sending check_in items to the DSN. Confirm your new host handles them, or move the monitor somewhere that does.
Error counts don't match after the switch. Compare client reports, not dashboards. Dedupe and sampling run before anything leaves the process.
The tap shows bigger events than prod. If your load generator runs inside the app process, its own breadcrumbs end up in every event. Use a separate process.
Request bodies in your error store. The default Node setup attached the JSON body and client IP. Scrub in beforeSend before you own the disk.
The tap breaks on attachments. It assumes one JSON line per item. Attachments carry a length header and binary payloads, so add parsing for them if your app sends files.

I'm building Epure, a lightweight self-hosted error tracker for small teams, and it only keeps the event rows of that first table. That's why I ran this before telling anyone to swap a DSN. The tap doesn't care where you're going, though, so run it whatever you pick.

What tracesSampleRate is in your production Sentry.init right now?

Build vs buy exception monitoring, tested with a 36-line DIY ingest

Amorizz — Tue, 20 Oct 2026 05:47:55 +0000

I almost wrote my own error tracker once. Not as a product, just a small endpoint so a side project could stop paying for a plan it barely used. This week I finally built that small version, pointed the official Sentry SDK at it, and threw errors at it until it showed me where the real work is.

(Housekeeping: this was meant to go out on Sunday. The box I write and test these posts on spent the weekend stuck at boot. Fitting, for a post about running your own infrastructure.)

Everything below ran on 2026-10-20 on a Linux box with Node 20.19.2 and @sentry/node 11.8.0. The "buy" numbers come from public pages I read the same day: Sentry's pricing page, the self-hosted Sentry docs and its docker-compose.yml at release 26.10.0, and GlitchTip's install docs. Prices and requirements move, so check them again before you decide anything.

TL;DR

An ingest that accepts real Sentry SDK events, groups them and writes them to disk fits in 36 lines of Node. Getting events in is the small part.
My DIY version showed three problems in a few seconds of testing: grouping by message turned 3 bugs into 32 issues, a 502-error burst delivered 63 events, and 7 errors thrown while it was down never arrived.
If you buy instead, the choice is SaaS (Sentry Team from $26/mo with 50K errors), heavy self-hosting (self-hosted Sentry asks for 4 cores and 16 GB RAM plus 16 GB swap) or light self-hosting (GlitchTip runs on Postgres plus one service).

The mental model: an error tracker is four jobs

An error tracker does four jobs. Ingest takes events off the wire. Grouping turns thousands of events into the dozen bugs they really are. Storage keeps them long enough to be useful and short enough to fit on disk. Alerting tells a human, once, at the right moment.

The SDK side is already solved for you. The official Sentry SDKs send events as "envelopes" over HTTP to whatever host the DSN points at, so any server that reads that format can receive them. That's why several trackers accept the official SDKs, and it's also why "just build it" looks cheap. Ingest is the first job and the smallest one.

The build option: 36 lines that accept real SDK events

This is the whole thing. It checks the DSN key, unzips the envelope, keeps items that carry an exception, and groups them by a fingerprint built from the error type plus the innermost three in-app stack frames.

// catcher.js: the "build" option. Accepts Sentry SDK envelopes, groups errors, keeps them in a file.
const http = require('http'), zlib = require('zlib'), fs = require('fs'), crypto = require('crypto');
const KEY = process.env.KEY || 'devkey', OUT = process.env.OUT || 'events.jsonl';
const issues = new Map(); // fingerprint -> { title, count, first, last }

function fingerprint(ex) {
  const frames = (ex.stacktrace && ex.stacktrace.frames) || [];
  const inApp = frames.filter(f => f.in_app).slice(-3); // innermost 3 app frames
  const key = ex.type + '|' + inApp.map(f => `${f.module || f.filename}:${f.function}`).join('>');
  return crypto.createHash('sha1').update(key).digest('hex').slice(0, 12);
}

http.createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/issues') {
    return res.end(JSON.stringify([...issues.values()].sort((a, b) => b.count - a.count), null, 1));
  }
  const auth = (req.headers['x-sentry-auth'] || '') + req.url;
  if (!auth.includes(`sentry_key=${KEY}`)) { res.statusCode = 401; return res.end(); }
  const chunks = [];
  req.on('data', c => chunks.push(c));
  req.on('end', () => {
    let buf = Buffer.concat(chunks);
    if (req.headers['content-encoding'] === 'gzip') buf = zlib.gunzipSync(buf);
    for (const line of buf.toString().split('\n')) {
      let item; try { item = JSON.parse(line); } catch { continue; }
      if (item.discarded_events) console.log('client report:', JSON.stringify(item.discarded_events));
      if (!item.exception) continue;
      const ex = item.exception.values.at(-1);
      const fp = fingerprint(ex), now = new Date().toISOString();
      const i = issues.get(fp) || { fp, title: `${ex.type}: ${ex.value}`, count: 0, first: now };
      i.count++; i.last = now; issues.set(fp, i);
      fs.appendFileSync(OUT, JSON.stringify(item) + '\n');
    }
    res.end('{}');
  });
}).listen(9999, '127.0.0.1', () => console.log('catcher on :9999'));

The client report line was not in my first version. I'll get to why I added it.

The app is the official SDK with the DSN pointed at localhost, and three kinds of errors a real app throws: a "not found" with an id in the message, a TypeError from a missing field, and a JSON parse error.

// app.js
const Sentry = require('@sentry/node');
Sentry.init({ dsn: 'http://devkey@127.0.0.1:9999/1', tracesSampleRate: 0,
  release: 'demo@1.0.0', debug: !!process.env.DEBUG });

function loadUser(id) { throw new Error(`User ${id} not found`); }
function renderInvoice(inv) { return inv.total.toFixed(2); } // TypeError when total is missing
function parseConfig(s) { return JSON.parse(s); }

const N = Number(process.env.N || 30);
for (let i = 0; i < N; i++) {
  try { loadUser(1000 + i); } catch (e) { Sentry.captureException(e); }
}
try { renderInvoice({}); } catch (e) { Sentry.captureException(e); }
try { parseConfig('{bad'); } catch (e) { Sentry.captureException(e); }

Sentry.flush(5000).then(ok => console.log('flush ok:', ok, 'sent', N + 2));

To run it:

npm i @sentry/node@11.8.0
node catcher.js &
node app.js
curl -s localhost:9999/issues

Test 1: grouping is the actual product

Grouping is what separates an error tracker from a log file, and it's the first thing a DIY version gets wrong. Here's the default run, 30 "not found" errors plus the two others (timestamps trimmed from the /issues output):

flush ok: true sent 32
[
 { "fp": "e5f026ad1f00", "title": "Error: User 1000 not found", "count": 30 },
 { "fp": "51a3afcc1f57", "title": "TypeError: Cannot read properties of undefined (reading 'toFixed')", "count": 1 },
 { "fp": "38027f816eae", "title": "SyntaxError: Expected property name or '}' in JSON at position 1", "count": 1 }
]
events stored: 32
bytes on disk: 131768
groups if you key on type+message: 32

The stack fingerprint got it right: 3 bugs, 3 issues. The last line is what you get with the first idea everyone has, keying on type plus message. Every user id becomes its own "bug", and 3 problems show up as 32 rows.

My fingerprint is a compromise too, and it fails in the other direction. Rename loadUser in a refactor and the old issue looks new. Minified browser code is worse, because the function names are t and n until you apply source maps. I didn't test the browser case here, but that's why real trackers carry a source map pipeline and let you override fingerprints per project.

Test 2: 500 errors in a tight loop, 63 arrived

Bursts are where a homemade ingest loses data without telling you. Same app, N=500:

flush ok: true sent 502
events stored: 63 of 502 sent
bytes on disk: 259554

This one isn't the catcher's fault. With DEBUG=1 the SDK explains itself:

      1 Sentry Logger [error]: Skipped sending event because buffer is full.
    439 Sentry Logger [log]: Recording outcome: "queue_overflow:error"

The SDK transport keeps a bounded queue of in-flight requests. In @sentry/core it's DEFAULT_TRANSPORT_BUFFER_SIZE = 64 (in transports/base.js). A tight loop fills it, and new events are dropped and counted as queue_overflow. That's the right call for the SDK: your app shouldn't fall over because its error reporting got busy.

Here's the part that matters for build vs buy. After a drop, the SDK sends a client_report envelope item saying how many events it lost. My first catcher skipped every item without an exception, so it threw away the only message that said anything was missing. With the one extra line, the catcher log shows:

client report: [{"reason":"queue_overflow","category":"error","quantity":438}]
client report: [{"reason":"queue_overflow","category":"error","quantity":1}]

63 stored plus 439 dropped is 502. A tracker that reads these reports can tell you "you lost 439 events in this burst". A tracker that doesn't shows 63 and a calm dashboard.

Size is the other surprise. Those 63 events took 259,554 bytes as JSON lines, about 4 KB each, because every event carries its stack, modules list and context. Multiply by your worst day and you have your disk and retention budget. GlitchTip's docs give their own rough figure: about 30 GB of disk for an instance taking 1 million events a month.

Test 3: the tracker went down, the errors went with it

Your error tracker's uptime becomes part of your app's uptime the day you build it. I stopped the catcher, ran the app with N=5, then started the catcher again:

flush ok: true sent 7
events stored after catcher came back: 0

And with DEBUG=1:

     11 Sentry Logger [error]: Error while sending envelope: Error: connect ECONNREFUSED 127.0.0.1:9999
      7 Sentry Logger [log]: Recording outcome: "network_error:error"
      2 Sentry Logger [warn]: Dropping client report. Will not send outcomes (reason: network_error).

Two things to notice. flush() resolved true even though nothing arrived, so it means "the queue drained", not "the events made it". And with the default Node transport, the 7 errors were not queued for later. I didn't try any offline or retry transport options, so I can't speak for them. With the defaults, an outage of your tracker is a hole in your history. It even drops the client report about the hole.

The worst time for that is also the most likely one: the same incident that takes down your app box takes down the tracker you put next to it.

The buy options, priced from public docs

If you'd rather not own those three problems, there are three realistic ways to buy your way out, and they cost very different things.

Option	What you run	Published minimum	Cost model
Sentry SaaS, Team plan	Nothing	n/a	$26/mo billed annually with 50K errors; then $0.0003625 per error from 50K to 100K
Self-hosted Sentry	53 compose services (28 with `COMPOSE_PROFILES=errors-only`)	4 CPU, 16 GB RAM + 16 GB swap, 20 GB disk	Your box and your upgrades; FSL license
GlitchTip	Postgres 14+ and one service, Valkey optional	512 MB RAM recommended, 256 MB all-in-one	Your box; they ask self-hosters for a $5/user/mo donation
Build	Your code	Whatever you pick	Your time, forever

Some detail behind those rows:

Sentry SaaS. The Developer plan is free for one user. On the published Team table, 100K errors a month comes to $26 + 50,000 x $0.0003625, about $44/mo. Their calculator may round differently. You get everything their platform does, and your error data lives with them.
Self-hosted Sentry. The docs describe it as "the Business plan without any software limitations and no paid tier", with a few exceptions. One that matters for this post: spike protection isn't available self-hosted, because it's tied to billing quotas. The install script's hard floor for the errors-only profile is 7000 MB of RAM and 2 CPUs, against 14000 MB and 4 CPUs for the full set.
GlitchTip. The smallest footprint of the three. It also does uptime checks and log ingestion now, which you can switch off with environment variables.

You can count the self-hosted services yourself in a minute:

curl -sL https://raw.githubusercontent.com/getsentry/self-hosted/master/docker-compose.yml -o compose.yml
python3 count-services.py

# count-services.py (needs PyYAML)
import yaml
svcs = yaml.safe_load(open("compose.yml"))["services"]
always = [n for n, s in svcs.items() if not s.get("profiles")]
print("services defined:", len(svcs))
print("started with COMPOSE_PROFILES=errors-only:", len(always))

services defined: 53
started with COMPOSE_PROFILES=errors-only: 28

Those are compose service definitions, and two of them are cleanup jobs, so it isn't the same as 28 always-busy processes. It's still 28 things to keep upgraded together.

How I'd decide

Pick by what you can't give up, not by what the dashboard looks like.

Data can leave your servers, and you're a small team. Buy SaaS. At low volume it's probably cheaper than the hours you'd spend on any of the other rows.
Data has to stay on your servers, and you have a spare 16 GB box and someone who enjoys upgrades. Self-hosted Sentry, probably with the errors-only profile.
Data has to stay on your servers, and the box is small. A light self-hosted tracker: Postgres plus one or two containers.
You need ingest and a list, nothing else, and you accept the checklist below. Build.

If you build anyway: the failure checklist

Fingerprint on stack frames, not messages. Ids, emails and timestamps in messages will explode your issue count. My run: 32 groups by message, 3 by stack.
Read client_report items. They're the only signal that the SDK dropped events before they reached you.
Send rate limits back. The SDK transport reads rate limit headers from your responses (updateRateLimits in the same transports/base.js), so your server can tell clients to back off instead of drowning.
Monitor the monitor from somewhere else. Test 3 showed silent loss when the ingest is down, and flush() still returned true.
Budget disk per event. About 4 KB per Node event in my test. Set retention before the disk sets it for you.
Treat the DSN key as public. In a browser app anyone can read it from your bundle. The key check in my catcher stops typos, not abuse, so add origin checks and per-key rate limits.
Plan for source maps if you have browser JS, or your stack fingerprints will be minified names.

Where I land

I'm building Epure, which sits in the light self-hosted row: errors only, the official Sentry SDKs via a DSN swap, a Rust binary plus Postgres 16 in two containers. It doesn't do APM, replay, profiling or logs, and not every SDK feature has a counterpart. If that row fits you, the docs show the DSN swap. The tests above ran against my throwaway catcher, not Epure, so they say nothing about it either way.

Ingest is an afternoon. Grouping, drop accounting and staying up while your app falls over are the actual job. Build only the part you're willing to be paged for, and buy or self-host the rest.

Sentry caught my Express error. The request still hung forever

Amorizz — Fri, 16 Oct 2026 20:10:32 +0000

I assumed that with an error SDK installed, an async Express route that throws would at least give the user a 500. I had never tested it, so I did. The result surprised me.

The short version: on Express 4, an async handler that throws after an await is reported by Sentry, keeps the process alive, and never answers the request. The client just waits.

Everything below was run on 2026-10-16 on a Linux box with Node 20.19.2, Express 4.22.3, Express 5.3.0 and @sentry/node 11.8.0. The "Sentry server" is a throwaway 15-line Node script on localhost, so nothing leaves the machine. Outputs are pasted from the runs, with log noise trimmed.

TL;DR

Express 4 does not look at the promise your async handler returns. A throw after an await becomes an unhandled rejection, not a next(err) call.
Without an error SDK, Node 20 kills the process. With @sentry/node initialised, the process survives, the error is reported, and the request hangs. That is the nasty one, because everything looks healthy.
Fix on Express 4: wrap the handler and .catch(next). Or move to Express 5, where rejected promises go to your error handler on their own. The Express docs describe the Express 5 behaviour in their error handling guide.

The smallest app that reproduces it

// app.js
const express = require('express');
const app = express();
const sleep = (ms) => new Promise(r => setTimeout(r, ms));

if (process.env.SENTRY) {
  const Sentry = require('@sentry/node');
  global.Sentry = Sentry;
  Sentry.init({ dsn: 'http://key@127.0.0.1:9999/1', tracesSampleRate: 0 });
}

app.get('/sync',  () => { throw new Error('sync boom'); });
app.get('/async', async () => { await sleep(10); throw new Error('async boom'); });
app.get('/ok',    (req, res) => res.send('ok'));

if (process.env.SENTRY) global.Sentry.setupExpressErrorHandler(app);

app.use((err, req, res, next) => {
  console.log('[error middleware]', err.message);
  res.status(500).send('handled');
});

app.listen(3000);

Two routes that throw. One throws right away. The other throws after a 10 ms await, which is what any real handler does the moment it calls a database.

And the "Sentry server", just enough to see what arrives:

// sink.js
const http = require('http'), zlib = require('zlib');
http.createServer((q, r) => {
  const b = [];
  q.on('data', c => b.push(c));
  q.on('end', () => {
    let buf = Buffer.concat(b);
    try { buf = zlib.gunzipSync(buf); } catch {}
    for (const l of buf.toString().split('\n').filter(Boolean)) {
      try {
        const j = JSON.parse(l);
        if (j.exception) {
          const e = j.exception.values[0];
          console.log('SINK event:', e.type, e.value, 'mech=', e.mechanism && e.mechanism.type);
        }
      } catch {}
    }
    r.end('{}');
  });
}).listen(9999);

The test is the same every time: start the app, curl the sync route, curl the async route with -m 5, then curl /ok to see if the server is still alive.

Run 1: Express 4, no SDK

GET /sync  -> http=500 curl_exit=0
GET /async -> http=000 curl_exit=52
GET /ok afterwards -> (curl_exit=7)
process: dead

[error middleware] sync boom
/app.js:10
app.get('/async', async () => { await sleep(10); throw new Error('async boom'); });
                                                       ^
Error: async boom
Node.js v20.19.2

The sync throw gets a clean 500 from my error middleware. The async one never reaches it. Node 20 treats the unhandled rejection as fatal and the whole process exits. The client sees an empty reply (curl exit 52) and the next request is refused (exit 7).

That is loud, at least. Your process manager restarts it and you notice.

Run 2: Express 4, with Sentry

Same app, SENTRY=1:

GET /sync  -> http=500 curl_exit=0
GET /async -> http=000 curl_exit=28
GET /ok afterwards -> ok (curl_exit=0)
process: alive

SINK event: Error sync boom  mech= auto.middleware.express
SINK event: Error async boom mech= auto.node.onunhandledrejection

Exit code 28 is curl giving up after my 5 second limit. The error was reported. The process did not crash. And the request got nothing.

Two things I noticed in the output:

The sync error came in through auto.middleware.express, meaning the Express error handler saw it. The async one came in as auto.node.onunhandledrejection. Sentry caught it, but from the global rejection hook, not from Express.
The app log shows Node's usual "This error originated either by throwing inside of an async function without a catch block" warning, and then nothing else. The process keeps running.

To check this is a real hang and not a slow response, I left one request open for 25 seconds. curl timed out at 25.0 s with no status code, and ss still showed one established connection on port 3000 twenty seconds in. The socket stays open until the client or a proxy gives up.

So the SDK did its job. It reported the error and kept my process alive. What it cannot do is write a response for a request Express no longer knows about. In production that "stays up" behaviour is what turns a crash into a silent hang, and the hang is probably cut off by your load balancer's idle timeout, not by anything in your app. I did not test a proxy here, so I can't tell you what yours does.

Run 3: Express 5

I made a second project with Express 5.3.0 and the same app.js:

GET /sync  -> http=500 curl_exit=0
GET /async -> http=500 curl_exit=0
GET /ok afterwards -> ok (curl_exit=0)
process: alive

[error middleware] sync boom
[error middleware] async boom

With Sentry on, both events arrive with mech= auto.middleware.express. The async error goes through my error middleware, the client gets a 500, and nothing hangs.

This matches what the Express docs say: route handlers that return a promise call next(value) when they reject, and async functions always return a promise. No code change needed, only the version.

The fix if you are stuck on Express 4

A small wrapper that hands rejections to next:

const wrap = (fn) => (req, res, next) =>
  Promise.resolve(fn(req, res, next)).catch(next);

app.get('/async', wrap(async () => {
  await sleep(10);
  throw new Error('async boom');
}));

Run on Express 4.22.3 with Sentry initialised:

wrapped express4: http=500 time=0.029606s
SINK event: Error async boom mech= auto.middleware.express

500 in about 30 ms, and the event now comes through the Express error handler, same as the sync case. Sentry's setupExpressErrorHandler has to go after your routes and before your own error middleware, which is where it was in my app.js.

What I would check in your app today

Which Express version are you on? npm ls express. If it says 4.x, every async handler without a catch is a hang waiting for a bad day.
Grep for handlers that look safe but are not. app.get('/x', async (req, res) => { with no try and no wrapper. Also setTimeout and event emitter callbacks, which Express never sees on any version. The Express docs say to catch those yourself and call next(err).
Look at the error source in your tracker. In my run the hung request showed up as an unhandled rejection, not as an Express error. If your tracker has a mechanism or "handled" field, filter for the unhandled rejection ones. Those are the routes without a wrapper.
Add a response timeout in front of the app. I did not benchmark this, so I will not give you a number. The point is that a hung request should end with a 504 from your proxy in seconds, not a user closing the tab.
Do not turn off the crash to make the noise stop. Crashing was the better failure mode in Run 1. A restart loop is easier to spot than a silent hang.

Where I run this

I build Epure, a small self-hosted error tracker: one Rust binary and Postgres 16, errors only, and you point the official Sentry SDK at it by swapping the DSN. The results above come from the throwaway sink, not from Epure, so they say nothing about it either way. If you want to see what that DSN swap looks like, the docs have it.

What this does not show

One Node version (20.19.2), one SDK version. A different @sentry/node major or Node's --unhandled-rejections flag could change the hang.
A single route, tested with curl. I did not run a real database or a proxy in front.
I read the Express 5 behaviour from the docs and confirmed it with the run above. I did not read the router source.

Which Express version is your production API on right now, and does it have a request timeout in front of it?

Your .env hit git. Here's how I keep secrets on a VPS

Amorizz — Thu, 15 Oct 2026 13:31:31 +0000

I pushed a small Node + Postgres Compose stack to a private repo and only then noticed .env was in the first commit. DATABASE_URL had a real password. Adding .gitignore afterward did nothing useful: Git still had the blob, and git check-ignore stayed quiet while the path was tracked.

This is the layout I use on a VPS now. Everything below was run on the box on 2026-10-15 with git 2.47.3, Docker Engine 26.1.5, Compose 2.26.1 and Node 20. Numbers and command output are from those runs.

TL;DR

Put .env in .gitignore before the first commit. Commit .env.example with fake values. If .env already landed in git, git rm --cached only stops future commits. The old blob stays until you rewrite history or rotate the secret.
On the VPS, keep real secrets in a file mode 600 (for example secrets.env) and point Compose at it with env_file. Do not put ${POSTGRES_PASSWORD} in the YAML if that value comes from a project .env you might commit.
Compose interpolates ${...} in the Compose file itself, including inside command: strings. A JS template literal like `token=${t}` becomes empty before the container starts. Prefer a script file, or write $$t.

What are you actually protecting?

Three different places can hold the same password. Fixing one does not fix the others.

Place	What goes wrong	What fixes it
Git history	Clone, fork, or a future public flip exposes the blob forever	Never commit it; if you did, rotate and treat history as burned
File on the VPS	Any local account can read a `644` secrets file	`chmod 600`, owner = the deploy user
Process environment	`docker compose config`, crash dumps, child processes	Prefer `env_file`; avoid baking secrets into resolved YAML via `${}`

.gitignore only affects the first column, and only for paths that are not already tracked.

How does .env end up in git?

I reproduced the failure in a throwaway repo. First commit included .env. Then I added:

.env
.env.*
!.env.example

After that commit, Git still listed .env:

$ git check-ignore -v .env
# (no output: still tracked, so ignore rules do not apply)

$ git ls-files | grep env
.env
.env.example

$ git show "$(git rev-list --max-parents=0 HEAD):.env" | head -3
DATABASE_URL=postgres://app:[REDACTED]@db:5432/app
EPURE_DSN=https://[REDACTED]@ingest.example.com/1
NODE_ENV=production

git rm --cached .env plus a new commit makes check-ignore work again:

$ git rm --cached .env
$ git commit -m "stop tracking .env (too late for history)"
$ git check-ignore -v .env
.gitignore:1:.env   .env

The first commit still has the file. git rev-list --objects --all still showed the .env blob. Ignore files are not a time machine.

The correct order is boring and it works. Create .gitignore and .env.example first, then add:

$ git status --short
A  .env.example
A  .gitignore
A  app.js
# .env is absent from the index

$ git check-ignore -v .env
.gitignore:1:.env   .env

$ git ls-files
.env.example
.gitignore
app.js

No secret string in git grep across $(git rev-list --all).

What goes on the VPS?

I keep two files next to the Compose project:

.env.example in git: keys and fake values, so a teammate knows what to create.
secrets.env on the host only: real values, mode 600, listed in .gitignore.

$ chmod 600 secrets.env
$ stat -c '%a %n' secrets.env .env.worldreadable
600 secrets.env
644 .env.worldreadable

On the lab box, Python reported other_read=False for 600 and other_read=True for 644. Same owner can read both. Every other account on the VPS can read the 644 copy.

A minimal systemd unit can load the same file:

[Service]
WorkingDirectory=/opt/app
EnvironmentFile=/etc/app/secrets.env
ExecStart=/usr/bin/node /opt/app/server.js

Same rule: /etc/app/secrets.env is 600, not in git.

Compose: env_file vs ${PASSWORD} in the YAML

Compose loads a project .env for interpolation in the Compose file. That is separate from what the container sees as process.env.

Bad shape (host .env supplies ${POSTGRES_PASSWORD}, and the resolved config prints it):

# compose.bad.yml
services:
  db:
    image: postgres:16.15-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: app
  app:
    image: node:22.23.3-alpine
    environment:
      DATABASE_URL: postgres://app:${POSTGRES_PASSWORD}@db:5432/app
      APP_SECRET: ${APP_SECRET}

$ docker compose -f compose.bad.yml config | grep -E 'PASSWORD|DATABASE_URL|APP_SECRET'
      APP_SECRET: from-host-dotenv
      DATABASE_URL: postgres://app:HostInterpolatedSecret@db:5432/app
      POSTGRES_PASSWORD: HostInterpolatedSecret

That was 2 plaintext hits for HostInterpolatedSecret in the full config dump. Fine for local debugging. Bad to paste into an issue, a CI log, or a screenshot.

Better shape: put secrets in secrets.env, mount them with env_file, and keep the YAML free of ${SECRET}:

# compose.yml
services:
  db:
    image: postgres:16.15-alpine
    env_file:
      - secrets.env
    environment:
      POSTGRES_USER: app
      POSTGRES_DB: app
      # POSTGRES_PASSWORD comes from secrets.env

  app:
    image: node:22.23.3-alpine
    working_dir: /app
    volumes:
      - ./print-env.js:/app/print-env.js:ro
    env_file:
      - secrets.env
    environment:
      POSTGRES_USER: app
      POSTGRES_HOST: db
      POSTGRES_DB: app
    command: ["node", "print-env.js"]

// print-env.js
const u = process.env.POSTGRES_USER;
const p = process.env.POSTGRES_PASSWORD;
const h = process.env.POSTGRES_HOST;
const d = process.env.POSTGRES_DB;
console.log("password_in_process_env", Boolean(p));
console.log("password_len", (p || "").length);
console.log("password_equals_envfile", p === "EnvFileOnlySecret");

With the project .env renamed away so only secrets.env could supply the password:

$ docker compose -f compose.better.yml run --rm --no-deps app
password_in_process_env true
password_len 17
NODE_ENV production
password_equals_envfile true
password_equals_host_dotenv false

password_len 17 matches EnvFileOnlySecret. The host-interpolated value never appeared.

The ${} gotcha inside command: strings

I put a JS template literal in the Compose command once. Compose warned and emptied it:

level=warning msg="The \"t\" variable is not set. Defaulting to a blank string."

Resolved command became console.log('template', \token=); with no t. The container never saw ${t}. Fix: move the script to a file (as above), or escape as $$t in the YAML so Compose emits a literal $t for the shell or Node source.

Failure checklist

.gitignore after the first commit. Tracked paths ignore the ignore file. Run git ls-files .env and git check-ignore -v .env.
git rm --cached without rotating. Future commits are clean. Clones of the old history still have the password.
.env.example missing or filled with real values. People copy the wrong file. Keep fake placeholders only.
Mode 644 on the VPS secrets file. Any local account can read it. Use 600.
${POSTGRES_PASSWORD} in Compose YAML. docker compose config prints the value. Prefer env_file and build URLs inside the app from separate env vars.
JS ${...} inside Compose command:. Compose eats it. Use a file or $$.
Assuming a private GitHub repo is enough. Private today can be public tomorrow, or the laptop backup can leak. History still carries the blob.
Committing secrets.env "just for the team". That is the same bug with a different filename. Use a password manager, sealed secrets, or a one-time scp onto the VPS.

Where this shows up for me

The stack I'm building, Epure, is a two-container Compose file (Rust binary + Postgres 16) for exception-only error monitoring. Same rule as above: example env in git, real DSN and DB password only on the host. The product is not APM, replay, or logs. If you self-host it, the docs assume you already keep secrets out of the repo.

After you stop tracking .env, do you rotate the leaked secret, or only fix the ignore file?

Write a Docker Compose healthcheck that actually checks your app

Amorizz — Sun, 11 Oct 2026 13:40:44 +0000

I wrote what looked like a careful Compose file for a small Node + Postgres app: a pg_isready healthcheck on Postgres and depends_on: condition: service_healthy on the app. On a fresh volume the app died with ECONNREFUSED while docker compose ps showed the database as (healthy).

Then I stopped Postgres to see what my readiness endpoint would say, and the Node process crashed instead. Six restarts in 40 seconds. Both bugs came from the same mistake: I was checking that something was running, not that it could do its job.

This is the setup I ended up with. Everything below was run on Docker Engine 26.1 with Compose 2.26, postgres:16.15-alpine and node:22.23.3-alpine, and the outputs are pasted from those runs, trimmed to the relevant lines (ps cut down to the SERVICE and STATUS columns, Start/End timestamps dropped from inspect).

TL;DR

pg_isready with no -h talks to the Unix socket, and during first-boot init the official Postgres image runs a temporary server that only listens on that socket. Add -h 127.0.0.1 so the check uses TCP like your app does.
Split your app's probes: /healthz answers "is the process alive", /readyz answers "can I serve a real request right now" (boot finished, database answers SELECT 1 within 1 second).
Plain Docker never restarts a container for being unhealthy. Pick what the healthcheck probes based on what reacts to it in your setup.

What does a healthcheck actually prove?

A healthcheck proves only what its command touches. "Process up", "port open", "app booted" and "dependencies reachable" are four different facts, and most Compose files check the first or second one and assume the rest.

Check	What it proves	What it misses
Container is `Up`	PID 1 hasn't exited	Everything else
Port accepts a TCP connection	Something is listening	App may still be booting, DB may be down
`pg_isready` (socket)	A Postgres server answers on the socket	First-boot temp server, TCP, auth, your database
`GET /healthz` returns 200	Event loop runs, HTTP stack works	Database, migrations, config
`GET /readyz` returns 200	Boot finished and the DB answered `SELECT 1` in 1s	Slow queries, other dependencies you didn't add

Kubernetes splits this into liveness and readiness probes. Docker gives you one healthcheck per container, so you have to decide which question it answers.

Why did pg_isready say healthy too early?

Because it asked the wrong server. The official image's entrypoint runs initdb, starts a temporary Postgres that listens only on the Unix socket, runs your /docker-entrypoint-initdb.d scripts, then shuts it down and starts the real server on TCP. pg_isready -U app uses the socket by default, so it reports "accepting connections" while the temp server is up.

My repro has a slow init script (an 8 second sleep standing in for a schema load):

#!/bin/sh
# initdb/01-slow-init.sh: stands in for a first boot that does real work
echo "init: loading schema (slow on purpose)"
sleep 8
psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "CREATE ROLE reporting NOLOGIN;"

And a Compose file that trusts plain pg_isready:

# compose.lie.yml: the version that lies
services:
  db:
    image: postgres:16.15-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: app
      POSTGRES_DB: app
    volumes:
      - ./initdb:/docker-entrypoint-initdb.d:ro
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "app"]
      interval: 1s
      timeout: 3s
      retries: 30
  app:
    build: ./app
    environment:
      DATABASE_URL: postgres://app:app@db:5432/app
    depends_on:
      db:
        condition: service_healthy

$ docker compose -f compose.lie.yml up -d
$ docker compose -f compose.lie.yml ps -a
SERVICE   STATUS
app       Exited (1) 3 seconds ago
db        Up 5 seconds (healthy)
$ docker inspect --format '{{json .State.Health}}' lab-2026-10-11-healthcheck-db-1
{"Status": "healthy", "FailingStreak": 0, "Log": [{"ExitCode": 0, "Output": "/var/run/postgresql:5432 - accepting connections\n"}]}

Look at the Output: /var/run/postgresql:5432. That's the socket. The logs show the order (timestamps trimmed):

db-1   13:23:59.918  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db-1   13:24:00.056  init: loading schema (slow on purpose)
app-1  13:24:00.901  listening on :3000
app-1  13:24:00.904  boot failed: ECONNREFUSED
db-1   13:24:08.065  CREATE ROLE
db-1   13:24:08.190  listening on IPv4 address "0.0.0.0", port 5432

The app gave up about 7 seconds before Postgres listened on TCP. Adding psql -c 'SELECT 1' over the socket doesn't fix it either. I tried: the temp server answers that too, and the app still got ECONNREFUSED. The fix is to probe the way your app connects, over TCP.

The Compose file that tells the truth

Probe Postgres over TCP, give the app its own healthcheck against /readyz, and gate startup on both. This is the full file.

# compose.yml
services:
  db:
    image: postgres:16.15-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: app
      POSTGRES_DB: app
    volumes:
      - ./initdb:/docker-entrypoint-initdb.d:ro
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      # -h 127.0.0.1 forces TCP. The first-boot temp server only listens on the socket.
      test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U app -d app && psql -h 127.0.0.1 -U app -d app -tAc 'SELECT 1' | grep -q 1"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 30s
      start_interval: 1s

  app:
    build: ./app
    restart: unless-stopped
    environment:
      DATABASE_URL: postgres://app:app@db:5432/app
    ports:
      - "127.0.0.1:3000:3000"
    healthcheck:
      test: ["CMD", "node", "-e", "fetch('http://127.0.0.1:3000/readyz').then(r=>process.exit(r.ok?0:1),()=>process.exit(1))"]
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 20s
      start_interval: 1s
    depends_on:
      db:
        condition: service_healthy
        restart: true

volumes:
  pgdata:

What each knob does:

start_period: failures in this window don't count toward retries, but the first success marks the container healthy right away. Size it for your slowest cold boot.
start_interval (Engine 25+): how often to probe during start_period. 1s here means up --wait returns as soon as things are ready instead of waiting out a 10s interval.
timeout: kept above the app's own 1s DB timeout, so /readyz returns a clean 503 before Docker kills the probe.
restart: true under depends_on: Compose restarts the app when you restart db with a Compose command. It doesn't fire when the container runtime restarts a crashed db.

The /healthz and /readyz handlers

/healthz should be cheap and depend on nothing, so it only fails when the process itself is broken. /readyz should fail fast when the app can't serve a real request: boot not finished, or the database not answering within a second.

{ "name": "health-lab", "private": true, "dependencies": { "pg": "8.13.1" } }

// server.js: /healthz says "the process works", /readyz says "I can do real work"
const http = require("node:http");
const { Pool } = require("pg");

const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 5,
  connectionTimeoutMillis: 1000, // a hung DB must fail the probe, not hang it
});

// Without this, a DB restart kills the whole process (unhandled 'error' event).
pool.on("error", (err) => console.error("pg idle client error:", err.code || err.message));

let booted = false; // flips after startup work (migrations, caches) is done

async function boot() {
  await pool.query("CREATE TABLE IF NOT EXISTS hits (id serial PRIMARY KEY, at timestamptz DEFAULT now())");
  booted = true;
  console.log("boot: migrations done");
}

function send(res, code, body) {
  res.writeHead(code, { "content-type": "application/json" });
  res.end(JSON.stringify(body));
}

const server = http.createServer(async (req, res) => {
  if (req.url === "/healthz") return send(res, 200, { status: "ok" });

  if (req.url === "/readyz") {
    if (!booted) return send(res, 503, { status: "starting" });
    try {
      await pool.query({ text: "SELECT 1", query_timeout: 1000 });
      return send(res, 200, { status: "ready", db: "ok" });
    } catch (err) {
      return send(res, 503, { status: "not_ready", db: err.code || err.message });
    }
  }

  try {
    await pool.query("INSERT INTO hits DEFAULT VALUES");
    send(res, 200, { hello: "world" });
  } catch (err) {
    send(res, 500, { error: err.code || err.message });
  }
});

server.listen(3000, "0.0.0.0", () => console.log("listening on :3000"));

// Retry boot instead of exiting: stay live (/healthz 200), report not ready (/readyz 503).
(async function bootLoop() {
  for (let attempt = 1; !booted; attempt++) {
    try {
      await boot();
    } catch (err) {
      console.error(`boot attempt ${attempt} failed: ${err.code || err.message}`);
      await new Promise((r) => setTimeout(r, Math.min(1000 * attempt, 10000)));
    }
  }
})();

FROM node:22.23.3-alpine
WORKDIR /app
COPY package.json ./
RUN npm install --omit=dev --no-audit --no-fund
COPY server.js ./
USER node
CMD ["node", "server.js"]

My first version had neither the pool.on("error") line nor the retry loop. When I stopped Postgres, pg emitted 57P01 terminating connection due to administrator command on an idle client, nothing listened for it, and Node exited. The restart policy brought it back, boot failed because the DB was still down, it exited again: RestartCount was 6 after 40 seconds.

Check it: start, then break the database

Run it with --wait so the command only returns once every healthcheck passes. Then stop and pause Postgres and watch what each probe says.

$ time docker compose up -d --build --wait
 Container lab-2026-10-11-healthcheck-db-1  Healthy
 Container lab-2026-10-11-healthcheck-app-1  Healthy
real    0m14.674s

$ docker compose ps
SERVICE   STATUS
app       Up 1 second (healthy)
db        Up 11 seconds (healthy)

$ docker inspect --format '{{json .State.Health}}' lab-2026-10-11-healthcheck-db-1
{"Status": "healthy", "FailingStreak": 0, "Log": [{"ExitCode": 0, "Output": "127.0.0.1:5432 - accepting connections\n"}]}

$ curl -s localhost:3000/healthz; curl -s localhost:3000/readyz
{"status":"ok"}
{"status":"ready","db":"ok"}

The inspect output now says 127.0.0.1:5432, so the check went over TCP. Now take the database away:

$ docker compose stop db
$ curl -s -w ' HTTP %{http_code}\n' localhost:3000/healthz
{"status":"ok"} HTTP 200
$ curl -s -w ' HTTP %{http_code}\n' localhost:3000/readyz
{"status":"not_ready","db":"Connection terminated due to connection timeout"} HTTP 503

# 40 seconds later
$ docker compose ps -a
SERVICE   STATUS
app       Up 43 seconds (unhealthy)
db        Exited (0) 42 seconds ago
$ docker inspect --format '{{.RestartCount}} {{.State.Health.Status}}' lab-2026-10-11-healthcheck-app-1
0 unhealthy

That 0 unhealthy is the line most people don't expect. restart: unless-stopped reacts to the process exiting. It ignores health. A hung database is a different failure from a stopped one, so pause it too:

$ docker compose start db     # app goes back to (healthy) on its own
$ docker compose pause db
$ time curl -s -w ' HTTP %{http_code}\n' localhost:3000/readyz
{"status":"not_ready","db":"Query read timeout"} HTTP 503
real    0m1.012s

One second and a 503, instead of a probe that hangs until Docker's timeout.

Should the Docker healthcheck call /healthz or /readyz?

It depends on what reacts to unhealthy in your setup. If nothing restarts unhealthy containers, point it at /readyz so depends_on and up --wait mean "ready to serve". If something does restart them, point it at /healthz, or a database blip turns into every app container restarting at once.

What reacts to `unhealthy`	Point the healthcheck at	Why
Nothing (plain Compose, `restart:` policy only)	`/readyz`	Gates `depends_on` and `up --wait` on real readiness, restarts nothing
Swarm, or an autoheal-style sidecar	`/healthz`	A DB outage shouldn't restart healthy app processes
Reverse proxy health checks (Traefik, Caddy, a load balancer)	`/readyz` from the proxy	Pull traffic from a replica that can't serve
Deploy script after `docker compose pull`	`/readyz` with `curl -f`	Fail the deploy before you switch traffic

Failure checklist

When the status and reality disagree, it's usually one of these. I reproduced 1 through 5 and 7 in the lab above.

pg_isready with no -h. It checks the socket, so it passes during first-boot init. Use -h 127.0.0.1.
curl isn't in the image. node:22.23.3-slim ships neither curl nor wget, and the check fails with exec: "curl": executable file not found in $PATH. Use node -e "fetch(...)", or busybox wget on Alpine.
One string in exec form. ["CMD", "pg_isready -U postgres"] makes Docker look for a binary literally named pg_isready -U postgres. Split the arguments, or use CMD-SHELL when you need && or pipes.
localhost resolves to ::1. In the Alpine container, localhost resolved to ::1. My test server listened on 0.0.0.0 (IPv4 only), and wget http://localhost:3000/ got Connection refused. Use 127.0.0.1.
No pool.on("error") in Node. A Postgres restart kills the process with an unhandled 'error' event, and the restart loop hides it.
/readyz with no timeout. Without connectionTimeoutMillis and query_timeout, a hung DB makes the probe wait until Docker kills it, and the health log only says Health check exceeded timeout (3s). Time out inside the app, below the healthcheck timeout.
Expecting unhealthy to restart anything. It doesn't, in plain Docker. It only feeds depends_on, --wait, docker ps and whatever tool you add.
$ in a CMD-SHELL string. Compose interpolates $VAR before the shell sees it. Write $$POSTGRES_USER if you want the container's env var.

Where this runs for me

The stack I'm building, Epure, is a two-container Compose file: one Rust binary and Postgres 16. It does exception-only error monitoring for apps that already use the official Sentry SDKs, no APM, replay or logs. Its production checklist has the same lesson in another form: /health returns ok over plain HTTP while login loops, because the session cookie is HTTPS-only. A green probe only tells you the process answered.

The lab is small enough to copy: compose.yml, server.js, package.json, the Dockerfile above, and one init script with a sleep 8 in it if you want to see the first-boot race yourself.

What does your /readyz check besides the database?

Your sampleRate won't save you from an error flood. Here's what does

Amorizz — Wed, 07 Oct 2026 20:21:45 +0000

I wrote a flood test for the error tracker I'm building: one bug thrown 997 times in a retry loop, plus three rare bugs thrown once each. In the run that looked most like production, all three rare bugs vanished before they left the Node process.

Then I tried the fix everyone reaches for, sampleRate: 0.1. Across five runs, 2 of the 15 rare errors made it. The loop that caused the mess was still the biggest thing on the screen.

TL;DR

sampleRate drops errors at random. It cuts the noisy loop by 90% and your one-off bugs by the same 90%, so the error you needed is the one most likely to disappear.
The SDK drops events before sampling even runs: Dedupe eats exact repeats, and a tight burst overflows the 64-slot transport buffer.
What worked: send everything, then rate limit stored bodies per fingerprint on the server. Count 100% of hits, keep the first 100 stack traces per bug per minute, let every new bug through.

Why doesn't sampleRate fix an error flood?

Because it doesn't know which error is the flood. sampleRate is a per-event coin flip in the SDK. At 0.1, every error has a 10% chance of being sent, whether it's the 900th copy of the same TypeError or the only RangeError you'll see all week.

That's fine for volume and bad for signal. A bug that fires once has a 90% chance of never reaching you. The count you do see is off by about 10x, because the server gets roughly 100 events and nothing in them says "multiply me". Client reports tell the server how many events were dropped and why (sample_rate, queue_overflow), but not which bug they belonged to.

The flood lab: one loop, three rare bugs

You can reproduce this in a few minutes with Node 20.19 or newer. No Docker, no account. ingest.js is a fake endpoint that parses just enough of the Sentry envelope format to count errors per fingerprint. flood.js uses the real @sentry/node SDK.

mkdir flood-lab && cd flood-lab
npm init -y >/dev/null
npm i @sentry/node@11.5.0 --save-exact

ingest.js. The fingerprint is the exception type plus the top in-app frame (file, function, line). The valve is a token bucket per fingerprint: 100 tokens, refilled at 100 per minute.

// ingest.js: fake error ingest with a per-fingerprint spike valve
const http = require("node:http");
const zlib = require("node:zlib");
const crypto = require("node:crypto");

const VALVE = process.env.VALVE !== "off";
const RATE = 100; // stored bodies per fingerprint per minute
const buckets = new Map();
const issues = new Map();

function allowStore(fp) {
  const now = Date.now();
  const b = buckets.get(fp) ?? { tokens: RATE, last: now };
  b.tokens = Math.min(RATE, b.tokens + ((now - b.last) / 60000) * RATE);
  b.last = now;
  buckets.set(fp, b);
  if (b.tokens >= 1) { b.tokens -= 1; return true; }
  return false;
}

function fingerprint(event) {
  const ex = event.exception?.values?.at(-1) ?? {};
  const frames = ex.stacktrace?.frames ?? [];
  const top = frames.filter((f) => f.in_app).at(-1) ?? frames.at(-1) ?? {};
  const key = `${ex.type}\0${top.filename}:${top.function}:${top.lineno}`;
  const fp = crypto.createHash("sha256").update(key).digest("hex");
  return { fp, title: `${ex.type}: ${ex.value}` };
}

http.createServer((req, res) => {
  if (req.url === "/stats") return res.end(JSON.stringify([...issues.values()]));
  const chunks = [];
  req.on("data", (c) => chunks.push(c));
  req.on("end", () => {
    let body = Buffer.concat(chunks);
    if (req.headers["content-encoding"] === "gzip") body = zlib.gunzipSync(body);
    const lines = body.toString("utf8").split("\n");
    for (let i = 1; i + 1 < lines.length; i += 2) {
      if (JSON.parse(lines[i]).type !== "event") continue;
      const { fp, title } = fingerprint(JSON.parse(lines[i + 1]));
      const issue = issues.get(fp) ?? { title, count: 0, stored: 0 };
      issue.count += 1;                                // every hit is counted
      if (!VALVE || allowStore(fp)) issue.stored += 1; // bodies are rationed
      issues.set(fp, issue);
    }
    res.statusCode = 202; // the client never sees the valve
    res.end("{}");
  });
}).listen(9000, () => console.log(`ingest on :9000 (valve ${VALVE ? "on" : "off"})`));

flood.js. Three env switches: SAMPLE_RATE, DEDUPE=off and PACE=off.

// flood.js: one bug stuck in a retry loop, plus three rare bugs you care about
const Sentry = require("@sentry/node");

const SAMPLE_RATE = Number(process.env.SAMPLE_RATE ?? 1);
const DEDUPE = process.env.DEDUPE !== "off";
const PACE = process.env.PACE !== "off";

Sentry.init({
  dsn: "http://public@localhost:9000/1",
  sampleRate: SAMPLE_RATE,
  tracesSampleRate: 0,
  integrations: (defaults) =>
    DEDUPE ? defaults : defaults.filter((i) => i.name !== "Dedupe"),
});

function chargeCard() { throw new TypeError("Cannot read properties of undefined (reading 'id')"); }
function applyCoupon() { throw new RangeError("discount out of range: -5"); }
function sendReceipt() { throw new Error("SMTP 421 try again later"); }
function exportCsv() { throw new SyntaxError("Unexpected token < in JSON"); }

const rare = { 250: applyCoupon, 500: sendReceipt, 750: exportCsv };

(async () => {
  for (let i = 1; i <= 1000; i++) {
    try { (rare[i] ?? chargeCard)(); } catch (e) { Sentry.captureException(e); }
    if (PACE && i % 50 === 0) await Sentry.flush(2000);
  }
  await Sentry.flush(5000);
  const issues = await (await fetch("http://localhost:9000/stats")).json();
  console.log(`sampleRate=${SAMPLE_RATE} dedupe=${DEDUPE ? "on" : "off"} pace=${PACE ? "on" : "off"}`);
  console.log("issue".padEnd(36), "count".padStart(6), "stored".padStart(7));
  for (const { title, count, stored } of issues) {
    console.log(title.slice(0, 36).padEnd(36), String(count).padStart(6), String(stored).padStart(7));
  }
})();

Run the ingest in one terminal and the flood in another. The ingest keeps state in memory, so restart it between runs.

# terminal 1
VALVE=off node ingest.js

# terminal 2
PACE=off node flood.js

Run 1: the SDK drops more than you think

Before any sampling, the SDK already throws events away. Here's the naive run: defaults, no pacing, valve off.

sampleRate=1 dedupe=on pace=off
issue                                 count  stored
TypeError: Cannot read properties of      4       4
RangeError: discount out of range: -      1       1
Error: SMTP 421 try again later           1       1
SyntaxError: Unexpected token < in J      1       1

997 TypeErrors thrown, 4 received. That's dedupeIntegration, on by default. It compares each event with the previous one and drops exact repeats, so you get one TypeError per stretch between the rare errors. With debug: true you'll see this 993 times:

Sentry Logger [warn]: Event dropped due to being a duplicate of previously captured event.

Real floods come from many browsers or workers, each with its own "previous event". So I turned Dedupe off:

PACE=off DEDUPE=off node flood.js

sampleRate=1 dedupe=off pace=off
issue                                 count  stored
TypeError: Cannot read properties of     64      64

This is the run that bit me. 64 events arrived and all three rare bugs were gone. The transport allows 64 requests in flight (DEFAULT_TRANSPORT_BUFFER_SIZE = 64 in @sentry/core). A synchronous burst fills it and everything after is dropped. With debug: true, 936 times:

Sentry Logger [log]: Recording outcome: "queue_overflow:error"

The rare bugs were events 250, 500 and 750. They never had a chance.

In the lab, pacing with await Sentry.flush() every 50 events fixes it, and so does transportOptions: { bufferSize: 2000 }. Both gave me 997 + 1 + 1 + 1. In a real incident, a bigger buffer just means you're queueing the flood in your app's memory.

Run 2: sampleRate 0.1 loses the rare bugs

Now the popular fix. Paced, Dedupe off, valve off, sampleRate at 0.1, five runs:

DEDUPE=off SAMPLE_RATE=0.1 node flood.js

sampleRate=0.1 dedupe=off pace=on
issue                                 count  stored
TypeError: Cannot read properties of    120     120
SyntaxError: Unexpected token < in J      1       1

That was one of my two lucky runs: one rare bug out of three. The other lucky run caught the SMTP error instead. The remaining three runs got 96, 102 and 85 TypeErrors and zero rare bugs. Yours will differ, it's random. That's the point.

	Loop bug (thrown 997x)	Rare bugs (3 per run)
Arrived over 5 runs	85 to 120 per run	2 of 15
What you conclude	count is off by about 10x	most of them never happened

The loop still owns the issue list. The bugs you'd want to look at are the ones that disappeared.

Run 3: count everything, ration the bodies

Same flood, no sampling, valve on.

# terminal 1
node ingest.js

# terminal 2
DEDUPE=off node flood.js

sampleRate=1 dedupe=off pace=on
issue                                 count  stored
TypeError: Cannot read properties of    997     100
RangeError: discount out of range: -      1       1
Error: SMTP 421 try again later           1       1
SyntaxError: Unexpected token < in J      1       1

Three runs, same output each time. The count is exact. The loop bug keeps 100 full stack traces and the rest are counted, not stored. Each rare bug gets its own bucket with 100 fresh tokens, so it always lands.

Every request got a 202, so the SDK has no reason to back off. All the throttling happens where you can see it: on the server, per bug.

How the spike valve works in Epure

Same idea, in Rust. Epure is the self-hosted error tracker I'm building. It takes errors from the official Sentry SDKs through a DSN swap and drops transactions and replays. Here's the core of the valve from crates/server/src/pipeline/spike.rs, trimmed:

const DEFAULT_RATE_PER_MINUTE: f64 = 100.0;

pub enum SpikeDecision { AllowStore, CounterOnly }

impl TokenBucket {
    fn check(&mut self) -> SpikeDecision {
        self.refill();
        if self.tokens >= 1.0 {
            self.tokens -= 1.0;
            SpikeDecision::AllowStore
        } else {
            SpikeDecision::CounterOnly
        }
    }
}

AllowStore queues the full event. CounterOnly queues a job that only bumps the issue counter. Buckets sit in a DashMap keyed by fingerprint, and idle ones get evicted once there are 20k keys. The fingerprint is a SHA-256 of the exception type plus the top in-app frame, unless the SDK sends a custom fingerprint.

From the outside, per the errors and rate limits docs:

Mechanism	Default	What the client sees
Spike valve	100 raw events / minute / fingerprint	`202`. Extra bodies aren't stored, the counter still goes up.
Project ingest cap	5000 events / hour / project	`403` `ingest_cap_exceeded`

No Redis. The buckets live in memory in the Rust process, next to Postgres 16, in a two-container compose file. The repo has an integration test that posts the same browser envelope 150 times and expects 150 202s, at least 150 counter increments and at most 100 stored rows.

Why not a project-level limit?

Project-level limits have the same blind spot as sampleRate. They protect the bill and can't tell which bug matters.

Sentry's SaaS has Spike Protection. Per its docs, it sets a per-project hourly threshold from your quota and the last 7 days of traffic, then discards events above it. That's a real safety net for your quota. But it limits the project, so a rare bug that lands mid-spike competes with the loop for the same allowance.

Epure doesn't dodge this either. Its hourly project cap is checked before the valve, and counter-only hits count toward it. Rough math: a loop sustained above about 83 events a minute (5000 / 60) burns the default cap within the hour. After that, every event from the project gets a 403 until the next clock hour, rare bugs included. You can raise the cap per project. The valve keeps a loop out of your database. It doesn't stop a sustained loop from eating the cap.

When is sampleRate still the right call?

When the upload itself costs you (a browser SDK on a lot of phones), or when your vendor bills per event and you need a ceiling today.

If one known error is the noise, an ignoreErrors entry or a beforeSend filter for that exact error beats a global coin flip. And don't mix up the knobs: tracesSampleRate is performance data, sampleRate is errors. Turning down traces is cheap. Turning down errors is how you lose the one-off bug.

Failure checklist

If your flood test doesn't match the outputs above:

Only a handful of events arrived. Dedupe is on by default. Filter it out or interleave different errors.
Exactly 64 events arrived. You hit the transport buffer. Pace with await Sentry.flush() or set transportOptions: { bufferSize }.
Rare errors got merged into the loop issue. Your fingerprint is too coarse. Group by message only and every Cannot read properties of undefined shares one bucket.
A deploy reset your throttling. Buckets are in memory, so a restart refills them. And since the frame key includes the line number, moving code gives the bug a new fingerprint.
You run several ingest processes. In-memory buckets are per process. Two processes can store up to 2x the bodies per minute.
Everything returns 403 ingest_cap_exceeded. That's the hourly project cap. The valve never returns an error.
202 but the issue list is empty. Accept is async, so the event may still be queued. Check the DSN's project and environment, then docker compose logs epure.
Fewer events than expected, no errors anywhere. Someone left sampleRate below 1 in shared config: grep -rn "sampleRate" src/.

What would you set?

I picked 100 stored stack traces per bug per minute because that's about what I'd scroll through during an incident, and I'm not sure it's right. If a loop hit your app tonight, how many full stack traces of the same bug per minute would you actually want kept?

Why Rust + Postgres RLS

Amorizz — Sun, 04 Oct 2026 13:20:31 +0000

TL;DR

We wanted multi-tenant exception ingest on a two-container stack: one Rust binary + PostgreSQL 16. No Redis. No Kafka.
Dashboard reads go through Postgres RLS with SET LOCAL app.current_org_id. App bugs must not cross orgs on a shared DB.
Ingest is DSN → project_id, not the app RLS role. RLS is defense-in-depth for dashboard /api/v1, not magic on every path.

I build Epure: exception-only error monitoring you can self-host. This week I am writing down the stack bet, not a migrate recipe. We already shipped the cheap VPS shape and the DSN swap. The question people ask next is why Rust plus Postgres row-level security instead of a bigger fleet or app-only filters.

Why Rust + Postgres RLS for exception ingest?

We optimized for a small blast radius on a shared Postgres: org-scoped dashboard reads enforced in the database, ingest scoped by DSN, two containers total.

Tenancy on a side-project VPS is easy to hand-wave. One engineer, one DB, WHERE org_id = ? in every handler. That works until a handler forgets the filter, a join leaks a neighbor row, or a reporting query skips the org check. On a multi-tenant exception store that is the whole product surface: other people's stack traces.

So we put the tenant key in Postgres. The dashboard role cannot see another org's rows even when application code is wrong. Rust keeps the binary one process: ingest HTTP, migrations, dashboard API. Postgres 16 holds events, issues, and the RLS policies. That is the whole product runtime besides your TLS edge.

If you run GlitchTip or self-hosted Sentry, the portable lesson is the same: decide where the tenant boundary lives before you scale the number of tenants on one database.

Threat model: who can read what if X fails?

Cookie and membership checks gate the dashboard. RLS is the backstop if a query forgets the org filter. Ingest trusts the DSN, not a client-supplied org id.

Failure	Without RLS (app filters only)	With `epure_app` + RLS
Handler drops `WHERE org_id = …`	Neighbor org rows can return	Policy still filters on `app.current_org_id`
Client sends another org's UUID	App must reject; easy to miss on one route	Session sets org from membership, not from the body
Stolen session cookie for Org A	Sees Org A (expected)	Still only Org A; no cross-org via forgotten filter
Leaked ingest DSN for Project P	Writes land on P	Same: ingest is DSN-scoped; not the app RLS role
Dashboard uses migrate/`epure` role by mistake	Full table access	Wrong. `epure` is for migrations. App must use `epure_app`

What we are not claiming: RLS stops a stolen DSN, a stolen cookie inside the right org, or a bug in the ingest writer. Different paths, different trust boundaries.

Why not a shared DB without RLS, app-only filters, or a heavier stack?

App-only filters fail open on one missed query. Heavier stacks buy isolation with ops cost we refused on a two-container box.

Option	Why it lost for this job
Shared DB, no RLS	Every read path must remember the org filter. One miss is a tenancy bug.
App-only org filters	Fine for a prototype. Weak as the only line when exceptions are the payload.
Separate database per customer	Strong isolation. Painful backups, migrations, and $5-class ops.
Full self-hosted Sentry (~65 containers)	Real product. Wrong unit when you only need exceptions ($5 VPS notes).
Redis / Kafka / extra workers	Useful for big fleets. We did not want them for exception ingest on one VPS.

GlitchTip is a fair lighter alternative on Django-shaped stacks. We still chose Rust + Postgres RLS because we are building the ingest path and want the tenant boundary in the database for dashboard reads. A GlitchTip reader can still steal the role split and SET LOCAL pattern below.

Working sketch: three roles, SET LOCAL, policy shape

Three Postgres roles on one epure database. Dashboard: cookie → membership → SET LOCAL app.current_org_id → queries. Ingest: DSN → project_id. Migrations: epure only.

Role / URL	Job
`epure` via `DATABASE_URL`	Migrations / bootstrap on startup
`epure_ingest` via `EPURE_INGEST_DATABASE_URL`	DSN-scoped event writes
`epure_app` via `EPURE_APP_DATABASE_URL`	Dashboard `/api/v1/*` with RLS

Compose injects all three for the app container. Missing migrate URL exits on start. Missing ingest URL yields storage errors on write. Missing app URL 500s the dashboard.

Dashboard request shape:

Valid session cookie.
Confirm the user is a member of the org.
SET LOCAL app.current_org_id = '<uuid>' on that transaction.
Run queries as epure_app. Never trust a client-supplied org id for the session setting.

Ingest request shape:

Authenticate with the project DSN.
Resolve project_id.
Write as epure_ingest. Do not set app.current_org_id from the client.

Illustrative policy shape (paraphrased from the docs pattern; pin your own names to migrations):

-- illustrative: dashboard role reads events only for the session org
CREATE POLICY events_org_isolation ON events
  FOR SELECT
  TO epure_app
  USING (org_id = NULLIF(current_setting('app.current_org_id', true), '')::uuid);

-- FORCE ROW LEVEL SECURITY so table owners cannot bypass by accident
ALTER TABLE events FORCE ROW LEVEL SECURITY;

Events land in monthly partitions events_YYYY_MM. Project retention TTL is 14 / 30 / 90 days; old partitions drop. Issue aggregates remain after event TTL. That retention story is separate from RLS, but it is part of the same cheap-ops bet.

You do not need our binary to try the pattern. Any multi-tenant read API on Postgres can use a restricted role, SET LOCAL for the tenant key, and FORCE ROW LEVEL SECURITY.

Honest limits and failure checklist

Exception-only. RLS is defense-in-depth for dashboard reads. Not a full security whitepaper. Never "100% Sentry compatible."

Limits we accept:

No APM, session replay, profiling, logs product, or mobile suite.
Partial Sentry wire for error tracking. Official SDKs via DSN swap (migrate post). Not a clone of every SaaS feature.
Ingest is not protected by the same org RLS session as the dashboard. DSN leakage is a project-level problem.
RLS does not replace authn, authz, TLS, or password hygiene.
Idle stack ~82 MiB was measured 2026-09-12 on our box. Treat that as one data point, not a promise.

Failure checklist:

Dashboard connected with DATABASE_URL / epure. Migrations role can see everything. Use EPURE_APP_DATABASE_URL for /api/v1.
Forgot SET LOCAL app.current_org_id. Queries as epure_app return empty or error depending on policy. Fix the session setter, do not weaken the policy.
Trusting org id from the JSON body. Membership must drive SET LOCAL. Client UUID is not a credential.
DSN pasted into a dashboard /api/v1 call. Wrong path. Ingest uses the envelope/store routes and the ingest role.
Example passwords on a public EPURE_PUBLIC_URL. Boot refuses documented laptop defaults and leftover CHANGE_ME off localhost. Use the production env template.
Expecting RLS to hide events after a leaked project DSN. Rotate the key. RLS does not rewrite ingest auth.
Assuming event TTL deletes issue rows. Partitions drop; issue aggregates remain. Read retention docs before promising "everything gone."

Configuration map for the three URLs and roles: self-hosting configuration.

Discussion

Where do you put the tenant boundary for multi-tenant reads: app filters only, Postgres RLS, or separate databases per customer?

Migrate Sentry -> Epure in 10 min

Amorizz — Thu, 01 Oct 2026 13:27:10 +0000

TL;DR

Keep the official Sentry SDK. Change only dsn to your Epure project.
Set traces, profiles, and replay sample rates to 0. Epure is exception-only.
Throw one test error, expect ingest HTTP 202, then an Issues row. Know what you drop.

I already ship @sentry/node (and browser) against SaaS Sentry. This week I pointed the same init at our self-hosted ingest: same SDK, new DSN, one verified event. The bill and the data residency story were enough; I did not want a client rewrite.

About ten minutes of wall clock once the ingest host is healthy. That is DSN copy, init edit, one throw, Issues check. Not a stopwatch claim for compose, TLS, or CORS.

What do you need before the clock starts?

Ingest already up. Official Sentry SDK already in the app.

Prerequisite	Done means
Epure host	`/health` returns `{"status":"ok"}` on the public origin you will put in the DSN
Project + DSN	Dashboard → Settings → SDK connection (or Setup). Key created.
App SDK	You already call `Sentry.init` with a SaaS DSN
Scope match	You only need exceptions / issues, not APM, replay, profiling, logs, or mobile

Host not up yet? Bring the two-container stack online first (Error monitoring on a $5 VPS), then come back. This post does not retell compose.

Timed steps: DSN swap and one event

Copy the project DSN, replace only dsn, zero unused rates, throw once, confirm Issues.

1. Copy the project DSN

Open the project → Settings → SDK connection (or Setup). Create a DSN key if none exists. Copy the full string. Do not paste a project UUID from the URL bar into the DSN.

Shape looks like:

https://YOUR_PUBLIC_KEY@errors.example.com/YOUR_PROJECT_ID

2. Swap `dsn` only

CI fixtures pin @sentry/browser / @sentry/node 7.120.0. Match that if you want the same line the docs exercise. On Sentry JS v8+, also set enableTracing: false when the option exists.

import * as Sentry from "@sentry/node"; // or @sentry/browser@7.120.0

Sentry.init({
  dsn: "https://YOUR_PUBLIC_KEY@errors.example.com/YOUR_PROJECT_ID",
  environment: "production", // match the Issues filter chip
  release: "my-app@1.2.3",
  tracesSampleRate: 0,
  profilesSampleRate: 0,
  replaysSessionSampleRate: 0,
  replaysOnErrorSampleRate: 0,
});

Migrating from Sentry SaaS: keep the same init options. Replace only dsn.

3. Send one test error

Sentry.captureException(new Error("Epure migrate test"));
// short workers: await Sentry.flush(2000) before process exit

4. Confirm ingest, then Issues

Expect ingest HTTP 202 with a body like { "id": "..." }. Refresh Issues. Match the environment filter to Sentry.init (production above, or local in docs examples). You should see an unresolved issue for that message.

Empty feed after a 202? Wait a second, check the env chip, confirm you opened the project that owns the DSN.

What do you keep / drop?

Exceptions and issue triage stay. Replay, APM, profiling, logs, and mobile do not.

Keep	Drop (out of scope)
Official Sentry SDKs (`@sentry/node`, `@sentry/browser`, …)	Session replay
Envelope / store error path	APM / distributed tracing product
`environment`, `release`, issue grouping	Continuous profiling
Self-hosted data path	Full SaaS suite extras

Never "100% Sentry compatible." Partial wire for error tracking. If GlitchTip is a better fit for a lighter Django-shaped stack, that is fine. This post is that migrate path.

Deeper SDK init, CORS, and compose details live in the earlier how-to: Point the official Sentry SDK at self-hosted ingest (DSN only). Soft-link only. Do not retell every init step here.

Failure checklist

CORS blocks the browser SDK. SPA origin missing from EPURE_CORS_ORIGINS. Fix origins before blaming the package (CORS post).
Short worker exits before flush. Node/Go lambda-style processes drop the last event. Call flush / Flush before exit (flush post).
DSN revoked or wrong key. Secret shown once at create. Rotate in Settings → SDK connection if you pasted a dead key.
Public URL mismatch. /health on localhost is not the host inside the DSN. Align EPURE_PUBLIC_URL / proxy with what the SDK posts to.
Environment chip hides the row. Issues filter is staging while init says production. Match them.
Second Sentry.init still on SaaS. A leftover init in a shared bootstrap still ships to ingest.sentry.io. Grep for every Sentry.init and every dsn:.

When the host is healthy and you want the install map in one place: Epure quickstart.

Discussion

What blocked your last DSN swap: CORS, auth, or a missing flush on short workers?

Error monitoring on a $5 VPS

Amorizz — Mon, 28 Sep 2026 20:46:13 +0000

TL;DR

We run exception-only ingest on a $5-12 class VPS: two containers (app + Postgres 16), official Sentry SDKs via DSN swap.
We ruled out full self-hosted Sentry (dozens of containers) and a SaaS bill for errors-only work.
Postgres stays off the host port. TLS edge is a separate recipe (Caddy post); this post is cost + footprint + constraints.

I run Epure exception ingest on the same class of box I use for side projects: one cheap VPS, one engineer, no Kubernetes. The job is narrow. Catch production exceptions, group them, keep Postgres off the public internet. Not APM. Not session replay. Not a 65-container Sentry clone on a droplet that costs less than lunch.

Here are the notes from that choice: constraints, what we ruled out, a minimal compose sketch, and the failure checklist we hit while wiring it.

What constraints did we accept?

$5-12/month VPS, one operator, official Sentry SDKs pointed at our own DSN. Errors only.

Concrete box:

Constraint	What we mean
Budget	$5-12 class VPS (1-2 vCPU, modest RAM). Not a managed cluster.
Headcount	One engineer who can SSH, edit compose, read logs.
Client path	Official Sentry SDKs (`@sentry/node`, `sentry-go`, etc.). DSN swap. No custom client.
Scope	Exceptions / issues. No APM, replay, profiling, logs, or mobile suite.

If you need the full Sentry product surface, this post is the wrong shape. Keep SaaS or self-host the big stack on hardware sized for it.

What did we rule out?

Full self-hosted Sentry (~65 containers) and paying SaaS just for exception ingest.

Option	Why it lost for this job
Self-hosted Sentry	Real product. Also a real fleet (Kafka, ClickHouse, Redis, workers, …). Fine on a dedicated host. Painful on a $5 box when you only need errors.
SaaS Sentry (exceptions-only)	Works. You pay for a bill and a vendor boundary you may not want for a side project or a privacy-sensitive ingest path.
GlitchTip	Fair lighter alternative. Open source, Sentry-protocol-ish, smaller than full Sentry. We still chose Epure because we are building it and want Rust + Postgres RLS for our threat model. A GlitchTip reader can still use the VPS constraints below.

We are not saying "kill Sentry." We are saying: if the job is exceptions on a cheap VPS, the 65-container install is the wrong unit of deployment.

Epure is exception-only. Official SDKs + DSN. Not "100% Sentry compatible." If your checklist includes replay, APM, or profiling, stop here and pick a suite that actually ships those.

Mental model before compose

Internet → TLS edge (Caddy :443) → app:8080 (Docker net)
                                  → postgres:5432 (Docker net only, NO host publish)

Two containers for the product. Proxy in front for HTTPS. Database never bound to 0.0.0.0:5432 on the host.

For the TLS edge recipe (Caddyfile, 80/443, Postgres unpublished), use the live post: Put self-hosted exception tracking behind Caddy. This article does not rehash every Caddy stanza.

Minimal two-container compose

App + postgres:16. Publish the app (or leave it for Caddy on the Docker network). Do not publish Postgres to the host.

Sketch shaped like production (passwords via .env, no example defaults on a public URL):

# docker-compose.errors.yml (sketch; pin your own image tags)
services:
  app:
    image: ghcr.io/epure-sh/epure:latest   # pin a release tag in prod
    restart: unless-stopped
    env_file: [.env]
    environment:
      DATABASE_URL: postgres://epure:${POSTGRES_PASSWORD}@postgres:5432/epure
      EPURE_PUBLIC_URL: https://errors.example.com
      EPURE_CORS_ORIGINS: https://app.example.com
    depends_on:
      postgres:
        condition: service_healthy
    ports:
      - "8080:8080"   # or omit and let Caddy reach app:8080 on the Docker net
    networks: [web]

  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: epure
      POSTGRES_USER: epure
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U epure -d epure"]
      interval: 5s
      timeout: 5s
      retries: 10
    # NO ports: stay on the Docker network only
    networks: [web]

volumes:
  postgres_data:

networks:
  web:

Bring it up:

docker compose -f docker-compose.errors.yml up -d
curl -sS http://127.0.0.1:8080/health

Expected:

{"status":"ok"}

Then put Caddy (or nginx) on 80/443 in front of 127.0.0.1:8080, point DNS, register, copy a project DSN, and captureException from an official SDK.

Canonical install steps (prod overlay, ./configure --prod, PaaS templates): Epure self-hosting installation.

Expected footprint (honest)

Qualitative only until we publish measured numbers.

Stack	Container count (order of magnitude)	Fit on $5-12 VPS?
Full self-hosted Sentry	Dozens (~65-class fleet with deps)	Usually no, not for errors-only
GlitchTip	Small multi-service compose	Often yes (lighter than full Sentry)
Epure (what we run)	2 (app + Postgres 16)	Designed for this class of box

We will not invent RAM figures. If you measure, paste docker stats and free -h in the comments. We will update this section with real numbers when we have them on a named plan.

Failure checklist

Postgres published to the host (5432:5432 "just for debugging"). Scanners find it. Drop the ports: block; reach DB only as postgres:5432 on the Compose network.
Laptop compose used as prod. Example passwords + localhost URL rules do not belong on a public hostname. Use the production overlay / env template from the docs.
No TLS, cookies broken. Secure session cookies need HTTPS. Finish the Caddy edge before you debug "login flaps."
Health is green, DSN host is wrong. /health on localhost is not the same as the public origin in the DSN. Match EPURE_PUBLIC_URL / proxy host to what the SDK posts to.
CORS blocks browser SDKs. SPA origin missing from allowlist. Fix origins before blaming the SDK (see our CORS post if you already hit this).
OOM while "just trying self-hosted Sentry." The fleet, not your app. Errors-only on two containers is the cheap-VPS shape; full Sentry wants a bigger host.
Expecting APM / replay / profiling. Exception-only stacks will not grow those features because you wished hard. Pick the product that matches the checklist.

If the two-container shape matches what you need, follow the install doc once, put TLS in front, swap the DSN on an official SDK, and confirm a test exception lands: self-hosting installation.

Discussion

What do you run on a cheap VPS for errors, and what blew the RAM first?

DEV Community: Amorizz

What the Sentry SDK drops after a 429 quota response

TL;DR

How the SDK handles a rate limit

Step 1: a stub that plays "quota exceeded"

Step 2: an app that captures on a timer

What happens at the cap: 7 of 10 errors never leave the process

Category limits only block that category

How long it backs off, and when it tries again

The client report says 7. I lost 8

How to detect it from your app and your ingest

Failure checklist

Run it yourself

Verify a minified stack against its JavaScript source map

TL;DR

The mental model: a map is a position lookup with a two-part key

Step 1: a bundle with two named functions

Step 2: capture the frames the SDK actually sends

Step 3: apply the map and print the original line

Step 4: break it with a stale map

Step 5: miss the release key

Turn it into a check you run before upload

What about node --enable-source-maps?

Failure checklist

Where this fits for me

Takeaway

Do you scrub request bodies before they reach your error tracker?

Why I can't shrug this off

Where I'm not sure

Log what your Sentry SDK actually sends before you leave SaaS

TL;DR

The mental model: the DSN gets envelopes, envelopes hold typed items

The tap: 30 lines that log every item type

The app: a small Express API with everything turned on

Result 1: the "everything on" init sent 546 KB, 1.2% of it errors

Result 2: the errors-only init sent 9.1 KB

Gotcha: tracesSampleRate: 0 is not the same as deleting it

Gotcha: 10 failed checkouts, 1 event

Gotcha: every error carried the request body

What you keep, what you give up

Failure checklist

Build vs buy exception monitoring, tested with a 36-line DIY ingest

TL;DR

The mental model: an error tracker is four jobs

The build option: 36 lines that accept real SDK events

Test 1: grouping is the actual product

Test 2: 500 errors in a tight loop, 63 arrived

Test 3: the tracker went down, the errors went with it

The buy options, priced from public docs

How I'd decide

If you build anyway: the failure checklist

Where I land

Sentry caught my Express error. The request still hung forever

TL;DR

The smallest app that reproduces it

Run 1: Express 4, no SDK

Run 2: Express 4, with Sentry

Run 3: Express 5

The fix if you are stuck on Express 4

What I would check in your app today

Where I run this

What this does not show

Your .env hit git. Here's how I keep secrets on a VPS

TL;DR

What are you actually protecting?

How does .env end up in git?

What goes on the VPS?

Compose: env_file vs ${PASSWORD} in the YAML

The ${} gotcha inside command: strings

Failure checklist

Where this shows up for me

Write a Docker Compose healthcheck that actually checks your app

TL;DR

What does a healthcheck actually prove?

Why did pg_isready say healthy too early?

The Compose file that tells the truth

The /healthz and /readyz handlers

Check it: start, then break the database

Should the Docker healthcheck call /healthz or /readyz?

Failure checklist

Gotcha: `tracesSampleRate: 0` is not the same as deleting it

2. Swap `dsn` only