DEV Community: Bryan Clark

A nearest-neighbour matcher turned a timing lag into a fake direction bug

Bryan Clark — Fri, 11 Sep 2026 15:40:45 +0000

TL;DR — If your comparison against golden data matches each expected event
to the nearest computed event of any kind and then asserts the kind, you have
coupled two independent error dimensions. A timing lag will surface as a
wrong-label failure and your oracle will confidently blame the wrong thing.
Match same-kind-to-same-kind for timing, and test direction separately at the
expected timestamp. Jump to the fix.

We validate our offline tidal-current engine against the hydrographic
authority's own published current predictions: for each published maximum flood
or maximum ebb, does the model produce the same event, at about the same time,
at about the same speed, and pointing the same way? A validation run over the
Canadian stations came back with two stations flagged as having a reversed
flood axis — the model claiming flood where the authority says ebb. That is
the one failure you cannot ship. Both stations were quarantined.

Both stations were fine. The matcher was broken.

This post is about the matcher — how you compare model output against golden
data. It is the sequel to
porting a tide engine to Swift with the original as the test oracle,
which is about where golden data comes from. Same test suite; the bug is in the
other half.

The problem: a label-flip failure that wasn't

The failing assertion looked like this, and its output is what got two stations
quarantined:

✕ maxEbb at 2026-07-14T14:12:00Z: engine labeled slack (0.03 kn)
✕ maxEbb at 2026-07-15T02:41:00Z: engine labeled maxFlood (0.61 kn)

Read that at face value and there is only one conclusion available: the engine
is mislabelling direction. For a current station, direction comes from a single
piece of metadata — the flood direction, the compass bearing the channel
floods toward, which the model projects the fitted velocity onto. Get that
bearing wrong by 180° and every flood becomes an ebb. It is a real, documented
failure mode: the published floodDirection is occasionally wrong for the
actual channel axis. So "the engine labelled ebb as flood" reads as "the flood
axis is reversed", which is a data-quality problem at the station, not a bug in
the model — and the response is to quarantine the station rather than ship a
current that runs backwards.

Here is the harness that produced it:

for e in fx.events where e.kind != "slack" {
    let t = parseISO(e.time)
    // Nearest computed event by TIME (avoids cross-cycle mis-match at weak,
    // mostly-single-direction stations), then require kind + tolerance.
    let m = try #require(
        computed.min { abs($0.time.timeIntervalSince1970 - t.timeIntervalSince1970)
                     < abs($1.time.timeIntervalSince1970 - t.timeIntervalSince1970) },
        "no computed event near \(e.time)")
    let kind: CurrentEventKind = e.kind == "maxFlood" ? .maxFlood : .maxEbb
    let timeErr = abs(m.time.timeIntervalSince1970 - t.timeIntervalSince1970) / 60
    let speedErr = abs(abs(m.speed) - abs(e.speed))
    #expect(m.kind == kind, "\(e.kind) at \(e.time): engine labeled \(m.kind) (\(m.speed) kn)")
    #expect(timeErr < 20, "\(e.kind) at \(e.time): time off \(timeErr) min (phase field wrong?)")
    #expect(speedErr < 0.3, "\(e.kind) at \(e.time): speed \(m.speed) vs \(e.speed)")
}

Four lines of idiom that every golden-data comparison in the world has written
at some point: find the nearest computed thing, then assert everything about
it.

Diagnosis: the matcher couples two independent errors

Two properties of the model are being measured here, and they are independent:

Timing — is the model's max ebb at the same minute as the published max ebb?
Direction — does the model think the water is going the same way?

The nearest-neighbour match couples them, because the search is over time and
the assertion is over kind. The candidate list from station.events() contains
slacks as well as extrema. So the match picks the wrong event as soon as the
timing error exceeds roughly half the spacing to the neighbouring event of a
different kind — and then the kind assertion fires on an event that was never
supposed to be the counterpart.

At a strong, cleanly semidiurnal station you never hit that condition: slack to
max is about three hours, so a matcher would need to be ninety minutes wrong
before it grabbed the wrong event. But the stations that got quarantined are
weak and slow-reversing — narrow, shallow, tidally awkward places where the
velocity loafs around zero, the model's own timing error runs to tens of
minutes, and the extrema are broad and low. That is exactly where the gap
collapses and the nearest event of any kind stops being the counterpart.

So the failure output is a lie in a very specific way. The engine is not
labelling anything wrong. It is late, and the matcher converts lateness into a
label-flip report. Two error dimensions, one nearest-neighbour search, and the
oracle can no longer tell you which one is broken.

The sign test settles it in one line and is not fooled by timing at all: ask
what the modelled velocity actually is at the moment the authority reports a
maximum ebb. At both quarantined stations it is negative at every single one —
0 wrong signs out of 19, and 0 out of 24. Directionally perfect. The
quarantine was manufactured by the comparison idiom.

What we tried (and why it failed)

Attempt 1: believe the failure and quarantine the stations. This is the
attempt that cost the most, because it worked in the sense that the test went
green and the bad data stopped shipping. Two stations dropped out of the offline
bundle with a note saying the flood axis was reversed, and downstream consumers
grew a stale line in their docs saying those stations "are known to flip."
Nothing about the quarantine looked wrong until someone asked why two stations
in completely different bodies of water, from different fits, would both have a
bad flood bearing.

Attempt 2: match nearest-by-time first, then require the kind. This is the
idiom above, and it deserves a proper Chesterton's-fence reading, because the
comment sitting on top of it is not decoration:

// Nearest computed event by TIME (avoids cross-cycle mis-match at weak,
// mostly-single-direction stations), then require kind + tolerance.

That comment is describing a real earlier bug. If you filter to the same kind
and then take the nearest, at a weak, mostly-single-direction station you can
match across a cycle boundary — grabbing yesterday's ebb, producing a wildly
wrong timing number. Matching by time first was a defence against that. It just
traded a loud, obviously-wrong timing number for a quiet, plausible direction
error, which is the worse trade: an absurd timing number gets investigated, a
credible-looking label flip gets believed.

Attempt 3: drop slacks from the candidate list. The obvious minimal patch —
if the nearest event is a slack, stop offering slacks:

let m = computed.filter { $0.kind != .slack }.min { /* nearest by time */ }

It removes the most common wrong match and none of the mechanism. The opposite
extremum is still in the list, and at a station running far enough late, the
nearest non-slack event to a published max ebb is the model's max flood. Same
coupled search, same false direction report, now with fewer symptoms — which
mostly means it takes longer to notice.

The fix

Stop asking one question and reading two answers out of it. Match same-kind for
timing; test direction at the golden timestamp, where timing cannot interfere:

/// Nearest computed event of the SAME KIND as the golden event.
private func nearestSameKind(_ computed: [CurrentEvent],
                             _ kind: CurrentEventKind, to t: Date) -> CurrentEvent? {
    computed.filter { $0.kind == kind }
        .min { abs($0.time.timeIntervalSince1970 - t.timeIntervalSince1970)
             < abs($1.time.timeIntervalSince1970 - t.timeIntervalSince1970) }
}

/// Sign of the station's velocity at the golden extremum time — the sound
/// direction test (flood positive, ebb negative).
private func signedSpeed(_ station: CurrentStation, at t: Date) -> Double {
    station.speeds(from: t.addingTimeInterval(-30), to: t.addingTimeInterval(30), step: 60)
        .first?.speed ?? 0
}

and then the loop asserts the two things separately:

let kind: CurrentEventKind = e.kind == "maxFlood" ? .maxFlood : .maxEbb
let m = try #require(nearestSameKind(computed, kind, to: t), "no computed \(e.kind) near \(e.time)")
let timeErr = abs(m.time.timeIntervalSince1970 - t.timeIntervalSince1970) / 60
let speedErr = abs(abs(m.speed) - abs(e.speed))

let v = signedSpeed(station, at: t)
#expect(e.kind == "maxFlood" ? v > 0 : v < 0,
        "\(e.kind) at \(e.time): modelled velocity \(v) kn has the wrong sign")
#expect(timeErr < 20, "\(e.kind) at \(e.time): time off \(timeErr) min (phase field wrong?)")
#expect(speedErr < 0.3, "\(e.kind) at \(e.time): speed \(m.speed) vs \(e.speed)")

Three assertions, three questions, no shared search. A late station now reports
one number — time off 22.4 min — and a reversed station reports a wrong sign.
Neither can impersonate the other.

The same separation, in the TypeScript harness that builds the offline bundle:

for (const event of observed) {
  const at = Date.parse(event.time);
  let best = Infinity;
  for (const candidate of predicted) {
    if (candidate.kind !== event.kind) continue;   // TIMING: same kind only
    best = Math.min(best, Math.abs(candidate.time.getTime() - at) / 60_000);
  }
  if (best <= MATCH_WINDOW_MIN) deltas.push(best);
}

let wrongSign = 0;
for (const event of extrema) {                      // DIRECTION: sign at the golden time
  const modelled = velocityAt(new Date(event.time));
  if ((event.kind === "maxFlood" && modelled < 0) ||
      (event.kind === "maxEbb" && modelled > 0)) wrongSign++;
}

Why it matters: same-kind matching is the safer test, not the laxer one

The instinct when you delete #expect(m.kind == kind) is that you have just
removed the check that catches a reversed axis. The opposite is true, and the
argument is worth internalising because it is what makes the fix safe to ship.

A genuinely reversed flood axis does not flip some labels. It flips every
event on the station, so the model's max ebbs land where the true max floods
are — about half a tidal cycle away, roughly six hours. Under same-kind
matching, the nearest computed max ebb to every published max ebb is therefore
six hours out, and blows through a ±20-minute timing gate at every single
extremum on the station.

So the reversed axis is caught twice — once loudly by the sign test, once
structurally by the timing gate — and a merely-late station is caught by
neither. Coupled, the two failures were indistinguishable. Separated, they can't
be confused: a real flip is systematic, a false one is occasional. The bundle
builder now encodes exactly that, quarantining only on systematic disagreement:

A reversed flood axis quarantines a station outright… The test is the sign of
the modelled velocity at CHS's own extremum times, and the bar is systematic
disagreement (≥ 60% of extrema): a genuinely reversed axis is wrong at nearly
every peak. Occasional wrong signs are timing error at a weak station, not a flip.

The transferable version

None of this is about tides. The shape is:

A nearest-neighbour matcher couples every dimension you didn't search on. You search on one axis (time, position, string distance, embedding distance) and then assert on others (kind, label, value). Every assertion downstream of the match inherits the search's error. The moment the search picks the wrong counterpart, the other assertions are the ones that fail — and they will fail with an entirely credible message about a problem you do not have.
The failure mode is a false positive that reads like a serious bug. A coupled matcher does not produce noise, it produces a specific, plausible, wrong diagnosis. That is worse than a flaky test: you act on it. We deleted two good stations from a navigation dataset and wrote the wrong explanation into the documentation downstream of it.
Match on the dimension you are measuring; test the others where the match can't reach. Same-kind matching for timing. Sign-at-the-expected-timestamp for direction. If a check can only be made after a match, ask whether a mismatch would make it lie.
Check that the strict version still catches the real bug — explicitly, with an argument, in a comment. Ours: a reversed axis is half a cycle away, so it fails the timing gate everywhere. If you can't make that argument, you have loosened the suite rather than fixed it.
A test-oracle suite needs its comparison layer audited like production code. Golden vectors get all the attention — where they come from, how they're generated, what tolerance they're gated at. The comparison idiom between the vectors and your output gets written once, in four lines, and never reviewed. It is just as capable of being wrong, and when it is wrong it discredits code that works.

The engine and the bundle builder are both MIT, and the tide/current work is
what keeps our own boat off the rocks in passes that only open for twenty
minutes: slackwater-engine
(Swift) and chs-constituents
(TypeScript). If a station near you disagrees with them, the validation output
will now tell you which of the two things is actually wrong.

A config warning on stdout made a healthy alarm lane look dead

Bryan Clark — Sun, 06 Sep 2026 16:04:57 +0000

TL;DR — A CLI reported a missing environment variable by printing a
Config warnings box to stdout. My pipeline was parsing that stdout, so
awk got box-drawing characters where data rows should be and matched
nothing. "Nothing" read as the alarm lane is dead, not your shell is
missing an env var. Forty minutes and one false outage report later:
prefix the environment for any command whose output you parse, not just
the ones that need the secret. Jump to the fix.

The problem

I run an agent gateway on a small boat server. It has an audit log, and the
cheapest health check for "are the agents actually running" is a one-liner over
the CLI:

gwcli audit --limit 12 | awk '$2=="agent_run"'

Which, one morning, returned nothing at all. No error. Exit code 0. Just an
empty result where a dozen rows had been the week before.

The obvious reading — the one I made, and the one that cost the morning — is
that there were no agent runs. Alarms flow through that gateway, so "no agent
runs" means the soft alarm lane is down: a low-battery or depth alarm on the
boat would raise, publish, and reach nobody. That's a real outage on the boat's
safety path, so it got reported as one.

The lane was healthy the entire time. The hook was firing about twelve seconds
after each test alarm, which is exactly the coalesce window. Nothing was down.

Here is what the command actually printed, once I stopped piping it:

$ gwcli audit --limit 12
┌ Config warnings ───────────────────────────────────┐
│ ! hooks.token: Missing env var GW_HOOKS_TOKEN      │
└────────────────────────────────────────────────────┘
ts                    kind        agent    status
2026-08-10T06:12:04Z  agent_run   watch    ok
2026-08-10T06:24:11Z  agent_run   watch    ok

The rows were there. So was a warning box, printed to stdout, sitting on top
of them. awk '$2=="agent_run"' doesn't crash on box-drawing characters; it just
doesn't match them, and it never got the chance to be confused, because in the
real invocation the box was all there was.

Diagnosis

The day before, I'd moved two credentials out of the gateway's config file and
into secret references, so the config could go under version control:

// ~/.gwcli/config.json
{
  "gateway": { "auth": { "token": { "$secret": "GW_AUTH_TOKEN" } } },
  "hooks":   { "token": "${GW_HOOKS_TOKEN}" }
}

Note the two forms. That asymmetry looks like sloppiness and it is not — more on
that below. The secrets themselves lived in an env file that the gateway's
launcher script sourced before exec'ing the daemon:

# run-gateway.sh
set -a; . "$HOME/.gwcli/secrets.env"; set +a
exec gwcli gateway

The daemon was fine, because the daemon goes through that script. A bare
gwcli ... typed in a shell, or run over SSH, does not. And with the env
missing, the two fields fail in two completely different, differently-visible
ways:

Field	Form	Channel	What you get
`gateway.auth.token`	typed secret ref	stderr	`SecretRefUnavailableError: ... unavailable in this command path`
`hooks.token`	`${VAR}` string	stdout	a `Config warnings` box, mixed into the data you were parsing

The stderr one is a good failure. It's loud, it names the field, and it stops
you. The stdout one is the dangerous half: it doesn't stop anything, it doesn't
look like an error to a program, and it arrives inside the data stream. Every
consumer that treats stdout as structured output — awk, jq, grep -c, a
Python subprocess.run(...).stdout — silently ingests a diagnostic as content.

And the two compound in the worst possible way, which is the part I'd want
another builder to take away:

# a completely normal scripting reflex
gwcli audit --limit 12 2>/dev/null | awk '$2=="agent_run"'

2>/dev/null suppresses the one failure that would have told me what was wrong,
and leaves the one that corrupts the answer. A clean-looking pipeline, producing
wrong data, with no visible error anywhere. That is a very hard thing to
distrust at 6 a.m.

What I tried, and why it failed

Attempt 1: prefix the env only where the secret is needed. My first mental
model was "this command reads the audit log, it doesn't need the gateway token,
so it doesn't need the env." Wrong — the config is parsed on every invocation,
so an unresolvable reference warns on every invocation, whether or not the
subcommand touches it:

$ gwcli audit --limit 12 | awk '$2=="agent_run"'     # env not prefixed
                                                      # (nothing — the box isn't a match)
$ set -a; . ~/.gwcli/secrets.env; set +a
$ gwcli audit --limit 12 | awk '$2=="agent_run"'     # env prefixed
2026-08-10T06:12:04Z  agent_run   watch    ok
2026-08-10T06:24:11Z  agent_run   watch    ok

Same command, same server, same healthy lane. The only variable was my shell.

Attempt 2: a wrapper, so the prefix stops being something to remember. A
tiny script that sources the env and execs the real binary:

#!/usr/bin/env bash
# gw — run gwcli with secrets resolved
set -euo pipefail
set -a; . "$HOME/.gwcli/secrets.env"; set +a
exec gwcli "$@"

This works, and it survived the day. But it only helps the caller who remembers
to type gw instead of gwcli, which is the same class of discipline that
failed in the first place.

Attempt 3: a --selftest that asserts the environment instead of trusting
it — meant for the top of any script that parses this CLI's output. This is
the one worth writing down, because it passed while genuinely broken.

The CLI reports the identical condition in two different formats depending on
the subcommand:

$ gwcli audit
┌ Config warnings ───────────────────────────────────┐
│ ! hooks.token: Missing env var GW_HOOKS_TOKEN      │
└────────────────────────────────────────────────────┘

$ gwcli config validate
1 warning(s): ! hooks.token: Missing env var GW_HOOKS_TOKEN

I wrote the check against config validate — the subcommand whose entire job is
to answer this question — and grepped for the string I'd been staring at all
morning:

# before — passes with the token genuinely missing
out="$(gwcli config validate 2>/dev/null || true)"
if grep -qi "Config warnings" <<<"$out"; then
  echo "SELFTEST FAIL"; exit 1
fi
echo "SELFTEST PASS"

config validate never prints Config warnings. The grep cannot match. The
selftest passed instantly and cleanly with a required credential absent — the
exact state it existed to catch. Note it also carries the 2>/dev/null reflex,
so the loud channel was gone there too.

What caught it was not review and not a test. It was running the negative
control: unsetting the variable on purpose and watching whether the check went
red. It didn't.

# after — match the substring both formats actually contain
if grep -qi "missing env var" <<<"$out"; then

The fix

The real fix landed a day later, and it deleted the wrapper's entire reason to
exist.

Buried in the gateway's own configuration docs: the CLI auto-loads a dotenv
file from its config directory on every invocation — daemon, CLI, cron,
service unit, all of them. The secrets file already existed. It was just named
something the tool didn't know to read.

mv ~/.gwcli/secrets.env ~/.gwcli/.env      # that exact filename is the fix
chmod 600 ~/.gwcli/.env

$ gwcli audit --limit 12 | awk '$2=="agent_run"'   # plain shell, no prefix, no wrapper
2026-08-10T06:12:04Z  agent_run   watch    ok
2026-08-10T06:24:11Z  agent_run   watch    ok

Both credentials now resolve for anyone who runs the binary, from anywhere. The
launcher script stopped sourcing anything. The wrapper survives for exactly one
unrelated reason — PATH, because the Node version manager puts the binary in a
directory that only an interactive shell's rc file adds, so ssh host 'gwcli …'
gets command not found. That's a different bug wearing the same coat.

The selftest stayed, because renaming a file doesn't stop it going missing or a
key rotating out:

gwcli config validate 2>/dev/null | grep -qi "missing env var" && exit 1

Why it matters, and the gotchas around it

Prefix the env for any command whose output you parse, not just the ones that
need the secret. That's the rule I'd hand to anyone else running a CLI inside
a pipeline. It's counter-intuitive precisely because the failing subcommand had
nothing to do with the missing credential.

Diagnostics belong on stderr — and you are not in control of that. This is
the oldest rule in the Unix toolbox and every CLI breaks it eventually, usually
via a pretty-printing layer that wraps console.log. Assume a tool you depend
on will one day put a banner in your data. The defences are cheap: pass whatever
--json / --quiet / --no-color flag exists and parse a structured format
rather than a table; or fail loudly when the shape is wrong, instead of treating
an empty match as a fact.

An empty result is not evidence. awk matching zero rows means "no rows
matched", which covers there is no data, the data moved, and what I handed
you wasn't data at all. My health check couldn't tell those apart, and the one
it silently chose was the alarming one. A parse that can return empty needs a
sanity assertion — did the header row arrive? — before its emptiness is allowed
to mean anything.

2>/dev/null is a load-bearing decision, not punctuation. In this incident
it was the difference between a five-second diagnosis and a false outage report.
If you're silencing stderr because a command is chatty, silence it after
you've checked the exit code, or route it to a file you actually read.

You haven't tested a check until you've watched it fail. Break the thing on
purpose, confirm red, put it back, confirm green. A selftest that passes when
broken is worse than no selftest — it converts an unknown into a false known,
and it is exactly the kind of green tick you'd cite while reporting an outage
that isn't happening.

One deliberate non-fix. The two-form asymmetry in the config — one field a
typed secret-reference object, the other a ${VAR} string — is schema-forced,
not style drift: the second field's schema is string-only and cannot take the
object form, which is why it's the one that emits a warning rather than
throwing. That's now a comment in the config, because it reads as an
inconsistency and the obvious "cleanup" would put the whole failure straight
back.

This is part of the agent stack I'm building for an all-electric charter
catamaran that doesn't exist yet — the same lane the false report was about is
the one that carries SignalK
alarms to a phone. It stayed up. My confidence in my own health check did not.

Related: Monitor the delivery path, not just the alarm — the same alarm lane failing for real, and why a quiet system produces nothing to notice; and Running OpenClaw on a Raspberry Pi alongside SignalK — the gateway this CLI drives.

MapLibre's SDF icon edge is at 0.75, not 0.5

Bryan Clark — Sat, 05 Sep 2026 18:57:05 +0000

TL;DR — If you hand-author the alpha channel of an SDF icon for
map.addImage(id, img, { sdf: true }), the edge does not go at alpha 128.
MapLibre's shader thresholds the fill at inner_edge = (256.0 - 64.0) / 256.0
= 0.75, i.e. alpha 191, over an 8-texture-pixel field. Encode the edge
at 128 and the shader draws a contour 4 px inside your shape — anything
thinner than 8 px vanishes completely, silently, with correct geometry and no
error of any kind. Use alpha = 255 * (1 - cutoff - d / SDF_PX) with
cutoff = 0.25, SDF_PX = 8. Jump to the fix.

We draw station pins on a MapLibre chart in
Slackwater, an offline
tide-and-current app for the Salish Sea. The pins need to change colour with
live state — flooding, ebbing, slack — and they need a halo so they stay legible
over bathymetry contours. In MapLibre, icon-color and icon-halo-color only
do anything on SDF icons, so the pin has to be registered as a signed
distance field.

The glyphs are simple stroke geometry — a sine wave for a current station, a
dome over a datum line for a tide station — so rather than rasterise a path and
hope, we computed the field analytically: exact distance from each pixel to the
stroke centreline, minus half the stroke width. A canvas-rasterised path gives
you a 1 px coverage ramp, which is enough to tint an icon but nowhere near
enough to hang a halo on.

The geometry was right. The icons rendered as nothing.

The problem: a correct icon that draws zero pixels

The registration is unremarkable, and it succeeds:

map.addImage(id, pinGlyphImage(kind), { sdf: true, pixelRatio: PIN_PIXEL_RATIO });

So is the layer:

{
  id: "station-pins",
  type: "symbol",
  source: "stations",
  layout: {
    "icon-image": ["match", ["get", "kind"], "current", "pin-current", "pin-tide"],
    "icon-size": 1,
    "icon-allow-overlap": true,
    "icon-ignore-placement": true,
  },
  paint: {
    "icon-color": ["match", ["get", "state"], "flood", "#4a9fd8", "ebb", "#e8a33d", "#7d9cb8"],
    "icon-halo-color": "#0b1a2b",
    "icon-halo-width": 1.5,
  },
}

And the field generator looked right too. signedDistance returns the exact
distance to the union of strokes, negative inside; the encoding was the obvious
one:

const SPREAD = 8;
// ...
const d = signedDistance({ x: x + 0.5, y: y + 0.5 }, strokes);
// maplibre's SDF convention: 128 is the edge, higher is inside.
const a = Math.round(255 * Math.max(0, Math.min(1, 0.5 - d / (2 * SPREAD))));

That comment is the whole bug. It is stated with total confidence and it is
wrong.

There is no error. map.addImage resolves, map.hasImage(id) returns true,
the source has features, the layer is in the style, nothing appears in the
console, and WebGL is perfectly happy. You get a map with correct data on it and
no pins.

Diagnosis: read the shader

MapLibre's icons and its text glyphs go through the same fragment shader. The
whole answer is six lines of
src/shaders/glsl/symbol_sdf.fragment.glsl
in maplibre-gl 5.24.0:

#define SDF_PX 8.0
// ...
lowp float inner_edge = (256.0 - 64.0) / 256.0;   // line 34  -> 0.75
lowp float dist = texture(u_texture, tex).a;
// ...
highp float alpha = smoothstep(inner_edge - gamma_scaled, inner_edge + gamma_scaled, dist);

dist is your alpha channel, normalised to 0–1. The shader paints the contour
where dist equals 0.75 — alpha 191 — not 0.5. (In the sibling
symbol_text_and_icon shader the same constant is spelled buff, which is the
older Mapbox name you will see quoted around the web; the value is identical.)

Now the arithmetic. Take any linear encoding of the form
alpha = 255 * (A - d / R), where A is the value you put at d = 0 and R
is the distance over which the field ramps. The shader draws the contour where
A - d / R = 0.75, that is:

d = R * (A - 0.75)

That single line explains everything:

encoding	A	R	contour the shader draws
ours (broken)	0.5	16	`d = -4 px` — 4 px inside the true edge
edge at 0.5, correct ramp	0.5	8	`d = -2 px` — 2 px inside
correct	0.75	8	`d = 0` — the true edge

A has to be 0.75 or the shader renders an eroded version of your shape. R
does not affect where the edge lands at all — it only sets the ramp rate, which
is what halo widths and antialiasing are scaled against.

Our strokes are 0.14 * 44 = 6.16 texture pixels wide, so the deepest interior
point sits at d = -3.07. A 4 px erosion of a 6.16 px stroke leaves nothing:

Run the numbers over the actual field and the result is exact: peak alpha 176,
threshold 191.25, 0 of 1936 pixels above it. The antialiasing band does not
save you either — EDGE_GAMMA = 0.105 / u_device_pixel_ratio puts the
smoothstep's lower edge around alpha 178 at devicePixelRatio 2, still above the
field's maximum.

Why 0.75 is not an arbitrary number

The intuition that says "0.5" is really the intuition that the field should be
symmetric around the edge. MapLibre's is deliberately asymmetric, and the
reason is halos.

The encoding comes from
@mapbox/tiny-sdf,
whose defaults are radius = 8, cutoff = 0.25:

const scale = 255 / this.radius;
const base = 255 * (1 - this.cutoff);
for (let i = 0; i < len; i++) {
    const d = Math.sqrt(gridOuter[i]) - Math.sqrt(gridInner[i]);
    data[i] = Math.round(base - scale * d);
}

which is alpha = 255 * (1 - cutoff - d / radius). With cutoff = 0.25 and
radius = 8 the field saturates 2 px inside the shape and runs out 6 px
outside. The edge therefore sits 6 / 8 = 0.75 of the way up the range — and
that is the same 0.75, arrived at from the other direction. MapLibre uses
exactly these constants for its own glyph atlas, in
glyph_manager.ts
(radius: 8 * textureScale, cutoff: 0.25).

The 6 px is not a coincidence either. It reappears in the halo branch of the
same shader:

highp float halo_edge = (6.0 - halo_width / fontScale) / SDF_PX;   // line 53

Six pixels of outside room is precisely the widest halo the shader can express.
That is what the asymmetry buys.

What we tried (and why it failed)

The test that asserted the bug

The glyph generator shipped with a unit test written specifically to pin down
the alpha encoding. It passed:

it("puts the shape edge at alpha 128 — inside is brighter, far outside is dark", () => {
  const img = pinGlyphImage("tide");
  const mid = Math.floor(img.width / 2);
  expect(alphaAt(img, 0, 0)).toBeLessThan(128);
  const column = [];
  for (let y = 0; y < img.height; y++) column.push(alphaAt(img, mid, y));
  expect(Math.max(...column)).toBeGreaterThan(128);
});

Corner alpha is 0, which is less than 128. Peak column alpha is 176, which is
greater than 128. Green, on a glyph that renders zero pixels.

The test asserted the same wrong constant the implementation did, so it could
only ever confirm the code agreed with itself. toBeGreaterThan(128) is
satisfied by 176 — the value that proves the bug — and by 255. The assertion had
no power exactly where it mattered.

"Just tighten the spread until something appears"

The tempting empirical fix, once you know the icons are invisible but not
why, is to make the field steeper and see what shows up. It does eventually
produce marks, which is the trap:

ramp span `R`	contour drawn	what you see	outside room left
16	`d = -4.00 px`	nothing	8 px
12	`d = -3.00 px`	a 0.16 px thread	6 px
10	`d = -2.50 px`	a 1.16 px hairline	5 px
8	`d = -2.00 px`	2.16 px of a 6.16 px stroke	4 px

At R = 8 with the edge still at 0.5 you get something that looks plausible on
screen — a thin version of the glyph — and you ship a shape that is 2 px
narrower everywhere than the one you drew, with a field that runs out 4 px
outside instead of 6, so icon-halo-width values above 4 quietly clip. Tuning
by eye lands you here.

"Add a constant so the edge reads 192"

Also close, also wrong:

// edge lands at 192 — but the ramp rate is still 1/16 per px
const a = Math.round(255 * (0.5 - d / 16)) + 64;

The contour now falls at d = 0, so the shape is the right size. But the ramp
is half as steep as SDF_PX = 8 assumes, so every halo width comes out at the
wrong scale, and the far field bottoms out at a floor of 64 instead of 0 — the
alpha channel never reaching zero on an image you also might want to reuse
non-SDF. Two knobs, and this only turns one of them.

The fix

Encode the edge at the threshold the shader actually uses, and ramp at the rate
it actually assumes:

/**
 * maplibre's SDF constants, not ours to choose: the shader thresholds the icon
 * fill at `inner_edge = (256 - 64) / 256 = 0.75` over a field authored with
 * `SDF_PX = 8` texture pixels. Matches `@mapbox/tiny-sdf`'s `cutoff = 0.25`.
 */
const SDF_PX = 8;
const SDF_CUTOFF = 0.25;

const d = signedDistance({ x: x + 0.5, y: y + 0.5 }, strokes);
// 255 * (1 - cutoff - d/SDF_PX): edge (d=0) lands at ~191, the interior
// saturates to 255, and the field reaches 0 at 6px outside the stroke.
const a = Math.round(255 * Math.max(0, Math.min(1, 1 - SDF_CUTOFF - d / SDF_PX)));

One line, and it is just tiny-sdf's own formula. The edge lands at 191, the
interior saturates at 255, the field runs out 6 px outside.

The test gets the real invariant — saturation, not "brighter than the value we
assumed":

it("puts the shape edge at maplibre's 0.75 threshold, and saturates inside", () => {
  const img = pinGlyphImage("tide");
  const mid = Math.floor(img.width / 2);
  expect(alphaAt(img, 0, 0)).toBe(0);
  const column = [];
  for (let y = 0; y < img.height; y++) column.push(alphaAt(img, mid, y));
  expect(Math.max(...column)).toBe(255);
  expect(column.some((a) => a > 180 && a < 205)).toBe(true); // the edge band
});

toBe(255) is the assertion that has teeth. An interior that never reaches 255
is an interior that may never cross 0.75, and that is the failure this whole
post is about.

Why it matters, and the traps next door

The failure mode is silence. There is no error string to search for, which
is why this costs hours rather than minutes — you go looking for a bug in your
geometry, your projection, your GeoJSON, your collision settings. The symptom is
an absence.

It is worse than silent, actually. Our pins carry
"icon-halo-color": "#0b1a2b" — the same value as the map's background. The
halo branch thresholds at halo_edge = (6.0 - halo_width / fontScale) / SDF_PX,
which for any non-zero halo width is below the 0.75 fill threshold. So a
field that is too dark to draw a fill can still be bright enough to draw a halo.
With a halo colour that matches the water, the glyph paints itself in the
background tone. Worth knowing before you conclude "nothing rendered."

The docs do not tell you. MapLibre's
StyleImageMetadata
documents the sdf flag in full as: "Whether the image should be interpreted
as an SDF image." That is the entire specification of the alpha format.
mapbox/mapbox-gl-style-spec#97
— "Style reference: document sdf icons" — was opened in 2014 and never resolved;
maplibre/maplibre-native#2551
asks for the same thing and is still open. The one human-readable statement of
the number I could find anywhere is a paragraph on a Mapbox
troubleshooting page about recolorable images:
"values between 192 and 255 represent 'inside' a glyph and values from 0 to 191
represent 'outside'." Correct, and filed under glyphs, and never connected to
addImage.

The scale-invariant part is the part you must get right. A = 0.75 is fixed
— it is a threshold on a normalised alpha, so it holds whatever resolution you
author at. R scales with your image: we author at pixelRatio: 2 with
R = 8, while MapLibre's own glyph atlas uses radius: 8 * textureScale.
Getting A wrong makes your icon vanish; getting R wrong makes your halo
widths come out at the wrong scale. Check the halo by eye, but check the edge by
arithmetic.

And the general one. A test that asserts a constant the implementation also
asserts is not a test, it is a restatement. The useful assertion was not "is the
edge where we think" — it was "does the interior saturate," a property with a
single correct answer that does not depend on the belief being tested.

Slackwater is the offline tide and current app for the Salish Sea we run
alongside the boat's software stack — the sort of thing you want working at
anchor with no signal. Code:
slackwater-web,
src/pinGlyphs.ts.

Related: porting a tide engine to Swift with the original as the test oracle — the other half of getting numeric code to tell you when it is wrong, and offline tidal current predictions from harmonic constituents — what the pins in this post are actually showing.

claude setup-token fixed my launchd auth and dropped my connectors

Bryan Clark — Tue, 01 Sep 2026 23:54:55 +0000

If you added CLAUDE_CODE_OAUTH_TOKEN to a scheduled claude -p job, your claude.ai connectors — Gmail, Google Calendar, Google Drive — are gone, and nothing told you. A setup-token "can only make model requests", and it sits above your /login session in Claude Code's auth precedence, so setting it silently demotes the session the connectors ride on. They don't error; they never enter the tool list. Keep the keychain session primary and make the token an explicitly-degraded fallback. Jump to the fix.

I run a daily scheduled agent job on a Mac: a launchd agent fires a headless claude -p at 06:05, the agent reads a planning repo, sweeps my email and calendar for anything due, and writes a standup file. It has been reliable for months. Then it was blind for four days and every one of those runs told me it had succeeded.

The problem: launchd, a keychain, and a fix that worked

One morning the job just didn't produce anything. The log had the useful part:

=== daily-standup 06:05 ===
OAuth session expired and could not be refreshed
=== daily-standup FAILED (exit 1): no artifact written ===

(On current builds this surfaces as Login expired · Please run /login; the CLI documents both the warning and the expired state.)

This is the classic unattended-macOS problem. Claude Code stores subscription credentials in the encrypted login Keychain, and a job launched by launchd on a machine that woke from sleep is not the same thing as you sitting at a terminal. There is a long-running open bug where a process tree rooted at launchd reports "not logged in" against credentials that security find-generic-password can read perfectly well from the same context. Whatever the precise cause on any given day, the symptom is the same: an interactive login is not a durable credential for a background job.

Anthropic documents the answer, and it is a good one:

For CI pipelines, scripts, or other environments where interactive browser login isn't available, generate a one-year OAuth token with claude setup-token.

— Claude Code docs, Authentication

So I ran it, put the token in the env file the job already sources, and moved on:

# scheduled wrapper — the version that "fixed" it
set -a                          # the env file is bare KEY=val with no `export`,
source "$HOME/.agent/.env"      # so allexport is what makes the child inherit it
set +a

claude --print --permission-mode bypassPermissions --model "$MODEL" "$PROMPT"

That env file holds about twenty keys — analytics credentials, a couple of API keys, and now the Claude token. One line, one variable, done. The job ran green the next morning and every morning after.

Diagnosis: the run wasn't failing, it was half-blind

Four days later I noticed the standups had stopped mentioning email. Not "failed to read email" — just nothing, as if the week had been quiet. The agent's own summary of what it swept listed the repo and nothing else, and I'd skimmed past it four times.

The check took thirty seconds once I thought to run it:

$ claude mcp list
claude.ai Gmail              ✓ Connected
claude.ai Google Calendar    ✓ Connected
claude.ai Google Drive       ✓ Connected
signalk                      ✓ Connected

$ CLAUDE_CODE_OAUTH_TOKEN=sk-ant-oat01-… claude mcp list
signalk                      ✓ Connected

Same machine, same second, same config. The only difference is one environment variable, and three servers vanish. Not "failed", not "needs authentication" — absent.

Why: connectors ride on the account session, and the token outranks it

Two things in the docs explain it completely, and I had read neither, because I was solving a launchd problem and this is filed under authentication and MCP.

One. A setup-token is deliberately narrow:

This token authenticates with your Claude subscription and requires a Pro, Max, Team, or Enterprise plan. It can only make model requests, so it can't establish Remote Control sessions or fetch claude.ai connectors. MCP servers you configure locally still work.

— Authentication § Generate a long-lived token (emphasis mine)

Two. The MCP page spells out the exclusion list, and the last bullet is this exact situation:

Connectors from claude.ai are fetched only when your active authentication method is a claude.ai subscription login. They aren't loaded, even if you previously ran /login, when:

ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, or apiKeyHelper is active

A third-party provider such as Amazon Bedrock or Google Cloud's Agent Platform is active

ANTHROPIC_PROFILE, the federation variables, or an active Anthropic profile supplies the credential

CLAUDE_CODE_OAUTH_TOKEN holds a token from claude setup-token, which can only make model requests

— Connect Claude Code to tools via MCP § Use MCP servers from claude.ai

The mechanism is the authentication precedence list. CLAUDE_CODE_OAUTH_TOKEN is rank 5. Your /login subscription credential is rank 7 — dead last:

Rank	Credential	claude.ai connectors?
1	Cloud provider (Bedrock / Vertex / Foundry)	no
2	`ANTHROPIC_AUTH_TOKEN`	no
3	`ANTHROPIC_API_KEY`	no
4	`apiKeyHelper` output	no
5	`CLAUDE_CODE_OAUTH_TOKEN` (`setup-token`)	no
6	Anthropic profile / federation credentials	no
7	Subscription OAuth from `/login`	yes

Read as a table, the shape of the trap is obvious: connectors are available under exactly one of seven credential sources, and it's the one anything else beats. Setting any of the six above your login costs you the connectors. setup-token is just the one you're most likely to add on purpose, at 6am, to fix something else.

The part that actually cost four days

Getting it wrong is forgivable. Not being told is the expensive bit, and Claude Code is otherwise good about this. For a server you configured locally that needs authentication, headless runs get an explicit signal — as of v2.1.196, "Claude Code tells Claude that the server's tools are unavailable until you authorize it," so the model can name the server instead of answering as if it were never configured.

Connectors dropped by auth method get none of that, and structurally can't: they were never fetched, so there is no server object to mark unavailable. The list the model receives is simply shorter. The model has no way to know, and answers confidently from what's left.

So the run:

exits 0
writes its artifact
reports success
and is missing half its inputs

Which is a much worse failure than the crash I was fixing. The crash cost me one standup and announced itself before breakfast.

What I tried, and why each one failed

Attempt 1 — put the token in the shared env file. This is the whole bug, and its real lesson isn't about Claude at all: source-ing a general-purpose env file into an agent runner is an authentication change, not a configuration convenience. The blast radius of set -a; source .env is every variable in that file, including the ones another tool put there. I added a key to make a scheduler happy and silently changed which account session my agent ran under.

Attempt 2 — trust the exit code. The wrapper's health check was "did claude exit non-zero", and every degraded morning it said yes. Exit 0 from an agent run is a claim about the process, not about the work. Worse, I also had the agent report its own coverage in the output — and it did, accurately, listing only the sources it could see. A self-report is only as good as the reader; four days of "swept: repo" scrolled by unread.

Attempt 3 — check for today's artifact by exact filename.

ARTIFACT="reports/$(date +%F).md"
[[ -f "$ARTIFACT" && "$(stat -f %m "$ARTIFACT")" -ge "$STAMP" ]]

This looks airtight and isn't. On the first morning of the fix, the agent inferred the wrong date and filed a perfectly good report under a filename three days in the future. The check saw no reports/2026-08-01.md, declared total failure, and fired a redundant retry over a successful run. Two fixes fell out: the heartbeat should ask "did a new artifact appear", not "did this filename appear", and the prompt should never let the model infer the date:

PROMPT="TODAY IS $(date +%F). Use that date for the filename and every date computation — do not infer it.

${PROMPT}"

Attempt 4 — find -newermt for "any new artifact". The obvious one-liner:

wrote_artifact() { [[ -n "$(find reports -name '*.md' -newermt "@$STAMP" -print -quit 2>/dev/null)" ]]; }

It returned false inside the script while the same predicate, same STAMP, same directory, returned true when I ran it by hand. It fired two more redundant degraded runs before I gave up guessing and made the check say what it saw:

echo "heartbeat: STAMP=$STAMP now=$(date +%s) newest=$(ls -t reports/*.md | head -1) mtime=$(stat -f %m "$(ls -t reports/*.md | head -1)")"

The mtime was comfortably newer than STAMP. find disagreed anyway. I never did root-cause the find(1) quirk, and I'm at peace with that: two stdlib commands answer the question outright, so the correct move was to delete the clever predicate rather than debug it. A watchdog that produces false negatives is worse than no watchdog — it trains you to ignore it.

The fix

Two changes. Keychain session primary, token as an explicitly-degraded fallback, and success means "the artifact exists and is newer than this run started":

STAMP=$(date +%s)

# Ground truth: did THIS run produce an artifact? Not the exit code, not the
# agent's self-report. `stat -f %m` is BSD/macOS; GNU coreutils is `stat -c %Y`.
wrote_artifact() {
  local newest
  newest=$(ls -t reports/*.md 2>/dev/null | head -1)
  [[ -n "$newest" && "$(stat -f %m "$newest" 2>/dev/null || echo 0)" -ge "$STAMP" ]]
}

# Primary run: the claude.ai /login session, with the token pulled OUT of the
# environment so it can't outrank it. This is the only credential that carries
# the Gmail / Calendar / Drive connectors.
OAT="${CLAUDE_CODE_OAUTH_TOKEN:-}"
unset CLAUDE_CODE_OAUTH_TOKEN
claude --print --permission-mode bypassPermissions --model "$MODEL" "$PROMPT"
rc=$?

# Fallback: an expired session must not cost a whole day's run — retry with the
# token, and say out loud that this one is degraded.
if [[ -n "$OAT" ]] && ! wrote_artifact; then
  echo "primary (keychain session) run wrote nothing, exit $rc — retrying with CLAUDE_CODE_OAUTH_TOKEN, DEGRADED: no Gmail/Calendar/Drive"
  CLAUDE_CODE_OAUTH_TOKEN="$OAT" claude --print --permission-mode bypassPermissions --model "$MODEL" "$PROMPT"
  rc=$?
fi

wrote_artifact || { echo "FAILED (exit $rc): no artifact written"; exit 1; }

The token still earns its place — it's the difference between a degraded run and no run — it just isn't allowed to be the default. And the one-line verification, which is what I should have run on day one:

# run this through the SAME wrapper the scheduler uses, not from your shell
claude mcp list | grep claude.ai

Why it matters, and the traps next door

The connector rule generalises past setup-token. Anything you set to make a headless run authenticate — an ANTHROPIC_API_KEY for billing separation, an apiKeyHelper pulling short-lived creds from a vault, a Bedrock switch — takes your connectors with it. The docs note the same for Anthropic profiles: "Features that need your claude.ai login, such as claude.ai connectors and /schedule, aren't available while one of these sources is selected." If a scheduled agent needs Gmail or Calendar, it needs the subscription login and nothing above it.

Verify under the job's environment, not yours. Every check I ran interactively passed, because interactively the token wasn't set. claude mcp list piped through the same wrapper the scheduler invokes is a different command than the one you type.

Anthropic's own answer for this shape of work is routines, not cron — scheduled cloud runs that carry your connectors by default. /schedule requires a claude.ai subscription login for exactly the reason above. If your scheduled job's whole value is the connectors, that's the shorter path; I keep mine local because it needs the machine's filesystem and a private repo clone.

Exit 0 is not success — name the artifact. This is the durable one, and it isn't about Claude Code. Every unattended agent needs a success criterion expressed as a fact about the world: a file written after the run started, a row in a table, a message delivered. Process exit status tells you the interpreter didn't crash. Ask the filesystem instead.

A degraded mode must announce itself. The fallback path prints DEGRADED: no Gmail/Calendar/Drive into a log a human eventually reads, and the agent reports its actual coverage in its output. A silent fallback is just the original bug with more steps.

Version note. The claude mcp list behaviour above is my observation on Claude Code 2.1.22x (August 2026) — the docs describe connector visibility in terms of /mcp and /status, not claude mcp list, so treat the exact output as illustrative. The underlying rule is documented and stable. Separately, there are community reports of connectors also going missing in -p mode with a perfectly good login (#36060, #43298) — a different mechanism from this one, and worth ruling out before you blame your auth.

This scheduled job files commitments for an all-electric charter catamaran that doesn't exist yet — the agent stack around it is public at naturali-agents. The boat is the reason the "did anything actually happen" question matters more than the exit code: nobody is watching the log at 06:05.

Related: launchd's minimal PATH breaks MCP servers: uv command not found — the same class of trap one layer down, where the scheduler's environment silently isn't yours; When MCP tools break, isolate the server before blaming the agent — the debugging order for missing tools; and Monitor the delivery path, not just the alarm — the dead-man's-switch pattern this heartbeat is an instance of.

Your service worker's precache can't do Range requests — PMTiles needs 206

Bryan Clark — Tue, 01 Sep 2026 16:04:38 +0000

TL;DR — Workbox's precache stores and replays a full 200 OK. pmtiles
reads the archive with HTTP Range requests and requires 206 Partial Content, so a precached .pmtiles renders fine on a cold first visit — when
no service worker is in the path yet — and throws on every visit after that.
Take .pmtiles out of globPatterns and serve it from a CacheFirst
runtimeCaching route with rangeRequests: true, plus one non-ranged warm
fetch, because a ranged request can never populate that cache on its own.
Jump to the fix. Repo:
slackwater-web.

Slackwater is an offline
tide and current app for the Salish Sea — a PWA that computes predictions on
device so it still works with no signal, which is the only condition that
matters when you're actually in Dodd Narrows. Its map screen draws a coastline
from a committed 4.78 MB PMTiles archive (public/land.pmtiles, OSM land
polygons clipped to the Salish Sea, z0–14) rendered by MapLibre. No tile
server, no network.

Getting that archive to survive offline took two tries, and the failure mode of
the first one is the entire post: it worked on the first visit and broke on
every visit after.

The problem: the map draws once, then never again

The obvious way to make a file available offline in a vite-plugin-pwa app is
to precache it. That's one glob entry:

// vite.config.ts — the first attempt
workbox: {
  globPatterns: ["**/*.{js,css,html,svg,png,json,woff2,pmtiles}"],
  // land.pmtiles is 4.78 MB; the default 2 MB cap would silently skip it
  // and the map would have no land offline.
  maximumFileSizeToCacheInBytes: 6 * 1024 * 1024,
},

Load the app in a fresh tab: coastline draws. Reload it: blank map, and this in
the console —

Error: Server returned no content-length header or content-length exceeding
request. Check that your storage backend supports HTTP Byte Serving.

That string is thrown by pmtiles' own FetchSource.getBytes, and it is
correct. Something in the path really did return the whole file instead of
the slice it asked for. It just wasn't the storage backend.

Diagnosis: precache is 200-shaped, and it only joins the path on visit two

Two facts collide.

1. A precache response is always a full 200. Workbox precaches by
cache.put()-ing complete responses, and the Cache API flatly refuses to store
a partial one — try it and you get
TypeError: Failed to execute 'put' on 'Cache': Partial response (status code 206) is unsupported.
So the precache route can only ever answer with the entire body and status 200.
It doesn't read the Range header; it has nothing to slice.

2. pmtiles treats a 200 as a hard error, deliberately. Here's the check, from
the pmtiles source:

// some well-behaved backends, e.g. DigitalOcean CDN, respond with 200 instead of 206
// but we also need to detect no support for Byte Serving which is returning the whole file
const contentLength = resp.headers.get("Content-Length");
if (resp.status === 200 && (!contentLength || +contentLength > length)) {
  if (controller) controller.abort();
  throw new Error(
    "Server returned no content-length header or content-length exceeding request. Check that your storage backend supports HTTP Byte Serving."
  );
}

A 200 whose Content-Length exceeds what was requested means byte serving isn't
working — pmtiles aborts rather than silently downloading a multi-gigabyte
archive to read a 16 KB directory. Against a Workbox precache entry that's
exactly what it sees, every time.

And the timing is what hid it. A service worker only controls navigations
that start after it activates. On a first-ever visit the SW registers,
installs and activates, but the page that installed it was never controlled —
so /land.pmtiles went straight to the static host, which handles Range
properly and answered 206. Everything worked. On the next load the SW is in
the path, the precache answers, and it throws.

The bug is invisible in exactly the state you test in (fresh incognito tab,
hard reload after a deploy) and present in the only state a user is ever in.

What we tried, and why it failed

Attempt 1 — precache it and raise the size cap

The config above. Worth keeping the sub-lesson even though the approach was
wrong: maximumFileSizeToCacheInBytes defaults to 2 MB, and a file over
that is dropped from the precache manifest with a build-log line and no
runtime error. The app just quietly has no offline copy. If you precache
anything big — a tile archive, a model, a wasm blob — check the manifest, don't
assume the glob matched.

Attempt 2 — a smoke test that agreed with us

The offline smoke check was green throughout. It was green because it had been
written to be:

// M5: /map opens the discovery map directly. This has to run as the very
// first navigation, before anything else on this origin installs the PWA
// service worker: once the SW is controlling, its default Workbox precache
// route serves the precached land.pmtiles as a plain 200 with no
// Range/Content-Length support, and pmtiles' own byte-serving check throws
// on that — a real bug (land.pmtiles needs workbox-range-requests wired to
// its precache route to survive a returning/offline visit), but out of scope
// for this smoke-only task.

The comment names the bug, in full, and then orders the test to avoid it. A
passing check is a claim about what ran, not about what's true — and here the
claim was "the cold path works", which was never in doubt.

Attempt 3 — runtimeCaching with rangeRequests, and an empty cache

The documented answer is a runtimeCaching route with rangeRequests: true,
which wires up
workbox-range-requests:

runtimeCaching: [
  {
    urlPattern: ({ url }) => url.pathname.endsWith(".pmtiles"),
    handler: "CacheFirst",
    options: {
      cacheName: "land-pmtiles",
      rangeRequests: true,
      cacheableResponse: { statuses: [200] },
    },
  },
],

Ship that alone and the map still doesn't work offline — and now
caches.open("land-pmtiles") is empty. Nothing was ever stored.

The reason is in the plugin's one and only callback:

// workbox-range-requests/src/RangeRequestsPlugin.ts
cachedResponseWillBeUsed: WorkboxPlugin['cachedResponseWillBeUsed'] = async ({
  request,
  cachedResponse,
}) => {
  // Only return a sliced response if there's something valid in the cache,
  // and there's a Range: header in the request.
  if (cachedResponse && request.headers.has('range')) {
    return await createPartialResponse(request, cachedResponse);
  }

  // If there was no Range: header, or if cachedResponse wasn't valid, just
  // pass it through as-is.
  return cachedResponse;
};

cachedResponseWillBeUsed — that's it. The plugin never touches the outbound
request. It slices a response you already have; it has no opinion on how you
got one.

So the first request to reach the route is pmtiles' own, carrying Range: bytes=0-16383. CacheFirst misses, goes to the network, and the origin
correctly answers 206. Then cacheableResponse: { statuses: [200] } refuses
to store it — and you can't widen that to [200, 206], because the Cache API
rejects a 206 on put() anyway. The cache stays empty, the next read misses
again, and the loop never closes.

The fix

Two pieces: the route, and one plain fetch that primes it.

// vite.config.ts
workbox: {
  // pmtiles is deliberately NOT globbed here: a precache entry is a plain 200
  // with the full body and no Range support, but pmtiles reads the archive
  // with Range requests and needs 206.
  globPatterns: ["**/*.{js,css,html,svg,png,json}"],
  runtimeCaching: [
    {
      urlPattern: ({ url }) => url.pathname.endsWith(".pmtiles"),
      handler: "CacheFirst",
      options: {
        cacheName: "land-pmtiles",
        rangeRequests: true, // serve 206 partials from the cached full body
        cacheableResponse: { statuses: [200] },
        expiration: { maxEntries: 4, maxAgeSeconds: 60 * 60 * 24 * 365 },
      },
    },
  ],
},

// src/main.tsx — the warm fetch, without which the cache above is never filled
if ("serviceWorker" in navigator) {
  const warm = () => {
    fetch("/land.pmtiles").catch(() => {});
  };
  navigator.serviceWorker.ready.then(() => {
    if (navigator.serviceWorker.controller) warm();
    else navigator.serviceWorker.addEventListener("controllerchange", warm, { once: true });
  });
}

That fetch carries no Range header, so it gets a full 200, which
cacheableResponse accepts and stores. From then on every ranged read that
pmtiles issues hits the cache and RangeRequestsPlugin slices the stored body
into the 206 it wanted — offline, forever.

The one-line version: rangeRequests: true is a read path, not a write path.
You still have to get a whole 200 into the cache yourself.

Gotchas

`serviceWorker.ready` does not mean the service worker controls your page

ready resolves once there's an active registration for the scope. It says
nothing about navigator.serviceWorker.controller, and with
registerType: 'prompt' those two diverge on the first load: 'prompt' does
not set clientsClaim, so the page that installed the worker is never
controlled. Fire the warm fetch on ready alone and on a first-ever visit it
goes straight to the network, bypasses the route entirely, and caches nothing.

Hence the two-branch guard: run it now if controller is already set,
otherwise wait for controllerchange — which is what fires when a worker
does claim, whether that's registerType: 'autoUpdate' (vite-plugin-pwa
forces clientsClaim and skipWaiting for that mode) or an update takeover
later in the session.

Test the controlled path, because that's the one users are on

The rewritten smoke check stops dodging the service worker and goes looking for
it: load online, wait for the SW, reload to gain control, wait for the
cache entry to actually exist, then cut the network and reload again.

await mapPage.goto(`${URL}map`, { waitUntil: "domcontentloaded" });
await mapPage.waitForSelector(".map-canvas .maplibregl-canvas", { timeout: 10_000 });
await mapPage.evaluate(() => navigator.serviceWorker.ready);

// NOW the active SW controls the page, so the warm fetch and the map's own
// pmtiles reads go through the runtime Range route.
await mapPage.reload({ waitUntil: "domcontentloaded" });
await mapPage.waitForFunction(() => navigator.serviceWorker.controller != null, {
  timeout: 15_000,
});
await mapPage.waitForFunction(
  async () => (await (await caches.open("land-pmtiles")).match("/land.pmtiles")) != null,
  { timeout: 15_000 },
);

const mapCdp = await mapPage.createCDPSession();
await mapCdp.send("Network.enable");
await mapCdp.send("Network.emulateNetworkConditions", {
  offline: true, latency: 0, downloadThroughput: -1, uploadThroughput: -1,
});

await mapPage.reload({ waitUntil: "domcontentloaded" });
// This is the assertion the fix exists for.
await mapPage.waitForSelector(".map-canvas .maplibregl-canvas", { timeout: 10_000 });

Waiting on .maplibregl-canvas rather than the container div matters —
the container mounts whether or not MapLibre ever got a WebGL context up. And
don't trust navigator.onLine to tell you the emulation took: under CDP
offline it lies. Prove the network is down with a real fetch() to an
unreachable origin first, and attach your error listeners after that probe,
or Chrome's own resource-load error for the probe lands in your assertions.

The nice side effect

Once .pmtiles left the precache, the largest precached chunk was the map
screen's ~1 MB of JS — comfortably under the 2 MB default — so the
maximumFileSizeToCacheInBytes bump could go away. Fewer knobs is the correct
outcome of a fix.

Slackwater is the offline tide app I want in my pocket on an all-electric
charter catamaran with no cell signal, so "works on a return visit" isn't a
nice-to-have. Code: slackwater-web.

Related: Porting a tide engine to Swift with the original as the test oracle — the prediction engine behind this same app; and A persistent-queue HTTP reporter for distress traffic over flaky internet — offline-first on the other side of the wire.

A passing check is a claim about what ran, not what's true

Bryan Clark — Mon, 31 Aug 2026 16:04:53 +0000

TL;DR — A green check tells you a code path ran and produced the value it
was compared against. It does not tell you the subject was examined. Six
independent bugs in one month, across four unrelated codebases, were all the
same shape: something reported success for a region it never looked at.
Jump to the checklist.

I hit this six times in a month. Different languages, different repos, no shared
code: a JavaScript data package, a Python MCP server, a shell wrapper, a
TypeScript PWA. Every one of them produced a clean result for something that was
never actually examined — not a wrong answer, an absent answer, dressed as a
pass.

That's a nastier failure than a wrong answer. A wrong answer is a signal. An
absent answer that reports itself as a pass is silence you've been trained to
read as safety.

Here they are, shortest first, each with the rule I'd generalize out of it.

1. A boolean that returns false for "no" and for "I can't tell"

station-corrections
exists to audit the published positions of tide and current stations. Its
headline check: is this station's coordinate sitting on land? That's a
point-in-polygon test against a bundled coastline:

// src/coastline.js — before
export function isOnLand(lat, lon) {
  const at = point([lon, lat]);
  return coastline.features.some((feature) => booleanPointInPolygon(at, feature));
}

The coastline ships clipped to a bounding box — the full planet coastline is far
too large to bundle. Outside that box there are no polygons, so some() finds
nothing and the function returns false. Which is byte-identical to the answer
it returns for a station verified to be in open water.

Three CHS current gates — Blackney Passage, central Johnstone Strait, Weynton
Passage — sat north of the clip. The audit reported them valid. It had never
looked at them. And the registry was growing north, so the silent region was
growing too.

The fix is a coverage predicate derived from the data rather than hardcoded, so
rebuilding the coastline with a different clip updates it for free:

// src/coastline.js — after
export function isWithinCoverage(lat, lon) {
  const b = coverageBounds();   // walked off the bundled polygons, memoized
  return lat >= b.minLat && lat <= b.maxLat && lon >= b.minLon && lon <= b.maxLon;
}

Then the second-order fix, which matters more: a registry station outside
coverage became a validation failure, not a note — and the coastline build
script now derives its clip box from the registry's own extent plus a margin,
floored at the proven box so it only ever grows outward. The clip can no longer
fall behind the data it's meant to check
(7d52f67,
1c06273).

Rule: a predicate over partial data needs three states, not two. If
"no" and "I have no data here" collapse to the same value, every caller that
reads false as a verdict is reporting a result it never computed.

2. A validator that iterates one side of a pair

Same package, next day. Station slugs are public URL segments, so a rename is a
breaking change. A lock file pins the slug each station last shipped with, and
check-slugs runs in CI ahead of the data checks:

// src/slugs-lock.js — before
for (const [id, lockedSlug] of Object.entries(lock.slugs)) {
  const nowSlug = current.get(id);
  if (nowSlug === undefined || nowSlug === lockedSlug) continue;
  // ...fail if the slug moved without the old value in formerSlugs
}

Read the loop bound. It iterates the lock. A station present in the data but
absent from the lock is never visited — so a brand-new station's slug entered
the public API with nothing pinned to compare against, and that slug plus every
future change to it passed green forever. The one thing the lock exists to
prevent was invisible for exactly the stations most likely to get renamed: the
new ones.

The guard is now symmetric — the lock must reflect the current knowable slugs
exactly, in both directions:

// src/slugs-lock.js — after
for (const [id, lockedSlug] of Object.entries(lock.slugs)) {
  const nowSlug = current.get(id);
  if (nowSlug === undefined) {
    problems.push(`${id}: in the slug lock but no longer in the data — its slug "${lockedSlug}" is dead`);
    continue;
  }
  // ...moved-slug check unchanged
}

for (const [id, slug] of current) {
  if (!(id in lock.slugs)) {
    problems.push(`${id}: slug "${slug}" is not in the lock — run \`station-corrections slugs\``);
  }
}

The commit message named the pattern out loud, because it landed a day after the
coastline one: "Same failure class as the coverage bug: a clean result for
something never checked"
(99afa20).

Rule: a validator comparing two collections must iterate their union.
Iterating one side can only ever find disagreements about things both sides
already know about — which is the easy half of the problem.

3. A benchmark that scored the call, not the result

I run a benchmark over the boat agent's MCP tool routing: a set of golden asks,
each with the tools a correct answer requires. Scoring:

# poseidon/bench/scoring.py
def score_ask(ask, observed_tools, observed_args=None):
    if not set(ask.expected_tools).issubset(set(observed_tools)):
        return False
    # ...optional argument matching
    return True

Set containment on tool names. It measures routing — did the model reach for
battery_state when asked about the batteries — and routing is genuinely what
the benchmark was built to compare across models. But nothing in that function
ever opens the payload.

Meanwhile, in signalk-mcp:

async def battery_state(client: SignalKClient, bank: str = "0") -> dict:
    raw = await client.get_value(f"electrical.batteries.{bank}")

"0" is the SignalK instance convention, and it's a perfectly reasonable
default — except the vessel publishes electrical.batteries.house. So the bare
tool returned this, on a boat that had been publishing battery data the entire
time:

{"bank": "0", "soc_fraction": null, "voltage": null, "current": null, "display": null}

Right tool, right arguments, no data. Score: match. The agent only got a real
reading if the person asking happened to say the word "house". This survived
months of green benchmark runs and was found by hand, mid-benchmark, reading a
transcript for an unrelated reason
(e837d92).

The tool fix is small — try "0", then discover the vessel's own banks off the
electrical.batteries subtree, and never second-guess an explicitly named bank
because answering about a different battery is worse than answering "no data".
The interesting fix is the scoring one: at least one assertion per ask has to
reach into the returned payload.

Rule: a score computed from the call examines your harness's routing. A
score computed from the result examines the system. If nothing in your
scorer opens the response body, a tool that returns nulls forever is a
perfect performer.

4. A selftest that greps for a string the command never prints

A CLI wrapper on the boat gateway sources its secrets and execs the real
binary. Scripts parse its output, so a missing environment variable has to fail
loudly and early — hence a --selftest that proves the secrets resolved on a
real invocation.

The trap: the CLI reports the same condition in two different formats depending
on the subcommand.

$ mytool audit
┌ Config warnings ──────────────────────────────┐
│ ! api.token: Missing env var API_TOKEN        │
└───────────────────────────────────────────────┘

$ mytool config validate
1 warning(s): ! api.token: Missing env var API_TOKEN

The selftest called config validate and grepped for the box title:

# before — passed while the token was genuinely missing
out="$(mytool config validate 2>/dev/null || true)"
if grep -qi "Config warnings" <<<"$out"; then
  echo "SELFTEST FAIL"; exit 1
fi
echo "SELFTEST PASS"

config validate never prints Config warnings. The grep could not match. The
selftest passed, cleanly and instantly, with a required token genuinely absent
— a check whose entire job was catching that exact state.

# after — match the thing both formats actually contain
if grep -qi "missing env var" <<<"$out"; then

What caught it wasn't review and wasn't a test. It was running the negative
control: deliberately unsetting the variable and watching whether the selftest
went red. It didn't.

Rule: you have not tested a check until you have watched it fail. Break
the thing on purpose, confirm red, put it back, confirm green. A selftest
that passes when broken is worse than no selftest — it converts an unknown
into a false known.

5. An assertion that fired before the system finished failing

slackwater-web is an
offline-first tide PWA, so its Puppeteer smoke suite loads the map with the
network cut and asserts no unexpected console errors. It looked like this:

await mapPage.waitForSelector(".map-canvas .maplibregl-canvas", { timeout: 10_000 });
assert.deepEqual(mapErrors, []);   // green

MapLibre puts a canvas in the DOM as soon as it initialises the GL context —
well before it has finished trying, and failing, to fetch its tiles and glyphs
over a network that isn't there. The assertion fired into an empty array
roughly two seconds early. Green, for the wrong reason.

The fix separates collection from judgement — a watcher that starts before the
navigation and a settle() that drains every pending handle before anything is
asserted:

const settleMapErrors = watchErrors(mapPage);
await mapPage.goto(`${URL}/map`);
await mapPage.waitForSelector(".map-canvas .maplibregl-canvas", { timeout: 10_000 });
// The canvas appears well before the map has finished failing its external
// fetches, so asserting the instant it exists passed for the wrong reason.
const mapErrors = await settleMapErrors();
assert.deepEqual(mapErrors, []);

(The same commit fixed why those errors were unreadable in the first place:
Puppeteer's msg.text() renders an object argument as its class name, so
MapLibre's minified AJAXError arrived as [object Ae] with the host it named
invisible to the noise classifier. Resolving each argument through its
JSHandle makes collection async, which is what forces the settle-then-judge
shape anyway —
32bfe09.)

Rule: an assertion is a claim about an instant. If the system is still
working when you sample, you measured a different system. Collect over a
window, settle, then judge — and be suspicious of any wait keyed on the
first sign of life rather than the last.

6. When deleting the fix leaves every check green

Same app, same evening. resolvePinStates called with no dependency object let
every unsynced station fall through to a live upstream fetch — a request storm
on every map open, throttled behind a ~24 req/min limiter, with a single
setData gated on the slowest of them. In review the pins effectively never
coloured. The fix is one argument: a rejecting fetchFn that makes state reads
cache-only, so an unsynced station stays neutral, which is the honest unknown
(afa3df1).

Now: which check catches its removal?

tsc — green. The argument is optional.
The Vitest suite — green. jsdom has no WebGL, so the map component never mounts and no test observes the call.
The production build — green.
The Puppeteer smoke test — green. A request storm against a third-party API is not a console error.

Delete the fix and all four stay green while the storm comes back. The bug is
invisible to every conventional layer, which means writing a conventional test
would be theatre. So the guard reads the source file as text:

it("resolves pin states cache-only — never fetching", () => {
  // Guarded by source text because nothing else can catch it: jsdom has no
  // WebGL so the map never mounts, and dropping the fetchFn leaves tsc, the
  // suite, the build and the smoke test all green while every map open fires
  // one live request per unsynced station at a third-party API.
  const src = readFileSync(join(__dirname, "MapScreen.tsx"), "utf8");
  const call = src.match(/resolvePinStates\([\s\S]*?\)\s*\n\s*\.then/);
  expect(call, "resolvePinStates call site not found").toBeTruthy();
  expect(call![0]).toContain("fetchFn");
});

That test is ugly. It's coupled to formatting, it can't see a rename, and it
proves nothing about behaviour. It is also the only thing in the repo that goes
red when the fix is removed
(88ec107).

Rule: if you delete the fix and nothing goes red, the missing thing is the
check, not the bug. A real guard with a known ceiling beats an elegant guard
that doesn't exist — as long as the test says, in the test, why it's shaped
that way.

What they have in common

None of these were flaky. None were wrong answers. Every one of them was a
scope bug in the check itself: the set of things examined was smaller than
the set of things claimed, and nothing in the output distinguished the two.

That's the whole thesis. A green check is a claim about what ran. Reading it as
a claim about what's true requires an extra assumption — that the check's
subject and the check's coverage are the same set — and that assumption is
exactly what nobody verifies, because verifying it feels like testing the tests.

Two structural tells show up in five of the six:

The check consumed a partial data source and had no way to say so. The clipped coastline, the lock file, the jsdom environment, the two-second window. Partial coverage plus a boolean result is a machine for manufacturing false confidence.
Nobody had ever watched it fail. The negative control caught #4 within a minute. It would have caught #1, #2 and #3 just as fast.

The checklist

Six questions. They take a minute, and each one is a bug I actually shipped:

Does this validator iterate both sides of the pair? Or does it loop over one collection and silently exempt everything the other one knows about?
Can this predicate return "no" and "I don't know" as the same value? If yes, every caller reading it as a verdict is reporting a result nobody computed.
Does the score examine the result, or just the call? Set containment on tool names, exit codes, "did it get called" — all of these pass on an empty payload.
Did you run the negative control? Break the thing, watch the check go red, put it back. If you've never seen it fail, you don't know it can.
Does the assertion fire after the system has finished, or at the first sign of life? Waiting on the first artifact to appear is not the same as waiting for the work to finish.
If you deleted the fix, which check would go red? If the honest answer is "none", write that check first — even if the only shape available is an ugly one.

None of this is exotic. It's the difference between a test suite that tells you
the code works and one that tells you the suite ran.

I'm building the software stack for an all-electric charter catamaran — data
packages, SignalK plugins, MCP servers, an offline tide app — mostly in public.
Code for the bugs above lives in
station-corrections,
signalk-mcp and
slackwater-web.

Related: Porting a tide engine to Swift with the original as the test oracle — what a check looks like when it does examine the result · Monitor the delivery path, not just the alarm — the same silence, one layer out · Bench your own workload before you switch LLM vendors — the benchmark from vignette 3, before its scoring blind spot showed up

A current station's coordinate is a label, not the hydraulic control

Bryan Clark — Mon, 31 Aug 2026 14:11:48 +0000

TL;DR — A published tide or current station coordinate is an identifier for the place, not a measurement of where the physics happens. At Dodd Narrows the CHS current station sits 120 m from the pass's actual throat, on water twice the cross-sectional area. Any model that scales speed by continuity (u ∝ 1/A) divides by the wrong area there and over-predicts the throat by 91%. Fix it once, in shared reference data, with a source: field recording where the real coordinate came from. Jump to the fix.

Problem

We build offline tidal-current prediction for the Salish Sea. Harmonic prediction at a station is a solved problem — you fit constituents and evaluate them. Drawing a current field, a speed and direction at arbitrary points near a pass, is not.

The obvious source is a coarse ocean model. It is useless at rapids. Here is the coastal-ocean mesh against the fitted station truth at three BC/WA gates:

gate               mesh cell    truth      mesh reads   ratio
Dodd Narrows       516 m        6.2 kn     1.46 kn      0.24

A 516 m mesh cell cannot see an 80 m rock gut. It smears a 6-knot rapid into a 1.5-knot drift, and animating that would make bad data more persuasive than no data.

So instead: grow a small patch of certified geometry around each gate. Take a 10 m bathymetric DEM, trace a thalweg through the pass, cut cross-sections every 50–200 m, and scale the station's own fitted speed along the reach by mass conservation:

|u|(x) = |u|_anchor · A_anchor / A(x)

One anchor station, one reach, one law. It certified cleanly at Tacoma Narrows against held-out NOAA truth in both directions (speed median 0.32 kn and 0.20 kn against a 0.5 kn bar, no flood/ebb sign flips). It certified at Seymour Narrows. At Dodd Narrows it fell over, and the failure did not look like a data-quality problem:

$ ./sensitivity.py dodd-narrows
anchor area swung 19.9% under variant 'shift+0.5' — anchor placement is sensitivity-unstable

That number is not prediction error. It is anchor stability: perturb the section grid by half a spacing, or recompute areas at chart datum instead of mean water level, and see how much the anchor section's own cross-sectional area moves. Every scale in the patch is A_anchor / A(x), so a wobbly A_anchor multiplies straight into every cell. Over 10% and the pass ships nothing.

Dodd came in at 19.91 / 20.52 / 19.67% across the three variants. All three. Not marginal — double the bar, on every axis.

Diagnosis

The tell was in a field the pipeline prints and nobody reads:

$ ./sections.py dodd-narrows
  kept_range [18, 18] of [0, 41] — throat 80.0 m (strip minimum (anchor is not the throat)),
  flare 1.3 → limit 104.0 m

anchor is not the throat. That string exists because the code has to decide what "the throat" is before it can decide where the patch ends:

FLARE = 1.3        # first section wider than FLARE x throat ends the patch
THROAT_TOL = 1.1   # anchor may exceed the strip minimum by this and still be "the throat"

min_w = min(s["width_m"] for s in sections)
throat = sections[k]["width_m"]          # k = section nearest the anchor station
throat_ref = "anchor section"
if throat > min_w * THROAT_TOL:
    throat, throat_ref = min_w, "strip minimum (anchor is not the throat)"

Normally the anchor is the throat: you anchor at the rapid, the rapid is the narrowest water, scale is 1.0 there by construction. Seymour Narrows behaves exactly that way — anchor section, 770 m, done.

Dodd does not. The section nearest the published coordinate is 160 m wide. The narrowest section in the strip is 80 m, two sections away.

Two things follow, and they are different bugs.

1. The published rate is a throat number, so anchoring off the throat over-predicts. The reach-mean cross-sectional areas are 2322 m² at the published coordinate and 1216 m² at the gut. CHS publishes ~9.43 kn as Dodd's spring maximum — that is the fastest water in the pass, which is the water at the control. Feed it in at a section with 1.9× the area and continuity dutifully scales it up:

9.43 kn × 2322 / 1216 = 18.01 kn at the gut

Eighteen knots. Dodd is a nine-knot rapid. The model is not wrong; the anchor is in the wrong place, so the model is answering a question nobody asked.

2. The published coordinate sits on the flare, where geometry is unstable. Section 18 is on the shoulder where the channel opens out. Nudge the section grid 25 m and the area at that station moves a lot; do the same at the gut and it barely moves. That is where the 19.9% comes from.

Now the part that took the longest to accept: this is not a data error. Query the station metadata directly and it is internally consistent:

$ curl -s "https://api-iwls.dfo-mpo.gc.ca/api/v1/stations?code=..." | jq '.[0] | {officialName, latitude, longitude}'
{
  "officialName": "Dodd Narrows",
  "latitude": 49.134351,
  "longitude": -123.817132
}

Six decimals. That coordinate is not sloppy — it is precise about the wrong thing. It exists so a mariner can find "Dodd Narrows" in a list and a chart plotter can drop a pin near the pass. It was never a claim about which cross-section the predictions describe. The same is true across the region, and the giveaways are visible:

Seymour Narrows   50.133333 / -125.35      ← lands on its control section (by luck)
Dodd Narrows      49.134351 / -123.817132  ← 120 m SSE of an 80 m gut
Porlier Pass      49.015    / -123.585     ← three decimals, exactly round

Porlier's coordinate is 49.015 / -123.585. Three decimals, both values landing on a round thousandth. Nobody surveyed that. It is a label.

What we tried (and why it failed)

Blamed the section spacing

Dodd's gut is only ~200 m long, so we cut sections at 50 m — finer than the pipeline's usual 100–200 m band. Reasonable suspicion: 50 m is under-sampled noise. So we re-ran at 100 m.

spacing   anchor area swing (shift+½ / shift−½ / datum=CD)
50 m      19.91 / 20.52 / 19.67 %
100 m     27.62 / 33.64 / 20.33 %   ← worse

Worse on every axis, and the strip's narrowest section reads 100 m instead of 80 m — the coarser grid stops resolving the gut at all. Spacing was not the cause. Recorded in the pass inputs so it never gets re-suspected.

Moved the anchor to the DEM throat, expecting that to be the fix

It is the obvious next move, so we ran it as a diagnostic before touching any canonical data:

anchor placed at section 20 (the 80 m gut)
  shift +½ spacing    8.60 %   ✓ inside the 10 % bar
  shift −½ spacing    9.58 %   ✓
  datum = chart datum 22.67 %  ✕ worse than before the move

Two axes fixed. The third got worse. The datum=CD variant recomputes every area at chart datum instead of mean water level — at Dodd that removes 3.08 m of water from the section. The gut carries about 12 m over an 80 m width, so 3.08 m is a quarter of its depth: Dodd's cross-sectional area is a strong function of tide stage in a way Seymour's 90–150 m depths simply are not (datum=CD moves Seymour's anchor area by 4.56%).

This is the finding that reframed the whole thing. The position error and the datum sensitivity are two separate problems that happened to sum to the same failing number. Moving the anchor was necessary and not sufficient, and if we had only ever looked at the aggregate "19.9% vs 10%" we would have concluded the fix didn't work and reverted it.

Made the area tide-stage dependent

If area is a function of stage, stop treating stage as noise and model it: compute each section's area at seven chart-datum heights (0…6 m), evaluate the paired tide station's fitted model at runtime, interpolate the scale. The interpolation itself is fine — held-out half-metre heights come back within 0.9164%, and with the datum axis modelled rather than swept, the corrected anchor clears placement sensitivity.

And Dodd still ships nothing. With a stable anchor, the surviving certified range collapses to [20, 20] — a single transect, no neighbour to triangulate an interval from, zero cells packed.

Tried the other bathymetry source

The fallback grid, a CHS NONNA-10 tile, resolves the gut at 50 m. Same ending:

1.3× flare rule          → one transect
1.6× diagnostic widening → adjacent 80 m sections fail the unchanged 10 % bar by 12.6–32.9 %

No bar was relaxed and no singleton transect was inflated into an area. The ruling instead is a station-local indicator centred on Dodd's corrected coordinate, drawing no fill, outline or boundary — nothing that suggests a certified spatial extent.

The fix

The whole shipped change is one coordinate and one provenance string, in the registry that owns station identity for every consumer in the workspace:

 chs-dodd-narrows:
   name: Dodd Narrows
   context: Nanaimo
-  position: [49.1344, -123.8171]
+  position: [49.13546639419797, -123.81735084108287]
   provider: chs
   cities: [Nanaimo]
+  source: GSC West Coast Topo-Bathymetric DEM v2 hydraulic control section

Plus the test that pins it, because a coordinate with no test is a coordinate somebody rounds:

const resolved = resolve({ id: "chs-dodd-narrows" });
assert.equal(resolved.latitude, 49.13546639419797);
assert.equal(resolved.longitude, -123.81735084108287);

const dodd = loadRegistry(yaml).get("chs-dodd-narrows");
assert.equal(dodd.source, "GSC West Coast Topo-Bathymetric DEM v2 hydraulic control section");

Three details that are the actual work:

The coordinate was recovered, not inferred. The pipeline had already recorded "~130 m off the control", and it would have been easy to project 130 m along the thalweg bearing and call it done. Instead: regenerate the sections from the DEM and read the generated centre of the minimum-width section straight out.

$ ./sections.py dodd-narrows
$ jq '.sections | to_entries | min_by(.value.width_m) | {index: .key, center: .value.center, width_m: .value.width_m}' \
    passes/dodd-narrows.json

The offset between the two shipped positions comes out at 120.0 m on a 351° bearing — close to the recorded ~130 m, but not equal to it, because the recorded figure was section-centre to section-centre and the label is not on the centreline. Derive the number you are shipping from the artifact that produced it.

Full precision, no rounding. 49.13546639419797 is not false precision — it is the generated section centre, and rounding it re-introduces exactly the ambiguity the fix removes. The old value, 49.1344, was the published coordinate rounded to four decimals. Rounded labels are how a coordinate stops being traceable.

source: is the load-bearing field. The registry's default provenance is documented once, repo-wide: names are hand-written, positions come from the fitting pipeline. Dodd's no longer does — it comes from a specific DEM version and a specific derivation. Without that string, in six months this is an unexplained coordinate that disagrees with the government's, and the safest-looking action is to "fix" it back.

Dataset: Canada west coast topo-bathymetric digital elevation model, Geological Survey of Canada Open File 8963, 10 m grid, OGL – Canada.
Station predictions: CHS tidal current predictions via the IWLS API.

Why it matters, and the traps next door

Published reference coordinates are identifiers. Every published station, gauge, buoy and sensor coordinate you consume was chosen to name and find a thing. Your model may need to know where a physical process happens. Those are different requirements, and the data does not tell you which one it is satisfying. The 120 m that is a rounding error on a chart is a factor-of-two error in a cross-sectional area.

A rule that works at one station and not the next is not a rule you have. Seymour's coordinate lands on its control section, so Seymour certified and nobody learned anything. Dodd and Porlier both do not. If your first integration succeeds, you have one sample.

Detect it, don't eyeball it. The single most useful line in that pipeline is the THROAT_TOL branch — "the section nearest the anchor is not the narrowest in this strip." That is a cheap, mechanical assertion any geometry-anchored model can carry, and it turns a subtle physical misplacement into a printed string.

Watch the walk that keeps the anchor by construction. Dodd's flare limit is 1.3 × 80 m = 104 m, and the surviving kept_range is [18, 18] — the 160 m anchor section, which itself violates the limit. The bounds walk starts at the anchor and only tests neighbours, so a single-element range is never checked against its own rule. If you write an outward walk from a seed, decide deliberately whether the seed is exempt.

Fix it in the shared registry, not the consumer. The tempting patch is a local override in the pipeline that needed it. Then the app has a different Dodd from the agent tool surface, which has a different Dodd from the harmonic fitter, and the reason lives in a commit message in one repo.

Be honest about what "shared" buys, though. It is not instant propagation — one reader takes it on a version bump, one needs its build-time generator re-run, and one is Python with no npm at all and vendors a copy of the JSON. That last one keeps a drift test whose entire job is to fail when the copy goes stale:

@pytest.mark.skipif(not REAL.is_file(), reason="sibling station-corrections not present")
def test_bundle_matches_published_registry():
    assert json.loads(BUNDLE.read_text()) == json.loads(REAL.read_text()), (
        "vendored _registry.json drifted from station-corrections/data/registry.json"
    )

What the registry actually buys is that the derivation and the review happen once. Nobody re-derives the coordinate, nobody re-argues whether it is right, and nobody ships a second, quieter answer.

Don't let a consumer's need bend the registry's membership rule. The plan for this change also wanted to pair Dodd with Nanaimo Harbour as its tide reference, because the stage-aware model needed a water-level source. Nanaimo Harbour is not a CHS-designated reference port, and that designation is the registry's entire admission rule for tide stations — an external rule, chosen precisely so a hand-curated list can't quietly grow into a mirror of the whole provider station table. So the pairing did not land. Nanaimo stayed a runtime input to the pipeline that needed it, and the registry kept its rule. A position correction is the registry's job; a private modelling dependency is not.

The last one is the uncomfortable one. The model this coordinate was recovered for does not ship. Dodd draws no current field today, from either bathymetry source. The coordinate correction shipped anyway, and it was worth doing on its own: the pass's identity is right for every reader, anything drawn at Dodd is now drawn on the real constriction, and the next attempt at a field model starts from a good anchor and a written-down reason. Reference-data corrections outlive the pipelines that discover them. That is a good argument for landing them separately, in their own repo, with their own tests — not as a commit inside the feature that found the bug.

Close

This came out of building offline current prediction for a boat that will spend a lot of time waiting on tidal gates in the Salish Sea. The station registry is public and MIT: @sailingnaturali/station-corrections — names, contexts and positions for tide and current stations in both countries, shipping no provider-minted station id so it stays redistributable. The Dodd change is PR #15, released in v2.9.2.

A localeCompare in a build script minted different public URLs per machine

Bryan Clark — Sun, 30 Aug 2026 18:40:18 +0000

TL;DR — String.prototype.localeCompare with no locale argument uses
whatever collation the runtime happens to have. If a sort ordered by it
decides an identifier — a slug, a first-wins registry key, an index — that
identifier is pinned to the machine, not to the repo. Sort by codepoint
(a < b ? -1 : a > b ? 1 : 0) with an id tiebreak for anything a URL is
built from, and keep localeCompare for what a human reads.
Jump to the fix.

Slackwater is an
offline-first tide and current app for the Salish Sea. It bundles 133 NOAA
harmonic current stations, and each one gets a human-readable URL —
/deception-pass, /the-narrows, /hale-passage. Those URLs get shared. They
are the closest thing a static PWA has to a primary key.

NOAA gives a lot of those stations the same name. Of the 133 bundled stations,
40 share a landmark name with at least one sibling — 17 names in all —
distinguished only by a qualifier NOAA tacks on after a comma:

$ node -e 'JSON.parse(require("fs").readFileSync("src/data/currents.json"))
    .filter(s => s.name.startsWith("Hale Passage"))
    .forEach(s => console.log(s.id, JSON.stringify(s.name)))'
noaa/PUG1529 "Hale Passage, East end"
noaa/PUG1530 "Hale Passage, West end"
noaa/PUG1710 "Hale Passage, east of Lummi Point"

Three stations, one slug: hale-passage. So there's a deterministic ladder that
hands out the collisions — plain slug, then -current, then the qualifier
folded back in, then the NOAA station id:

export function assignSlug(
  r: { slug: string; name: string; context: string },
  id: string,
  used: Set<string>,
): string {
  const candidates = [
    r.slug,
    `${r.slug}-current`,
    toSlug(`${r.name} ${r.context}`),
    `${r.slug}-${id.replace("noaa/", "").toLowerCase()}`,
  ];
  const slug = candidates.find((c) => !used.has(c)) ?? candidates[candidates.length - 1];
  used.add(slug);
  return slug;
}

Deterministic — given an order. assignSlug is called in a .map() over the
station array, so the first station the array reaches claims rung 1. Which
station that is depends entirely on how the array got sorted.

Problem

The array gets sorted in a build script, one line, entirely unremarkable:

// scripts/build-currents.mjs
const stations = bundle.stations
  .filter(/* harmonic only, primary bin, in-bounds, non-zero amplitude */)
  .map(/* reshape */)
  .sort((a, b) => a.name.localeCompare(b.name));

writeFileSync("src/data/currents.json", JSON.stringify(stations));

Two facts about that output file turn a sort into a URL problem.

One: it's gitignored.

$ grep -n currents.json .gitignore
5:src/data/currents.json

Two: every developer-facing command rebuilds it.

"scripts": {
  "build:data": "npm run build:stations && npm run build:currents",
  "dev":   "npm run build:data && vite",
  "build": "npm run build:data && tsc && vite build",
  "test":  "npm run build:data && vitest run"
}

So: whichever machine runs dev, build or test regenerates the 186 KB
station file, in that machine's sort order, and the ladder assigns public,
permanent, already-shared URLs off the back of it. There is no tracked artifact
anywhere in the repo recording which station got hale-passage-current last
time.

Diagnosis

localeCompare called with no locale uses the runtime's default collation, and
collation is not codepoint order. The collision that bites this dataset is the
simplest one there is — case.

$ node -e '
  const a = "Hale Passage, West end", b = "Hale Passage, east of Lummi Point";
  console.log("localeCompare:", a.localeCompare(b));  // 1  -> West sorts AFTER east
  console.log("codepoint:    ", a < b);               // true -> West sorts BEFORE east
'
localeCompare: 1
codepoint:     true

Codepoint order puts W (0x57) before e (0x65). Collation folds case at the
primary level and puts east before West. Both are correct sorts. Only one of
them can be the URL.

I re-ran the whole pipeline under both comparators and diffed the assigned
slugs:

$ node scratch/probe.mjs
distinct base names: 110
names with >1 station: 17
stations in a collision group: 40
FLIP noaa/PUG1530 codepoint: hale-passage-current            localeCompare: hale-passage-west-end
FLIP noaa/PUG1710 codepoint: hale-passage-east-of-lummi-point localeCompare: hale-passage-current
FLIP noaa/PUG1546 codepoint: pickering-passage-current        localeCompare: pickering-passage-west-of-squaxin-island
FLIP noaa/PUG1547 codepoint: pickering-passage-off-graham-point localeCompare: pickering-passage-current
FLIP noaa/PUG1526 codepoint: the-narrows-current              localeCompare: the-narrows-north-end-west-side
FLIP noaa/PUG1524 codepoint: the-narrows-north-end-midstream  localeCompare: the-narrows-current
stations whose public slug differs: 6

Six stations, three name groups — Hale Passage, Pickering Passage, The Narrows.
Not a hypothetical: switching the comparator immediately reassigned six live
URLs.

The severity comes from the gitignore, not from the sort. A sort that varies is
a nuisance. A sort that varies, feeds a first-wins identifier, and writes to an
untracked file is a class of bug where the wrong output can never appear in a
diff. Nobody reviews it because there is nothing to review.

What I tried (and why it failed)

Pin the locale

The obvious fix. localeCompare is non-deterministic because the locale is
implicit, so make it explicit:

.sort((a, b) => a.name.localeCompare(b.name, "en-US"));

Identical output. So I widened it — every ladder outcome, under fifteen locales
and collation options:

$ node scratch/locales.mjs
distinct slug outcomes across locales: 1
  group 1 en-US, en-GB, de-DE, fr-FR, sv-SE, tr-TR, cs-CZ, da-DK, ja-JP,
          zh-CN, pl-PL, es-ES, en-US-u-kf-upper, en-US-u-ka-shifted, en-US-u-kn-true
codepoint vs any-locale differing slugs: 6

Every locale I could name agrees with every other locale, and all fifteen
disagree with codepoint order. Pinning the locale doesn't touch the axis that
actually flips this. The en-US-u-kf-upper case-first keyword doesn't do it
either — case-first reorders A against a for the same letter, not W
against e.

That's worth sitting with, because it's the opposite of the reassuring result.
The variation isn't "someone in Germany gets different URLs." It's:

ICU presence. Node's own docs mark String.prototype.localeCompare as "partial (not locale-aware)" under --with-intl=none, meaning it "carries out its operation just like the non-Locale version of the function." A Node without full ICU produces the codepoint order in the figure above — literally the other column. Node ships full-icu by default now, but self-built, distro-packaged and size-trimmed container images all still exist.
ICU and CLDR version. Collation tables ship with ICU, and ICU tracks CLDR. This machine is node v24.1.0 / icu 77.1 / cldr 47.0. A different Node major is a different table. Punctuation and symbol weights are exactly the part of the table that gets revised.

Both of those are properties of the machine. Neither is pinned by anything in
the repo. A comparator whose answer is a property of the machine cannot be
allowed to name a URL — and I couldn't demonstrate the flip by changing LANG,
which is precisely why it would have survived a casual look.

"Just use a stable sort"

Wrong lever, and a common reflex. Array.prototype.sort has been
required to be stable since ES2019;
V8 has been stable since Node 11. Stability guarantees that equal elements keep
their relative order. It says nothing about a comparator that decides
inequalities differently on a different machine. A perfectly stable sort with a
drifting comparator drifts.

Stability is worth a thought here, though, for the opposite reason: two
stations could genuinely have equal names, at which point stability preserves
whatever order the vendored extract happened to be in — which is also not
something the repo pins. Hence the id tiebreak in the fix.

Commit the generated file

Tempting: check currents.json in, and a reordered build shows up as a diff.
Real, but it's the wrong shape. It makes a wrong build reviewable, not
impossible — someone still has to notice 186 KB of reordered JSON in a PR and
work out that line 4,012 moving means a URL changed. And the file re-churns on
every NOAA re-vendor, so the signal drowns immediately. Fix the comparator; use
a test for the noticing.

The unit test that was already green

The ladder had tests. They were added in the same session, one commit earlier,
and they passed under both orderings:

it("pins the ladder order: exact slug, then -current, then the qualifier folded in", () => {
  const used = new Set<string>();
  const a = assignSlug({ slug: "alki-point", name: "Alki Point", context: "1 mile West of" }, "noaa/PUG1502", used);
  const b = assignSlug({ slug: "alki-point", name: "Alki Point", context: "West of" }, "noaa/PUG1516", used);
  const c = assignSlug({ slug: "alki-point", name: "Alki Point", context: "near the ferry dock" }, "noaa/PUG1599", used);
  expect(a).toBe("alki-point");
  expect(b).toBe("alki-point-current");
  expect(c).toBe(toSlug("Alki Point near the ferry dock"));
});

Read what it actually asserts. It constructs three station objects in an array
literal, in the order it wants, and hands them to assignSlug one at a time.
It proves the ladder walks its rungs correctly. It cannot fail because of a
re-sort, because it never touches the sort — the third argument is even a
freshly-constructed Set, so it doesn't touch the real used set either. A
green test here is a claim about the ladder, not about the URLs the app ships.

The fix

Two parts. Sort by codepoint, with the station id as the tiebreak:

// scripts/build-currents.mjs
// Codepoint compare, not localeCompare: this file is gitignored and rebuilt on
// whatever machine runs dev/build/test, and localeCompare with no locale pinned
// uses the runtime's default collation — which can mint different public slugs
// for the colliding station names on different machines. The id tiebreak keeps
// ties (equal names) deterministic too.
.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : a.id < b.id ? -1 : 1));

And pin the real pipeline's output — not a constructed case — for one known
collision:

it("pins the real Alki Point ladder outcome (catches a re-sort silently reassigning public slugs)", () => {
  // Pinned to actual output from the build pipeline, not constructed — a
  // locale-dependent or otherwise reordered sort in build-currents.mjs would
  // flip which one gets the bare slug without any other test catching it
  // (currents.json is gitignored, so there's no tracked diff to notice either).
  const oneMileWest = resolvedNoaaCurrentStations.find((s) => s.id === "noaa/PUG1502")!;
  const west = resolvedNoaaCurrentStations.find((s) => s.id === "noaa/PUG1516")!;
  expect(oneMileWest.slug).toBe("alki-point");
  expect(west.slug).toBe("alki-point-current");
});

resolvedNoaaCurrentStations is the module the app itself imports, built from
the generated JSON. npm test runs build:data first, so this assertion runs
against a freshly generated bundle on every machine and in CI. That's the whole
point: it's the only artifact in the repo that says which station owns which
URL, and it's an assertion rather than a 186 KB blob.

Why it matters / gotchas

Split the two jobs of string comparison. Collation answers "what order
should a human read these in." Codepoint comparison answers "which of these two
strings is canonically first." Sorting a station list for display should stay
localeCompare — an app used in Québec should file Île sensibly. Anything a
URL, cache key, content hash, or first-wins registry is built from must be
codepoint, because the answer has to be a property of the data and not of the
machine. Intl.Collator is not an escape hatch; it's the same ICU tables with a
nicer API.

The generalization is bigger than sorting. The dangerous pattern is
ordering that becomes an identifier in an untracked artifact. Slugs are one
instance. So are array-index ids in generated code, "first registration wins"
plugin keys, and anything downstream of Object.keys(), fs.readdir(), or a
glob. Ask of any generated file: if its order changed, would anything permanent
change with it — and would a diff show me?

Ship the redirect, but don't count it as the fix. Slackwater resolves a URL
segment in three passes: current slug, then a recorded former slug, then the
provider id, so an old shared link still lands on the right station. That's a
seatbelt. It only helps for a slug someone noticed had changed and wrote down.
The bug here is specifically the class where nobody notices.

Grep your build scripts, not just your app code. localeCompare in a
component that renders a list is fine. localeCompare in something that writes
a file is a question worth answering every time:

$ rg 'localeCompare|Intl\.Collator' scripts/

I run this stack on a boat — an offline-first PWA for tides and currents in the
Salish Sea, part of the software for an all-electric charter catamaran. The
Slackwater web app is public at
sailingnaturali/slackwater-web.

Related: two false walls in NOAA's currents API covers where these station records come from, and porting a tide engine to Swift with the original as the test oracle is the same test suite from the other end — pinning real output instead of eyeballing it.

plugins.allow is exclusive: the warning's suggested fix disabled 44 plugins

Bryan Clark — Sat, 29 Aug 2026 02:48:10 +0000

A config warning told me, every single turn, that plugins.allow was empty — and helpfully printed the one-line fix. I pasted it onto the live box. Enabled plugins went 47 → 3, because plugins.allow is an exclusive allowlist over all plugins, including the ~46 that ship bundled. The two that mattered were the fallback model provider and bundle-mcp, the plugin every MCP tool hangs off. Reverted in a minute. The durable fix wasn't code — it was a comment in the config recording why the warning stays unfixed. Jump to the fix.

The setup: OpenClaw running as an agent gateway on a Raspberry Pi next to a SignalK server, so I can DM a boat and ask it how much water is under the keel. I wrote up that build already. This post is about one line of its log output.

The problem: a warning with a fix attached

Every turn, the gateway logged this:

plugins.allow is empty; discovered non-bundled plugins may auto-load: deepseek

That string is documented behaviour: "When plugins.allow is unset and non-bundled plugins are auto-discovered from the workspace or global plugin roots, startup logs plugins.allow is empty; discovered non-bundled plugins may auto-load: ...".

It's a reasonable warning. Without an allowlist, a plugin that appears in a discovery root gets loaded without anyone approving it. And the fix looks trivial — there is exactly one non-bundled plugin on the box, deepseek, the provider for the primary model. So: allow it, and nothing else can sneak in.

{
  "plugins": {
    "allow": ["deepseek"]
  }
}

One key. One value. The thing the warning is asking for. I put it on the live gateway.

What it actually did

$ openclaw plugins list --enabled | wc -l
3

Three. It had been 47, out of 68 discovered.

The agent still answered on Telegram, cheerfully, in a way that made it look fine. Ask it for the depth and it no longer had a depth tool.

Diagnosis: "allow" is exclusive over everything

The mental model I had — and I suspect it's the common one — is that an allowlist governs the untrusted things: the plugins that show up from outside, the ones the warning is literally about. Bundled plugins ship with the product; you don't allowlist your own dependencies.

That is not what the key means. From the plugin docs:

plugins.allow is an exclusive allowlist. Plugin-owned tools outside the allowlist stay unavailable even when tools.allow includes "*".

"Exclusive" is the whole story, and it's exclusive over the entire inventory. The docs say so in the places where it bites a specific feature rather than in the place where you'd go looking. On ACP agents:

If plugins.allow is set, it is a restrictive plugin inventory and must include acpx, or the installed ACP backend is intentionally blocked

And in the Codex setup notes: enable the bundled codex plugin first, and include codex in plugins.allow if your config uses a restrictive allowlist. Both of those are bundled plugins. Both need to be in the list.

So "allow": ["deepseek"] doesn't mean "of the discovered plugins, permit deepseek." It means "the set of plugins on this system is: deepseek." Everything else is out, including 46 things I never chose to install and never thought of as plugins at all — because they arrived as part of the product.

The warning wasn't wrong. The suggestion was a correct fix for a narrower config than mine.

The asymmetry that makes it survivable — and confusing

Three plugins survived, not one. deepseek, because I allowed it. And then telegram and memory-core, which I hadn't.

They survived because of an escape hatch the docs call auto-activation:

Bundled opt-in plugins can auto-activate when config names one of their owned surfaces, such as a provider/model ref, channel config, CLI backend, or agent harness runtime.

My config has a channels.telegram block, so the telegram plugin came back on its own. The browser plugin docs state the same rule explicitly and confirm it beats the allowlist: an explicit root browser block "activates the bundled browser plugin even under a restrictive plugins.allow".

Now read that list of owned surfaces again — provider refs, channel config, CLI backends, agent harness runtimes. MCP servers are not on it.

My config names both of these:

{
  "channels": {
    "telegram": { "enabled": true }
  },
  "mcp": {
    "servers": {
      "signalk": { "command": "uvx", "args": ["signalk-mcp"] }
    }
  }
}

Naming the channel rescued its plugin from the allowlist. Naming the MCP server did not rescue bundle-mcp — the bundled plugin that exposes MCP server tools to the model. The MCP servers still launched. They still connected. They still passed their health probes. Their tools were simply not in the request. The MCP configuration page never mentions that a plugin is in that path at all, which is exactly why you don't think to check it.

The two lanes fail differently, and that's the trap: the lane that keeps working is the one you'd notice — messages still arrive, the agent still talks. The lane that goes dark is the one you have to go looking for.

What the two casualties actually cost

bundle-mcp was the visible one. The other was worse and would have been invisible for months:

anthropic-provider — the fallback model. The gateway runs a primary provider with a second one configured behind it precisely so that a provider outage or a lapsed key degrades instead of leaving the boat with no agent. With the allowlist on, the fallback plugin is not loaded. There is no fallback. Nothing tells you this; you find out the next time the primary is down, which is the worst possible moment and also the only moment it matters.
bundle-mcp — every tool from every MCP server. The agent goes blind to the vessel and answers from general knowledge instead, which reads as fluent and is exactly as useful as it sounds.

A resilience feature that silently stops being loaded is worse than never having configured it, because you stop budgeting for the failure it was supposed to absorb.

What I tried, and why each was wrong

1. The suggested one-entry list. Above. 47 → 3.

2. "Fine — I'll list the ones I actually use."

{
  "plugins": {
    "allow": ["deepseek", "anthropic-provider", "bundle-mcp", "telegram", "memory-core"]
  }
}

This is the seductive one, and it's a trap with a delay fuse. It fixes today's breakage and leaves ~42 bundled plugins off. Some of those are load-bearing in ways that don't show up in a smoke test — they're the ones behind a feature you use twice a year. Worse, the list is now a snapshot: the next OpenClaw upgrade that adds or renames a bundled plugin ships it into a gateway that is configured to refuse it, and the failure surfaces as "that feature just doesn't work anymore" with no error and no obvious link to a config file you last touched in August.

3. "Then enumerate all of them." Read openclaw plugins list, paste all ~46 bundled ids plus the one real one, done:

$ openclaw plugins list --json | jq -r '.[].id' | wc -l
68

Correct, and I'd have to re-audit that list on every upgrade forever. That's a standing maintenance tax to close a hole I hadn't yet measured. So I measured it.

4. Actually reading what the warning protects against. The risk is: a plugin lands in a discovery root and auto-loads without approval. Discovery reads configured paths, workspace roots, the global plugin root under ~/.openclaw, and the bundled set. On this box, the only thing that ever writes a non-bundled plugin into the global root is openclaw plugins install — which is already a privileged, deliberate act by someone with a shell on the machine. Someone who can run that can also edit the allowlist. The allowlist is not a boundary against them.

So the hole is real, and on this deployment it is one notch of defence-in-depth behind an act that already implies full control of the host. The cost of closing it is a hand-maintained 47-line inventory re-audited on every upgrade. That trade doesn't clear.

The fix

Don't set the key. Write down why.

// Plugin policy. `plugins.allow` is deliberately ABSENT — do not add it.
//
// Every turn logs:
//   plugins.allow is empty; discovered non-bundled plugins may auto-load: <id>
// and suggests `"plugins": { "allow": ["<id>"] }`. That suggestion breaks
// this gateway. Tried on the live box 2026-08-10: allow is an EXCLUSIVE
// allowlist over ALL plugins, including the ~46 bundled ones. Enabled
// collapsed 47 -> 3. Survivors: the one allowed id, plus telegram and
// memory-core, which auto-activate because config names a surface they own.
//
// What it costs if anyone tries it again:
//   anthropic-provider  the fallback model — a primary outage now leaves
//                       no agent at all, and nothing warns you
//   bundle-mcp          every MCP tool — the agent goes blind to the vessel
//
// Note the asymmetry: naming `channels.telegram` auto-activates telegram,
// but naming `mcp.servers.<name>` does NOT auto-activate bundle-mcp.
//
// The hole the warning points at is real but small: discovery only picks up
// non-bundled plugins from the global root, which nothing writes to except
// `openclaw plugins install` — already a privileged act. Closing it properly
// means listing all ~46 bundled ids and re-auditing them every upgrade.
// Not worth it. Living with the warning is the considered choice.

That comment is the actual deliverable. The code change was deleting three lines.

And the one-command check, before and after any plugin-policy edit:

$ openclaw plugins list --enabled | wc -l
47

If that number moves and you didn't intend it to, stop. It's the cheapest possible canary for a class of change whose failure mode is silence.

Why it matters

An allowlist that includes the batteries is a different feature than one that doesn't. Before you set one, find out whether it governs the things that arrived from outside or the things that arrived in the box. "Exclusive" is doing enormous work in that sentence, and the difference between the two readings is the difference between a two-item list and a 47-item one. The fastest way to find out is to count enabled units before and after on something you can revert in a minute — not to reason about it from the key name.

Suggested fixes in warnings are written for the median config. A warning that prints a ready-to-paste snippet is a genuinely good affordance, and the snippet was right for a gateway whose plugin inventory is small. Mine isn't. A one-line remediation is a hypothesis about your setup, not a patch.

The generalizable one: record why you ignored the linter. Every long-lived system accumulates warnings that are correct in the abstract and wrong to act on here. If the only artifact of "we looked into this and decided to live with it" is a chat log, then the decision has a half-life of about a month — after which someone (possibly you, possibly an agent with config-write tools) sees a helpful suggestion in a log, applies it, and re-discovers the outage from first principles. Suppressing the warning would have been worse: it hides the tradeoff instead of documenting it. A comment at the exact place someone would make the change is the cheapest durable defence there is. Mine is 30 lines guarding a key that isn't there.

There's a marine version of the same lesson: a persistent alarm nobody can explain gets muted, and then the one that mattered gets muted with it — alarm fatigue by design. The software version is quieter and the failure is the same shape.

The bit that should worry you

Nothing errored. The gateway started clean, the agent replied on the channel it always replies on, and the only symptom of losing 44 plugins was an answer that didn't have any vessel data behind it. Two of the three things that survived survived by accident — because their config keys happened to be on the auto-activation list.

I caught it in a minute because I ran the count. If I'd just sent one test message, I'd have shipped a gateway with no model fallback and no vessel access and called the warning fixed.

The gateway watches an all-electric charter catamaran that doesn't exist yet; the MCP server it went blind to is signalk-mcp (MIT).

Related: Running OpenClaw on a Raspberry Pi alongside SignalK — the build this config belongs to, including the tools.profile gotcha that hides MCP tools a second, unrelated way; When MCP tools break, isolate the server before blaming the agent — the debugging order for exactly this symptom; and MCP server or a curl recipe in AGENTS.md? — what those MCP tools were costing and earning in the first place.

Enabling Bash costs more context than seven MCP tool schemas

Bryan Clark — Thu, 27 Aug 2026 13:32:38 +0000

Nine days ago I measured MCP tool schemas against a curl recipe with a tokenizer and got a near-tie. This time I ran the agent: four arms, same model, same 12 asks, only the delivery mechanism varying. MCP was the cheapest arm — 109,779 billed input tokens for a 12-ask conversation against 158,021 for curl. The reason is not subtle: turning on the Bash tool costs ~2,700 tokens of harness prompt, more than all seven MCP schemas combined (1,160). The thing that actually costs money is neither — it is eagerly loading eleven MCP servers into every turn. Jump to the numbers.

Part one counted tokens on a desk: 874 for seven signalk-mcp tool schemas, 466 for the equivalent curl prose in an agent file, breakeven at about one tool call per turn. Static counting is honest as far as it goes, and it left the obvious question open — what does an agent actually spend when you hand it a shell instead of a tool?

So I ran it. This is part two, and the headline number went the other way from the folk wisdom.

The confound in the version of this I nearly published

The run that started this argument had an agent reading the boat with curl and looking cheap doing it. It also happened to be running with a bigger thinking budget than the MCP agent it was being compared against. Two variables, one conclusion, no experiment.

So: four arms, a 2x2 over where the domain knowledge lives and how the agent reaches it. Model, persona, thinking budget, ask set and vessel are pinned; only the delivery mechanism moves.

arm	reach	knowledge lives in
`mcp`	`signalk-mcp`'s 7 shaped tools	tested code
`curl-cold`	Bash + the server URL, nothing else	nowhere — the agent discovers it
`curl-warm`	Bash + `curl`	the system prompt (paths + SI conversions)
`cli`	Bash + `sk`, a CLI over the same `tools.py`	tested code, loaded on call

All four share one neutral persona and run flat — one agent, no subagents. That is deliberate: the production system prompt names MCP tools by hand and would have biased the Bash arms into failure.

PERSONA = """You are the ship's systems assistant aboard a sailing vessel.

Answer the crew's question in one or two sentences of natural spoken English.
Always give units in full — knots, metres, percent, volts, amps — never raw
symbols or abbreviations.

Use your tools to read live vessel data. Never guess or estimate a reading; if
you cannot get it, say so plainly."""

ARMS = {
    "mcp":       lambda: _options("", ["mcp__signalk"], _signalk_server()),
    "curl-cold": lambda: _options(_CURL_COLD, ["Bash"], {}),
    "curl-warm": lambda: _options(_CURL_WARM, ["Bash"], {}),
    "cli":       lambda: _options(_CLI, ["Bash"], {}),
}

curl-warm is the interesting arm — it is the MCP-is-a-tax recommendation implemented properly. Everything signalk-mcp encodes in Python gets moved into the prompt instead:

Read a single path with:
  curl -s $SIGNALK_URL/signalk/v1/api/vessels/self/<path/with/slashes>

Do NOT fetch the whole vessels/self tree — it is over 16 KB. Go straight to
the path you need. The paths that matter:

  environment.depth.belowKeel / .belowTransducer / .belowSurface
  environment.wind.speedTrue / .speedApparent / .directionTrue
  electrical.batteries.house.capacity.stateOfCharge / .voltage / .current
  tanks.freshWater.0.currentLevel / tanks.blackWater.0.currentLevel
  navigation.position / .speedOverGround / .headingTrue / .state
  propulsion.port.runTime / propulsion.starboard.runTime

SignalK is strictly SI. You MUST convert before answering:
  speed m/s -> knots (x1.94384)   angles radians -> degrees (x57.2958)
  ratios 0-1 -> percent (x100)    temperature kelvin -> celsius (-273.15)

Twelve asks, deliberately loaded with the SI traps — 7569827 seconds of engine time, 5.506 radians of wind direction, 0.265 of a fresh-water tank. Scope is SignalK-only: curl has no equivalent for the vault-backed servers, and benchmarking those would be a strawman.

The numbers

Three full runs, all agreeing. These are run 3, against a vessel motoring in Boundary Pass.

Per turn — what one ask costs:

arm	billed input/ask	Δ	output/ask	p50	cost/ask	correct
mcp	9,148	—	245	9.05 s	$0.0106	12/12
curl-cold	13,168	+4,020	302	5.54 s	$0.0159	12/12
curl-warm	14,881	+5,733	404	6.40 s	$0.0158	12/12
cli	13,869	+4,720	242	5.79 s	$0.0115	12/12

Per session — what holding the whole 12-ask conversation costs:

arm	billed input total	Δ	output total	session cost
mcp	109,779	—	2,936	$0.1273
curl-cold	158,021	+48,242	3,621	$0.1913
curl-warm	178,577	+68,798	4,847	$0.1895
cli	166,424	+56,645	2,902	$0.1378

Both views are here because they answer different questions, and the per-ask mean flatters the expensive arms. A resident tool schema is billed on every turn, so its weight grows with the conversation; dividing by ask count divides that back out. The session table is the one that matches a bill: the mcp-vs-curl-cold gap reads as 4k tokens per ask and 48k across one conversation.

Correctness was a wash — 144/144 across three runs. Every arm turned 7,569,827 s into 2,102.7 hours, 5.506 rad into 315°, 8.51 m/s into 16.5 knots, and all four enumerated both standing restricted-area alarms. Sonnet does not need the conversions done for it. "Shaped tools protect correctness" did not survive contact with the data, at least for this model on non-adversarial asks.

The Bash arms are consistently faster (p50 5.5–6.4 s vs 9.05 s) — a real result, and the only column where the CLI story holds up.

Why raw JSON loses, and keeps losing

A tool schema is written into the prompt once and read from cache on every subsequent turn. A curl payload lands in the transcript and is re-read, uncached-then-cached, for the rest of the session. One is a fixed cost; the other is a debt that compounds.

That is part one's ΔR — the per-read saving — playing out over a real conversation instead of a single turn. The curl arms started ~6k behind on ask 1 and finished ~48–69k behind.

Note also that curl-warm cost more than curl-cold. Moving the domain knowledge into the prompt is not free: the cheatsheet is resident on every turn, and it bought no correctness, because there was none left to buy.

The finding that inverts the intuition

The MCP-is-overhead argument treats tool schemas as the thing you pay for and a shell as the thing you get for free. So I measured the standing cost of each configuration directly, with an ask that calls no tool at all — everything in that number is prompt you pay for before anyone says anything.

configuration	fixed tokens
mcp — all 7 `signalk` schemas	1,160
Bash tool, no guidance	3,860
Bash + `sk` cheatsheet	3,943
Bash + paths + unit table	4,198
production agent — 11 MCP servers + subagents + crew prompt	~19,800

Enabling one Bash tool costs about 2,700 tokens more than all seven MCP schemas combined. The shell is not free; it arrives with its own harness prompt describing how to use it safely, and that prompt is bigger than the schemas it was supposed to replace. Whatever a CLI saves by not being resident, it spends immediately on being allowed to run at all.

Then there is the last row.

The bill is the fleet, not the protocol. A flat agent with one MCP server carries 1,160 tokens. The production agent daemon carries ~19,800 — 17x — because it eagerly spawns eleven MCP servers at startup and every one of them registers its schemas into every turn, whatever the question was. Ask it what time it is and you have paid for the COLREGs tool descriptions.

That is an eager loading problem, not an MCP problem, and the same servers cost nothing in Claude Code, which loads them lazily. Which is why the answer to "our MCP bill is too high" is almost never "rewrite the servers as CLIs" — it is "stop loading all of them on every turn."

One honest wrinkle in that ~19,800: it is the median of four measurements (19,791 / 19,798 / 19,800 / 19,812). A fifth, taken cold, read 14,912 — on a cold start not every uv-spawned server has registered its schemas yet, so an early measurement understates the resident cost. If you go measure your own, measure it warm.

What the bench got wrong, twice

Two defects in my own harness, both of which had been quietly producing plausible numbers.

1. total_cost_usd is cumulative per session. The SDK's ResultMessage reports the cost of the session so far, not of the turn that just finished. Summing it across 12 asks sums a running total — a triangular over-count that inflates a 12-ask session by roughly 6x, and inflates it more for the arms with more turns. Every cost number in an earlier draft of this post was wrong in the direction that flattered my hypothesis.

obs.cost_usd = float(getattr(message, "total_cost_usd", 0.0) or 0.0)
...
total = sum(r.cost_usd for r in results)      # WRONG: sums a running total

2. The bench scored which tool got called, not what came back. This is the one that stings. Scoring compared observed_tools against expected_tools — the agent called battery_state, the corpus said it should call battery_state, green. What battery_state returned was never examined:

{"bank": "0", "state_of_charge_pct": null, "voltage_v": null,
 "current_a": null, "display": "No battery data available"}

The tool defaulted to bank="0", the SignalK convention. Our vessel publishes electrical.batteries.house. So the shipped tool had been answering "no battery data" on a boat that had been publishing battery data the whole time — for months — behind a green bench.

It survived that long because the corpus ask was "How's the **house* battery doing?"*. The word "house" made the model pass bank="house" explicitly, which worked. The test was handing the agent the answer to the question the tool was failing.

The fix

Difference the cumulative cost, and keep both views:

# ResultMessage.total_cost_usd is CUMULATIVE for the session, not the turn.
prev_cost = 0.0
for ask in asks:
    result = await _run_one(client, ask)
    result.cost_usd_session = result.cost_usd            # as reported
    result.cost_usd = max(result.cost_usd_session - prev_cost, 0.0)
    prev_cost = result.cost_usd_session

There is a free consistency check hiding in keeping both: the differenced per-turn costs must sum to the final cumulative figure, and a test asserts it.

For the scoring bug, the fix is to grade against the world instead of against the transcript. Every ask carries the path its answer lives at, and the runner reads that path off the server itself, next to the agent's turn:

{
  "id": "battery",
  "prompt": "How's the battery doing?",
  "expected_tools": ["mcp__signalk__battery_state"],
  "truth_path": "electrical.batteries.house.capacity.stateOfCharge",
  "truth_unit": "ratio->percent"
}

def _fetch_truth(ask: Ask) -> str:
    """Read the ask's answer straight off SignalK, next to the agent's turn.

    Deliberately a raw urllib GET, not signalk-mcp's client: the whole point is
    an independent reading the arms cannot influence.
    """
    url = f"{signalk_url()}/signalk/v1/api/vessels/self/{ask.truth_path.replace('.', '/')}"
    ...

Independence matters more than convenience here. If the truth had been fetched through signalk-mcp's own client, the all-nulls bug would have been graded against itself and stayed green a second time.

And the ask lost a word: "How's the battery doing?". The corpus should never contain the hint that papers over the defect.

Gotchas worth carrying away

A benchmark that scores tool identity is a routing test, not a correctness test. Both are worth having; only one of them notices that the tool returned nulls. If your evals assert "the model called the right function", assume you have a class of bug that is structurally invisible to them.
Fix the vessel, not the corpus. The golden asks had drifted from where the boat was. Editing the expected answers would have been quicker and would have silently broken comparability with every earlier run; moving the vessel back kept a year of results comparable.
Session totals, not per-ask means. Anything resident is billed per turn, so per-ask averaging systematically flatters whichever arm has the fattest resident prompt. Report the number that looks like an invoice.
Small models may still want shaped tools. This is Sonnet, n=12, one vessel. A separate n=10 probe against a local qwen3.6 suggests the picture changes below the frontier: raw value retrieval was correct in every run — it can fetch a number — but interpreting one, "is the battery charging or discharging?", scored 3/10 with thinking off (one run degenerated into ...) and 10/10 with thinking on low. Retrieval is not the hard part; the semantics on top of it are, and that is exactly where a tool returning "discharging" instead of -3.1 earns its keep. Re-measure before betting an on-vessel local model on these results.
The vessel is ashore. Phase 0: the SignalK server runs a mock vessel plugin. Token counts, response shapes and defects are real; the readings are synthetic.

Where it lands

I published the argument that MCP is overhead, then built the experiment to prove it, and the experiment said the opposite. MCP was the cheapest arm on both views in all three runs, and the honest version of the CLI case is much narrower than the one I was making: sk earns its place for SSH sessions, cron, headless claude -p and non-MCP agents — reach an MCP server cannot provide. It is just not a token optimization, which is what I had proposed it as.

The money was never in the protocol. It is in eleven servers loading eagerly, 19,800 tokens deep, on a turn that only needed to know the time.

Both halves are public: signalk-mcp is the server, and the four-arm harness lives in naturali-agents under poseidon/bench/ — part of the agent stack I'm building for an all-electric charter catamaran that doesn't exist yet.

Related: MCP server or a curl recipe in AGENTS.md? Measure the breakeven — part one, the static token count and the keel-offset sign that decided it; Benchmark your own LLM workload before migrating — the same harness pointed at models instead of tool surfaces; and Running OpenClaw on a Raspberry Pi alongside SignalK — where the curl recipe came from.

Drop the vendor station ID: licensing as a data-model constraint

Bryan Clark — Wed, 26 Aug 2026 16:44:44 +0000

TL;DR — @sailingnaturali/station-corrections 2.0.0 deleted providerId, the one field in a public MIT registry that pointed into a provider's database. Consumers now join records by station name and resolve the provider's opaque handle at runtime, under their own licence to that API. It cost a major version and reshaped four repos — and it's a stronger control than a compliance note in a README, because the id can't leak from a field that doesn't exist. Jump to the fix.

"Don't redistribute the vendor's identifiers" is normally a compliance checkbox — a line in a NOTICE file, a paragraph in a licence audit, nothing that touches code. This one turned into an architecture. Here's what it actually cost, and what it bought.

Problem

station-corrections is a small public npm package that publishes station identity for tide and current stations in the Salish Sea: the friendly name, the corrected position, search aliases, and a stable key that URLs and caches key on. Every record looked like this:

chs-dodd-narrows:
  name: Dodd Narrows
  context: Nanaimo
  position: [49.1344, -123.8171]
  provider: chs
  providerId: 63aef1866a2b9417c035030f

The package ships a PROVENANCE.md that walks the schema field by field and says where each value came from, because the whole claim of the package is these are our own facts, not a copy of anyone's station export. Names are hand-written. Contexts and aliases don't exist in provider data at all. Positions come out of a harmonic-fitting pipeline and get audited against a bundled coastline by a person.

And then there was one row that read differently:

| `providerId`, `providerBin` | **The provider's own opaque handle**, necessarily obtained from the
  provider because it keys the provider's API (`63aef09f…` for CHS, `PUG1717` for NOAA). It is a
  reference/pointer — the minimum needed to be interoperable — and a fact. This is the one field
  that points *into* a provider's system, by construction. |

The contributor guide made the exception explicit:

- The `providerId` is the exception — it is the provider's handle and there is nowhere else
  to get it. That is expected and fine.

The Canadian data behind those hex strings comes from the Canadian Hydrographic Service's IWLS API, whose licence agreement is a bespoke Crown licence, not the Open Government Licence. Clause 3 is one sentence long:

The copyrights of CHS in the Data remain the property of CHS and shall not be
sold, licensed, leased, assigned or given to a third party.

An MIT npm package is, structurally, giving something to third parties. Nineteen CHS station handles were sitting in it, then re-published in a SignalK plugin's defaults.ts, then re-published again in a Python MCP server's vendored copy of the registry — fifty-seven committed copies of the same nineteen handles, across three packages anyone can npm install or pip install.

Diagnosis

Two things were true at once, and only noticing both makes the fix obvious.

One: an id is the weakest thing you can defend as "just a fact." The provenance doc leans on Feist (US) and CCH (Canada) — facts aren't copyrightable, and neither jurisdiction has an EU-style database right. That argument is strong for a position (independently derivable: chart, gazetteer, direct observation) and strong for a name (hand-written editorial work). It's weakest exactly where the contributor guide admitted the weakness: there is nowhere else to get it. A field whose only possible provenance is the vendor's database is the field where "we obtained this independently" stops being available as a defence. The sentence written to justify the exception is a precise statement of why the exception is the problem.

Two: the field was doing two jobs. It was the identity of a station — what caches, resource keys and cross-package joins keyed on — and it was the fetch handle the provider's API wants in a URL path. Those are different lifetimes. Identity should be stable forever and safe in a URL. A fetch handle is whatever the vendor's index says today. Conflating them is what made the id feel load-bearing and undeletable.

Separate the two jobs and the licence problem dissolves: the identity half is our own fact and publishable, the fetch-handle half never needs to be written down.

What we tried (and why it failed)

Attempt 1: document the exception harder. The first response was the provenance table above — a careful, honest, field-by-field account ending in "carried because interoperability requires them." It's good documentation and it moves zero bytes. The published tarball is identical before and after you write it. A README does not change what a package distributes, and clause 3 is about what you distribute.

Attempt 2: just delete the field. This is where it stops being a paperwork exercise. The plugin's entire online path is keyed on that id:

const defaultFetcher: DayFetcher = async (s, a, b) =>
  s.provider === 'chs'
    ? { events: await fetchChsEvents(s.stationId, a, b) }
    : fetchNoaaEvents(s.stationId, s.noaaBin ?? 0, a, b);

Delete providerId and there is no stationId to fetch with — you don't get a compile error in the registry package, you get a plugin that can no longer answer "when is slack at Dodd Narrows?" at all, online or off. The design spec had to spell out the three separate things a gate needs at runtime before anything could be removed:

Need                      | Old source (deleted)   | New source
--------------------------|------------------------|----------------------------------------
name / position / label   | defaults.ts            | station-corrections (by name/key)
live CHS id (online fetch)| defaults.ts stationId  | resolved live from IWLS /stations by name
offline constituents      | did not exist          | operator-triggered local build

The sibling fitting library had the same coupling, expressed as a guard that required the id to be present:

if (!key?.trim() || !entry.name?.trim() || !entry.providerId?.trim()) {
  throw new Error(`registry entry ${JSON.stringify(key)} has an empty key, name, or providerId`);
}
return { id: entry.providerId, label: entry.name, key };

Attempt 3: keep the id, but only where it's public domain. NOAA data is US-government public domain — PUG1717 is genuinely free to redistribute. So carve out CHS and keep the field for NOAA rows. This was rejected, and the reason is worth stating: a schema field whose legality depends on which row you're looking at is a field every future contributor has to reason about correctly, forever, in a package that takes drive-by PRs from anyone who spots a wrong station name. PUG1717 and its depth-cell bin came out too. The rule in the shared package is uniform — no provider-minted identifier, ever — and the one consumer that genuinely may ship a NOAA id keeps it in its own package, where the provider is fixed and the reasoning is local.

Attempt 4: assume the two provider endpoints share an id namespace. The plugin lists stations from api-iwls.dfo-mpo.gc.ca but fetches events from api-sine.dfo-mpo.gc.ca. If those two mint different ids, "resolve from IWLS, fetch from SINE" returns an empty array for every gate — no error, no 404, just currents quietly gone. That is not a thing to find out after wiring four repos together, so it got a throwaway probe first:

const stations = await (await fetch(`${IWLS}/stations`)).json();
const dodd = stations.find((s) => s.officialName === "Dodd Narrows");
const resp = await fetch(`${SINE}/stations/${dodd.id}/data?${params}`);
console.log(rows.length > 0 ? "PASS: shared id namespace" : "FAIL: ids differ — move event fetch to IWLS");

Twenty-three lines, one request, PASS. Had it failed, the live event fetch would have moved to IWLS as well — a different plan entirely, and much cheaper to discover before the refactor than during it.

The fix

1. The registry ships identity and nothing else. Two feat! commits and a major version:

chs-dodd-narrows:
  name: Dodd Narrows
  context: Nanaimo
  position: [49.1344, -123.8171]
  provider: chs

provider stays — "this is a CHS gate" is a fact about the world and tells a consumer which API to go ask. The validator drops providerId from its required-fields list and deletes the providerBin range check entirely. PROVENANCE.md's exception row became a prohibition:

| provider id | **Deliberately absent.** The registry carries no provider-minted identifier at
  all — not even as a reference. A consumer joins a record here to a provider's live data by
  name; the provider's own opaque handle is resolved at runtime by whoever holds a licence to
  that provider's API, and it never enters this repository. |

2. Consumers split identity from the fetch handle. In the SignalK plugin, stationId becomes the stable registry key and a new liveId carries the ephemeral one:

// Ephemeral IWLS station id for a CHS gate, resolved live at runtime (see
// resolveLiveIds). NEVER committed. `stationId` above is the stable identity
// (registry key for CHS); `liveId` is only the fetch handle.
liveId?: string;

const chsLiveId = (s: StationConfig): string => {
  if (!s.liveId) throw new Error(`no live id for ${s.label}`);
  return s.liveId;
};

That throw is the enforcement. There is no path where a CHS fetch silently falls back to something committed, because there is nothing committed to fall back to.

3. Resolve the handle at runtime, from the provider, by name. The whole resolver is twenty-odd lines:

export function currentStations(raw: RawStation[]): IwlsStation[] {
  return raw
    .filter((s) => (s.timeSeries ?? []).some((t) => t.code === 'wcsp1'))
    .map(({ id, officialName, latitude, longitude }) => ({ id, officialName, latitude, longitude }));
}

export async function resolveLiveIds(fetchFn: typeof fetch = fetch): Promise<Map<string, string>> {
  const resp = await fetchFn(`${IWLS_BASE}/stations`);
  if (!resp.ok) throw new Error(`IWLS ${resp.status}`);
  const stations = currentStations((await resp.json()) as RawStation[]);
  return new Map(stations.map((s) => [normalizeName(s.officialName), s.id]));
}

It runs at start(), holds the map in memory for the life of the process, and tolerates failure by falling through to the offline path. The id never reaches a file.

4. One normalizer, applied symmetrically, in every language that joins. TypeScript:

// Same folding rule chs-constituents uses, so "JUAN DE FUCA - EAST" matches the
// registry's "Juan de Fuca - East".
export function normalizeName(name: string): string {
  return name.toLowerCase().replace(/[^a-z0-9]+/g, ' ').trim();
}

Python, in the MCP server that reads the plugin's /currents resource and joins each reading back to a registry gate:

def _norm(name: str) -> str:
    """Join key for correlating a gate to a plugin reading: fold case and trim
    so a label/name that differs only in casing or spacing still matches."""
    return name.strip().casefold()

Applied on both sides — once when the cache is built, once on lookup — so a whitespace difference between the plugin's label and the registry's name can't silently drop a station.

5. Tests that keep it true rather than a rule that asks nicely. The fitting library's overlay test asserts the absence directly:

it("keys entries by normalized name and reads no id at all", () => {
  // No providerId field anywhere — proves the overlay is forward-compatible
  // with the registry dropping providerId in Phase 2.
  const overlay = registryOverlay(
    { "chs-dodd-narrows": { name: "Dodd Narrows", provider: "chs" } },
    "chs",
  );
  expect(overlay.get("dodd narrows")).toEqual({ key: "chs-dodd-narrows", label: "Dodd Narrows" });
});

and the MCP server's vendored copy of the registry is guarded by a drift test that fails the moment it stops byte-matching the published one.

Why it matters, and the gotchas

A licence term makes a better schema constraint than a policy. Compare "reviewers should check that nobody adds a provider id" against "the schema has no such field and the validator rejects unknown ones." The second one survives a contributor who never read PROVENANCE.md, which is most of them.

Name joins are fragile exactly where ids are robust — so put the renames somewhere. An opaque id survives a rename; a name join doesn't. The mitigation is that the registry is the thing that owns renaming: change a station's display name there and every downstream package follows, because they all read it. What the join can't survive is the provider renaming its station, and that is a real, unhandled failure mode — the gate goes offline-only until someone updates the registry name. The fitting library degrades gracefully (an unmatched live station keeps its official name and gets no key); the plugin logs and falls back. Neither pretends it matched.

Shipping the change broke the boat, twice, and both were migration bugs — not the design.

Nineteen duplicated gates. An operator config written before 2.0.0 still names a gate by its provider-minted UUID. That never collides with the registry's slug, so both entries survived the merge, and the stale one resolved no live id either:

// Re-key such an entry onto the registry slug so it lands as the OVERRIDE it was
// always meant to be: the slug is what resolveLiveIds works from, and every other
// field the operator set (set directions, estimate flags, bins) rides along.
// Deliberately keyed on label, not position: renaming a station is the registry's
// job and downstream is supposed to follow it.

It was invisible for weeks because downstream consumers key on the label and silently keep whichever copy they saw last. When you re-key a shared identifier, write the migration for configs you don't control — someone is running the old shape.

Ten stations throwing every poll cycle. The registry later grew tide reference ports alongside current gates. Tide ports publish no wcsp1 series, so resolveLiveIds finds nothing for them and the fetch guard fired ten times a cycle:

no live id for Point Atkinson
no live id for Campbell River
…

The fix wasn't a filter in the plugin — that filter existed and was already wrong. Selection moved into the package that owns the data, as an exported currentGates(). If a package can grow a new row type, only that package can be trusted to say which rows are which.

Scrub the whole repo, not just src/. After the code was clean, a CHS id was still sitting in a fenced code sample inside an old implementation-plan doc, where no grep of the source would find it:

-  { provider: 'chs', stationId: '63aef1866a2b9417c035030f', label: 'Dodd Narrows', … },
+  { provider: 'chs', stationId: '<24-hex-id>', label: 'Dodd Narrows', … },

Docs, specs, plans, test fixtures, issue templates, commit messages. Grep for the shape of the thing, not the name of the field.

Make the invariant structural where you can. The offline harmonic bundle a user builds locally is written to the server's data directory — outside the repo and outside the npm package — so it cannot be committed or published by accident. That's not discipline, it's geography, and geography doesn't get tired.

The reward is the part nobody asks for. Because nothing downstream stores a provider handle any more, renaming a station is now a one-line PR in one repo, and the version of the package that made this legally safe is the same version that made renames free. That was not the goal; it fell out of deleting the field that pointed sideways into someone else's database.

Built while putting a boat-agent stack together for an all-electric charter catamaran, where "the Canadian rapids stop working offline" is a licensing problem before it is a software one. The registry is station-corrections; the plugin is signalk-currents; the fitting pipeline that has to be run rather than downloaded is chs-constituents.

Related: Offline tidal currents from harmonic constituents — the same licence wall, one layer down, where it decides which predictions you can bundle at all. And the NOAA CO-OPS currents API gotchas for the other half of this cruising ground, where none of this applies.

The A/B that measured nothing: three ways my agent experiment was invalid

Bryan Clark — Tue, 25 Aug 2026 20:16:09 +0000

I set out to test whether an MCP tool should return a rendered sentence or raw SI numbers. The experiment was invalid three times, for three unrelated reasons: the tools had never been called (a gateway tool-policy gap), the two arms were byte-identical at the model (the gateway drops a tool's text content when structuredContent is present), and once it genuinely measured, the new shape lost. What actually fixed the behaviour was deleting a stale paragraph in the workspace instructions that told the model to do the opposite. Jump to the fix.

Two months ago I published a post here with a confident thesis:

Move the formatting into the tool response. […] a prompt rule isn't a fix, it's a suggestion the model is free to ignore the moment the raw value is still in front of it. You can't reliably instruct the model not to reformat data it can plainly read. The only reliable lever is the data itself.

— Fix LLM formatting in the tool layer, not the prompt, 2026-06-05

I still think that's mostly right. But when I finally tried to measure it — on a second agent runtime, with a different model — the experiment fell over three times before it produced a single valid number, and when it did produce one, the number pointed the other way. This is the write-up of the three invalidities, because each one is a trap you can hit on your own stack, and the second one is a genuine gotcha for anyone writing MCP servers.

Problem

The stack: a set of MCP servers over a boat's SignalK data, driving a voice-and-chat agent. Following the June post, every tool returns a pre-rendered display string next to the raw SI fields:

{
  "bank": "house",
  "soc_fraction": 0.65,
  "voltage": 12.39,
  "current": -2.95,
  "display": "65 percent, 12.4 volts, 3.0 amps discharging",
  "timestamp": "2026-08-10T04:00:00Z"
}

The agent's persona file carries a verbatim carve-out — quote the display string, don't re-render from the raw fields. Asked "depth and house battery?", the agent answered:

65% SOC, 12.39 V, -2.95 A

Raw SI, units abbreviated the way a datasheet does and a person doesn't, and a negative amperage — which the persona explicitly forbids, because "minus two point nine five amps" is not how you tell someone the battery is discharging. One run even helpfully explained the minus sign.

So: display exists, the rule exists, and the model ignores both. Reproduced on two models — a small fast open-weights one and a large cloud one. Asked to quote the rules back, both reproduced them verbatim, then broke them in the next sentence.

Diagnosis

My hypothesis, straight out of the June post: display is losing an attention contest. It sits below the raw fields in the JSON dump, so a model reading top-down meets soc_fraction first, decides it knows what a 0–1 fraction is, and renders from that. Fix the shape — put the sentence first, or in its own content block — and the failure mode disappears.

That hypothesis was testable. What I didn't check was whether anything in the chain between the tool and the model would let me test it. Three things wouldn't:

What I tried (and why it failed)

Attempt 1 — enforce it harder in the prompt

The first move was the one the June post says not to make, on the theory that this particular rule is about quoting a string I already hold, which that post explicitly carves out as a legitimate prompt instruction. So I ported the carve-out into the gateway's persona file:

When a tool returns a `display` string, quote it verbatim.
Never re-render the reading from the raw SI fields beside it.

Deployed, loaded, in context — the agent could recite it. It then ignored it, on both models, across two wordings.

Conclusion at the time: prompt-level enforcement of this carve-out does not hold, therefore the structural fix in the tool layer is required. That conclusion was worthless. Hold that thought.

Attempt 2 — return the sentence as `content`, the numbers as `structuredContent`

MCP lets a tool return both an unstructured content array and a structured structuredContent object. That looked perfect: make the text content be the answer, and let the numbers ride in the structured half where threshold reasoning can still reach them deliberately.

In the Python MCP SDK that's a tuple return:

@server.call_tool()
async def _call_tool(
    name: str, args: dict | None
) -> list[types.TextContent] | tuple[list[types.TextContent], dict]:
    ...
    # PILOT, depth_state only. `display` sits BELOW the raw SI fields in the
    # JSON dump, so a model reading top-down meets `below_keel_m` first and
    # re-renders from it. Make the rendered sentence the answer instead, and
    # put the numbers in structuredContent — still there for threshold
    # reasoning, but one deliberate reach away.
    if name == "depth_state" and isinstance(result, dict) and result.get("display"):
        return [types.TextContent(type="text", text=result["display"])], result

    return [types.TextContent(type="text", text=json.dumps(result, indent=2))]

Deliberately one tool only. The other six kept the old shape, so a single "depth and battery" ask A/B'd both shapes against each other in one reply, on one model, in one turn. Clean design. Five trials per arm. The new shape looked better.

It was measuring nothing. The gateway renders structuredContent and discards the tool's text content blocks. The sentence never reached the model at all; what arrived was the same JSON object the old shape sent, so both arms were byte-identical from the model's side.

This one is worth naming precisely, because it will bite other MCP authors. The gateway here is OpenClaw, and as of v2026.7.1-2 the behaviour is in src/agents/agent-bundle-mcp-materialize.ts:

  const structuredContentBlock =
    params.result.structuredContent !== undefined
      ? ({
          type: "text",
          text: `structuredContent:\n${JSON.stringify(params.result.structuredContent, null, 2)}`,
        } as const)
      : null;
  // Structured results replace mirrored text, but original non-text blocks
  // still carry images, linked resources, and audio that the JSON cannot mirror.
  const normalizedContent: AgentToolResult<unknown>["content"] = structuredContentBlock
    ? [
        structuredContentBlock,
        ...sourceContent
          .filter((block) => block.type !== "text")
          .map(mcpContentBlockToAgentContent),
      ]
    : content.length > 0
      ? content
      : ...

filter((block) => block.type !== "text") is the whole story: when structuredContent is present, every text block is dropped. Images, resource links and audio survive — that carve-out was added in July 2026. Text does not.

Two caveats before you copy this into your own mental model:

Version-pin it. This code path has flipped twice in ninety days. Before #87540 (merged 2026-05-28) the precedence was the other way round — content won and structuredContent was dropped, which was filed as a bug. Don't assume either behaviour without checking the version you're actually running.
It's a defensible reading of the spec, which is what makes it dangerous. The MCP specification says:

For backwards compatibility, a tool that returns structured content SHOULD also return the serialized JSON in a TextContent block.

— MCP specification 2025-06-18, Tools

The spec's own example returns content: [{"type": "text", "text": "{\"temperature\": 22.5, …}"}] beside structuredContent: {"temperature": 22.5, …} — the same data twice. So a client that has parsed structuredContent is entitled to treat the text block as a redundant mirror and drop it. My pilot did the one thing the spec doesn't anticipate: it put different information in the two halves. If your content is not a serialization of your structuredContent, you are outside what the spec promises, and at least one popular client will silently throw it away.

Attempt 3 — sentence first, numbers after, in one text block

Ordering inside a single text block is the only lever that survives every client, so that's what the pilot became:

    if name == "depth_state" and isinstance(result, dict) and result.get("display"):
        rest = {k: v for k, v in result.items() if k != "display"}
        text = f"{result['display']}\n\n{json.dumps(rest, indent=2)}"
        return [types.TextContent(type="text", text=text)]

Which produces a payload that is unambiguously different from the control:

3.7 metres under the keel

{
  "below_keel_m": 3.7,
  "depth_m": 5.2,
  ...
}

Except the numbers still didn't move, because of the third invalidity — the one that had also voided attempt 1.

The invalidity underneath the other two: the tools had never been called

Seven MCP tools, all probing healthy, none of them ever invoked. The gateway's tool policy admitted a base profile plus an explicit extra list — and the extra list didn't include the group that MCP server tools live in:

{
  "tools": {
    "profile": "minimal",
    "alsoAllow": ["group:runtime", "group:fs"]
  }
}

In OpenClaw, tools.profile "sets a base allowlist before tools.allow/tools.deny", and minimal allows session_status only. Configured MCP servers are "exposed as plugin-owned tools under the bundle-mcp plugin id", and the docs note that the broader profiles — "coding and messaging also implicitly allow bundle-mcp" — do that implicitly. minimal does not. The docs even describe the exact symptom, under sandboxing:

If mcp.servers is configured but sandboxed turns only show built-in tools, add bundle-mcp, group:plugins, or a server-prefixed MCP tool name/glob […] to tools.sandbox.tools.alsoAllow

The failure mode is nasty because nothing errors. The server connects, tools/list succeeds, every health probe is green, and the tools are filtered out before the request ever reaches the model. From the outside it looks identical to a working install. And the agent still answered questions about the boat perfectly well — because the workspace instructions predated the MCP servers and told it to shell out to curl against the HTTP API, which it dutifully did.

So no display string had ever reached a model. Attempt 1 measured a rule against a payload that didn't exist. Attempt 2's arms were identical for a second, independent reason on top of the structuredContent one.

The one-line fix is adding the group to the allow list:

{
  "tools": {
    "profile": "minimal",
    "alsoAllow": ["group:runtime", "group:fs", "bundle-mcp"]
  }
}

Worth noting on its own: the agent switched from curl to signalk__depth_state / signalk__battery_state immediately, with no change to the instructions telling it to use curl. Tool availability beat documented instruction. The prose was never what held the MCP path back.

And then the honest measurement said I was wrong

With the arm genuinely connected, one question exercising both shapes at once, five trials per arm, on the small fast model:

arm	quoted `display` verbatim
`battery_state` — plain JSON dump, `display` buried mid-object	3 / 5
`depth_state` — rendered sentence first, numbers after	1 / 5

Leading with the sentence didn't help. If anything it hurt. Small n, one model, so I won't claim the effect is real in the other direction — but there is no evidence for the hypothesis, and the pilot was carrying a special case in a dispatch function, so it went:

$ git revert --no-edit 4c43abd
$ git log --oneline -1
89d3b40 Revert the depth_state output-shape pilot: measured, did not help

The fix

The lever was neither the prompt nor the payload. It was a contradiction between them.

The workspace instructions file — written when the agent read the boat over curl, and never revisited after the MCP servers went in — contained this:

Units are SI. Convert for the user:
  m/s → kn   × 1.94384
  rad → deg  × 57.2958
  K   → °C   − 273.15

That is an explicit, specific, actionable order to re-render raw numbers — which is exactly what the persona's carve-out forbids. The model wasn't being non-compliant. It was obeying the more specific of two conflicting instructions. Rewriting that section to point at the tools instead:

Read the vessel through the `signalk__*` tools, not `curl`.
Each returns a `display` string that is already rendered for a
human. Quote it. Do not re-render the reading from the raw SI
fields beside it.

Three trials, both tools: 6 / 6 verbatim. No change to the tool layer at all.

Why it matters / gotchas

The generalizable lesson is the failure mode, not the fix: when an agent ignores a rule, check what else in its context tells it to do the opposite before concluding the model is non-compliant. An agent's context is assembled from a persona file, a workspace file, tool descriptions, and tool results, all written at different times by people optimizing for different things. "The model won't follow instructions" is very often "the model is following the other instruction."

That complicates the June post rather than overturning it. Formatting still belongs in the tool response — that's what makes display deterministic and available. But the tool layer is not sufficient on its own, because it competes with a prompt surface that can contradict it, and it is not causally the lever when a contradiction exists. Fix the contradiction first, then optimize the payload — and check that the payload is actually arriving.

The traps nearby, in the order they'll bite you:

A healthy tool is not a called tool. Green probes, successful tools/list, and a working agent are all compatible with the model never having seen your tools. Before benchmarking anything about tool output, prove a tool was invoked — read the session transcript for the call, don't infer it from a plausible answer. (Ours answered correctly the whole time, via curl.)
Don't put novel information in content when you also send structuredContent. The spec frames content as a backwards-compatibility mirror, and clients act accordingly. If you need the model to read a rendered string, put it in the one payload every client forwards.
A/B'ing two arms of a payload requires proving the arms differ at the model, not at the server. Log or dump what the client actually forwarded. Mine was a byte-for-byte match across a difference I'd written 40 lines of code to create.
Every "the model won't obey" conclusion is provisional until the plumbing is proven. I wrote a struck-through wrong conclusion into my own docs, twice, on the strength of experiments that hadn't measured anything.

Close

This is one thread out of building an AI ops layer for an all-electric charter catamaran — MCP servers over SignalK, a self-hosted agent gateway on a Raspberry Pi, and a voice front-end where "what's the battery doing" has to come back as something a human can hear. The server the code above is from is open source: signalk-mcp, including the revert.

DEV Community: Bryan Clark

A nearest-neighbour matcher turned a timing lag into a fake direction bug

The problem: a label-flip failure that wasn't

Diagnosis: the matcher couples two independent errors

What we tried (and why it failed)

The fix

Why it matters: same-kind matching is the safer test, not the laxer one

The transferable version

A config warning on stdout made a healthy alarm lane look dead

The problem

Diagnosis

What I tried, and why it failed

The fix

Why it matters, and the gotchas around it

MapLibre's SDF icon edge is at 0.75, not 0.5

The problem: a correct icon that draws zero pixels

Diagnosis: read the shader

Why 0.75 is not an arbitrary number

What we tried (and why it failed)

The test that asserted the bug

"Just tighten the spread until something appears"

"Add a constant so the edge reads 192"

The fix

Why it matters, and the traps next door

claude setup-token fixed my launchd auth and dropped my connectors

The problem: launchd, a keychain, and a fix that worked

Diagnosis: the run wasn't failing, it was half-blind

Why: connectors ride on the account session, and the token outranks it

The part that actually cost four days

What I tried, and why each one failed

The fix

Why it matters, and the traps next door

Your service worker's precache can't do Range requests — PMTiles needs 206

The problem: the map draws once, then never again

Diagnosis: precache is 200-shaped, and it only joins the path on visit two

What we tried, and why it failed

Attempt 1 — precache it and raise the size cap

Attempt 2 — a smoke test that agreed with us

Attempt 3 — runtimeCaching with rangeRequests, and an empty cache

The fix

Gotchas

serviceWorker.ready does not mean the service worker controls your page

Test the controlled path, because that's the one users are on

The nice side effect

A passing check is a claim about what ran, not what's true

1. A boolean that returns false for "no" and for "I can't tell"

2. A validator that iterates one side of a pair

3. A benchmark that scored the call, not the result

4. A selftest that greps for a string the command never prints

5. An assertion that fired before the system finished failing

6. When deleting the fix leaves every check green

What they have in common

The checklist

A current station's coordinate is a label, not the hydraulic control

Problem

Diagnosis

What we tried (and why it failed)

Blamed the section spacing

Moved the anchor to the DEM throat, expecting that to be the fix

Made the area tide-stage dependent

Tried the other bathymetry source

The fix

Why it matters, and the traps next door

Close

A localeCompare in a build script minted different public URLs per machine

Problem

Diagnosis

What I tried (and why it failed)

Pin the locale

"Just use a stable sort"

Commit the generated file

The unit test that was already green

The fix

Why it matters / gotchas

plugins.allow is exclusive: the warning's suggested fix disabled 44 plugins

The problem: a warning with a fix attached

What it actually did

Diagnosis: "allow" is exclusive over everything

The asymmetry that makes it survivable — and confusing

What the two casualties actually cost

`serviceWorker.ready` does not mean the service worker controls your page

Attempt 2 — return the sentence as `content`, the numbers as `structuredContent`