What a few days of playtest telemetry found that my own testing never did

Chris Longden — Sat, 26 Sep 2026 19:10:23 +0000

I'm building Farshore, a first-person mystery that runs in the browser: three.js, no engine, every sound synthesised live in Web Audio. In its first days of open playtest, nine people found problems I never had, for one reason: I always know where to go.

The logger

With ?playtest=1, the game logs story beats (changes to its own story flags), what players examine and use, hints, fall rescues, "unstick" presses, position every 10 s and frame rate every 30 s. Nothing personal. Every event is stamped with active time (seconds actually playing), so a tea break doesn't look like being stuck. A small script turns the logs into stalls: three active minutes with no progress, plus what the player tried in the meantime.

Four lessons

1. Read state the player can change live. A chapter copied a story flag into a constant when it loaded. The player did the right thing, but the hints kept sending them back to do it again.

const answeredAtBuild = flags.subAdultWakened; // starting visuals only
const isAnswered = () => flags.subAdultWakened; // everything else

2. "Safe" has to mean dry ground. An invisible floor under the sea counted as a safe spot, so pressing unstick in the water said "You climb back to solid stone" and left the player where they were. Water floors are now flagged, and never saved as safe.

3. Edges over water are walls, not drops. One player got six fall rescues in three seconds: the net put them back on the edge they'd fallen from, and the key they were holding walked them off again.

4. Coming back must say what changed. The first player to finish wrote: "I thought I came back to the same level to do the same thing because I haven't finished something." Returns now say what's moved on.

Blind AI playtesters

I also ran AI players that see only screenshots and on-screen text. They're bad at navigation (all three stalled on a key every human found in 35 seconds), but they walk into every edge. Treat them as a pointer to where to look, not a verdict.

The takeaway

The hard part of playtesting isn't finding players; it's knowing what happened when you weren't watching. A few hundred lines of logging turned "it was confusing" into "they stood here, tried this four times, and left". Most fixes shipped the same day.

Farshore plays in the browser. If you get stuck, that's exactly what the logs are for.

DEV Community: Chris Longden

What a few days of playtest telemetry found that my own testing never did

The logger

Four lessons

Blind AI playtesters

The takeaway