DEV Community

Todd Sullivan
Todd Sullivan

Posted on

My AI Model Was 89% Accurate. It Missed 99% of the State I Cared About

#ai #machinelearning #coreml #gamedev

I trained a small CoreML classifier to estimate whether a player was bored, in flow, frustrated, or about to quit.

The training pipeline looked respectable:

  • headless game simulator
  • bots with randomised aim, dodging, recklessness and patience
  • deterministic seeds
  • player-level train/test split
  • class-weighted loss
  • 132,139 labelled rows across 2,400 simulated players

The test accuracy was 89.4%.

Then I looked at the classes instead of the headline number:

bored          recall 95.5%
flow           recall 94.8%
frustrated     recall 98.2%
aboutToQuit    recall  1.1%
Enter fullscreen mode Exit fullscreen mode

The model was almost completely missing the state I cared about most.

There was another problem hiding underneath it. While watching the simulator, I found bots freezing in cleared rooms. The navigation was fine. The bot's stuck recovery was not.

Every 30 frames it checked whether it had moved. If not, it sidestepped for 40 frames:

if clock % 30 == 0, distance(pos, lastPos) < 0.5 {
    detour = 40
}
Enter fullscreen mode Exit fullscreen mode

A sidestep into a wall was still active at the next check, so it could trigger another sidestep and loop forever. I had generated plausible-looking training rows from an agent that was sometimes just stuck.

The fix was tiny: make the detour shorter than the check interval and alternate sides.

if clock % 30 == 0, distance(pos, lastPos) < 0.5 {
    detour = 20
    detourSide = -detourSide
}
Enter fullscreen mode Exit fullscreen mode

I also added a simulator invariant: if a bot stays in a cleared room for more than 60 seconds, count it as a stall and fail the data run. The original issue appeared in 75 of 450 sessions. After the fix, the detector found zero stalls across 2,400 simulated players.

Clean data did not magically fix the rare class. Overall accuracy stayed at 89.4%, while aboutToQuit recall actually fell from 15.4% to 1.1%.

That was the useful result.

The simulator bug and the label problem were separate failures. Fixing corrupt behaviour made the dataset honest; it did not make the label learnable. The rare state still overlaps heavily with frustration and needs better labels or real player data, not another round of metric tuning.

My rule now is:

  • simulators need behavioural invariants, not just successful exits
  • splits must be by actor, not by row
  • every important class gets its own metric
  • low-confidence model output falls back to deterministic rules

An AI pipeline can be reproducible, testable and 89% accurate while still failing its actual job.

Top comments (0)