Stress Test? Pen Test? Yes Tests!

Why Your Game's Security Tests Need to Be Adversarial—Not Just Automated

Published on DHSeaDev. A hard-won lesson from shipping a browser-based TCG.

When we stress-tested Prismwar, our original gate suite reported everything was fine. Then we ran a mutation campaign, and 31 out of 70 carefully crafted bugs survived without raising a flag. Here's what went wrong—and how we fixed it.

The Test That Tested Itself

The first blind spot was the most embarrassing: our test harness shared defects with the code it was supposed to verify.

We had a replay system that recorded matches as a seed, both decks, and the player's actions. The Rival's moves are derived at playback time by running the same AI code. When we tested replay fidelity, we recorded and played back ten matches—and all ten matched perfectly.

Sounds good, right? Wrong.

The issue: our harness omitted the AI's defence step in the replay player, and our scripted human player omitted it in the recorder. So the test round-tripped through the same bug and agreed with itself. A round trip cannot test its own driver. Neither can it test its engine. We rebuilt the test to drive the AI's defence window directly and assert the match phase advances—that found it immediately.

The lesson applies everywhere: when testing a system that has two paths (encode/decode, server/client, record/replay), don't let both paths share the same code on the test side. Property assertions work. Round trips don't.

44% of Bugs Stayed Silent

The mutation campaign replaced 70 isolated lines of code with deliberate defects. 39 made the game unplayable. But 31 made the game fail silently:

• Dropped guards: validateDeck checks deck legality before a match starts. We deleted the lower-bound check. The gate passed. Invalid 4-card decks loaded fine and then threw an uncaught error in checkWin as soon as someone attacked. The error boundary swallowed it into a "Something went wrong" message, and the assertion "no console errors" stayed green.

• Discarded state: An attacker declared blockers but the code forgot to store which ones. The match proceeded as if nothing was blocked. Replays still worked and reproduced correctly (for the wrong reason—the entire engine had the same bug).

• Unsigned profiles: The integrity check exempted unsigned saves to allow mid-session work before signing. An attacker simply wrote a JSON with no sig field, granting 720 cards and a million wins. It imported as integrity:'ok'.

Every one of these would have made it into production without sabotage-driven proof.

What Actually Worked

We stopped relying on "no crash" and "match outcomes are deterministic" as evidence. Instead:

Entry guards at the choke point. The game's state machine enters through newGame(). We made that the single place that refuses invalid inputs: non-array decks, decks under 10 cards, missing players. Everything downstream assumes the precondition is true. One sabotage per boundary, and each one must actually throw (not silently default). We verified that the guard itself catches that attack—not just that the game doesn't crash.

Property assertions instead of outcome comparison. Instead of "does the game end in 50 turns or fewer," we assert that the stall detector measures progress—and if you remove that measurement, the test fails. The property is "this mechanism exists and is consulted," not "bad state doesn't happen."

Pinned baselines for non-observable code paths. The AI engine has zero randomness once seeded. We computed a SHA-256 fingerprint over the outcomes of a 512-game corpus. Any change to the rules, the AI decision weights, or the game logic moves that hash. We assert it doesn't move. If it does, we decide to bump it—or revert. But "no change" is measured, not assumed.

Instrument-proved test instrumentality. Before we ran our 70 mutations, we proved the harness could see a break by installing an attacker key and verifying that it unlocked a legendary card it shouldn't. When we replaced the stall detector with a board-count check, we proved the harness caught it by deliberately deleting the right code and watching the test fail.

The Adversarial Mindset

Security tests are not about passing. They're about finding specific ways the code could fail and then proving you catch every one:

1. Write a defect (a mutation, a forgery, a code path omission).
2. Run the test to prove it catches the defect. If it doesn't, the defect is a gap.
3. Verify that removing the defence that catches it makes the test fail. If removing the defence doesn't move the needle, you have a false positive.
4. Keep the defect as a proof that the test was actually running.

This is inverse from "does my happy path work." You're asking: "What specifically am I refusing?"

For Prismwar:

• ECDSA: we proved the harness detects payload tampering, signature forgery, and algorithm confusion. Nine attacks. All caught.
• XSS: 350 real-UI trials on a stripped CSP, zero escapes (our HTML builder uses textContent, not string concatenation).
• Determinism: identical results across three runs when entropy is disabled.
• Persistence: crossing two concurrent save cycles doesn't lose data (a known limitation, documented as unreachable in the shipped UI).
• State legality: illegal actions are refused with a clean Error, not a crash or silent skip.

What We Shipped

Final count: 122 assertions, 99 sabotages, 0 escapes, control green. The gates ran against a real build packaged for the Chrome Web Store. Not a best-case scenario. The actual game.

Three automated checks broke themselves in the process:

• A brace-balance one-liner couldn't see regex literals or comments. Fixed with a real lexer.
• An AWK scan counted lines across multiple files and reported false positives. Replaced with scoped iteration.
• A "no console errors" assertion didn't discriminate real errors from false red runs. Now it routes the specific assertion to the relevant scope.

Ship It Adversarially

When you're shipping something that handles user data—saves, settings, payment—or something competitive where cheating matters, your test suite needs to be adversarial:

• Can you add security properties that would break if someone removed the defense? If not, you don't have evidence the defense is real.
• Do your round-trip tests share machinery with the code they're testing? If so, rebuild one path to go a different way.
• Is your baseline pinned? A green build on "determinism" without a fingerprint is a hope, not a fact.
• Did you prove your test is actually running? Before you trust a suite, break something on purpose and watch it fail.

Prismwar ships with a ledger wall where users post their best cards. That wall is read-only. The game's state is signed, and the signature is verified on load. Every one of these decisions is backed by a sabotage that proves what fails if you remove it.

That's not perfection. It's shipping with evidence.

---

Prismwar is live at play.dhseadev.online./ The gate suite is in the DHSeaDev arcade build at scripts/verify-prismwar.mjs and sabotage-prismwar.mjs on GitHub.