DEV Community

Todd Sullivan
Todd Sullivan

Posted on

I Used 25 Simulated Players to Verify an AI-Assisted Swift Refactor

#ai #swift #agents #testing

I let Claude co-author a fairly invasive refactor: split an adaptive iOS game's reusable AI engine out of the app repository and consume it as a versioned Swift package.

The change touched 47 files. The app commit removed 2,664 lines and added 112.

A clean build was not enough evidence that this worked.

The engine contains the parts that influence behaviour: telemetry, player modelling, the adaptive Director, difficulty guardrails, rendering support and persistence. The app keeps its own level rules, scoring, upgrades and room templates. A dependency can compile while a moved default, changed data boundary or subtly different seed path changes how the AI behaves.

So the architecture rule was:

  • reusable policy and infrastructure go into the engine package
  • game-specific rules stay in a local StationRules package
  • game-specific room data stays in the app
  • the app pins the engine at 0.1.0

Then I used the headless simulator as a behavioural checksum.

I built and ran the same deterministic 25-player simulation immediately before and after the split. Both runs produced exactly:

levels completed: 237
run overs: 1
boss attempts: 317
boss wins: 239 (75%)
rooms: 1,987
deaths: 61
shot accuracy: 0.490
labels: 2,410 total — 191 bored, 2,171 flow, 36 frustrated, 12 aboutToQuit
Enter fullscreen mode Exit fullscreen mode

The numbers matched exactly.

All eight StationRules tests passed too, but those tests only cover isolated scoring, upgrades and progression rules. The simulator crosses the useful boundary: generated rooms, bot behaviour, combat, telemetry, adaptive decisions and engagement labels all running together.

That matters more when an AI coding agent is doing the refactor. Agents are good at moving symbols, fixing imports and making the compiler green. They are less able to prove that a stateful adaptive system still behaves the same after ownership moves across packages.

The useful pattern was not “review every changed line harder.” It was:

  1. make the simulation deterministic
  2. capture a small set of behaviour-level outputs
  3. perform the architecture change
  4. require exact output equality

That turns a large AI-assisted refactor into a boring comparison.

For deterministic systems, a behavioural checksum is often a better refactor test than another pile of mocks. The compiler proved the package boundary was valid. The simulator proved I had not changed the game hiding behind it.

Top comments (0)