DEV Community

Halil Coşgun
Halil Coşgun

Posted on

Before You Hunt a Desync, Prove Your Own Simulation Is Deterministic

The test nobody runs first

When the system catches a desync between two players, the match dies, someone files a bug, and the hunt begins. Which client was wrong? Which subsystem? Which tick? Two machines, two builds, two network stacks, two sets of hardware, and somewhere in there one value went a different way.

The hardest part of a desync is the hunt itself. What makes it worse is that a large share of those bugs were never about the second machine at all. The simulation was not deterministic on one machine either, and nobody had checked.

There is a test for that. It needs one computer, one recording, and one command. I call it the self-check, and it is the first thing I would run on any simulation that claims to be deterministic.

What the self-check actually is

Determinism means a simple thing: same starting state, same inputs, same result. Every time. On any machine.

The multi-machine version of that promise is what lockstep netcode depends on. But there is a weaker version hiding inside it, and the weaker version is much easier to test:

Same starting state, same inputs, same result, on the same machine, twice in a row.

If your simulation cannot pass that, it will never pass the hard version. And unlike the hard version, you can test it alone, offline, in a few seconds, before any player is involved.

The procedure is three steps:

  1. Play a session and record it. Tickwise writes the inputs and a hash of every tick into a .rec file.
  2. Feed those same recorded inputs back through your simulation and record that run too.
  3. Compare the two recordings.
tickwise compare original.rec replayed.rec
Enter fullscreen mode Exit fullscreen mode

If the verdict is anything but identical, your simulation is not deterministic, and you just found out before your players did.

Why this catches so much

The interesting part is what fails this test. Almost none of it involves the network.

Iteration order. You store entities in a HashMap and iterate it during the update. Rust randomizes hash seeds per process, so the second run walks the same entities in a different order. If anything in that loop is order-sensitive, and in a physics step it usually is, the two runs diverge. The same class of bug exists in every language with unordered containers, and it is one of the four chaos modes shipped with Tickwise for exactly this reason.

Time leaking into the simulation. Somewhere deep in a subsystem, someone reads the wall clock or a frame delta instead of the fixed tick. The first run took 16.2 milliseconds on that frame, the replay took 15.9, and the simulation quietly took a different branch. This one hides well, because on a fast machine the numbers look close enough to be invisible until they are not.

Uninitialized or stale state. A scratch buffer that is not cleared between ticks, a value carried over from the previous run, a lazily built cache that exists on the second pass but not the first. The replay starts from a slightly different world than the recording did.

Global mutable state. A static counter, a shared random generator, a data or system that survives between sessions. The first run leaves fingerprints that the second run reads.

Every one of these fails on a single machine. None of them needs a second client, a network, or another player. Which means every one of them can be found before a match is ever played.

Watching it fail on purpose

Tickwise ships with a reference simulation and a --chaos flag that injects a known class of non-determinism at a tick you choose. That flag exists so the tool can prove it works, and it doubles as a way to see the self-check fail without breaking your own code first.

Record a clean session, then record a second one with chaos turned on from tick 4021:

cargo run -q -p tickwise-refsim --example record_demo -- clean.rec
cargo run -q -p tickwise-refsim --example record_demo -- chaotic.rec --chaos stale-value 4021
tickwise compare clean.rec chaotic.rec
Enter fullscreen mode Exit fullscreen mode
  verdict        first divergence at tick 4021, caught by the light hash,
                 confirmed by the full hash at tick 4200, last agreement at tick 4020
Enter fullscreen mode Exit fullscreen mode

Two recordings that should have been identical, one exact tick where they stopped being identical. That is the whole shape of the self-check, whether the cause is a deliberate chaos flag or a real bug in your code.

The stricter version

Comparing two recordings tells you that a divergence happened. There is a second mode that tells you the moment it happens, while the replay is still running.

When you replay a recording, Tickwise can verify every tick against the hashes stored in the file:

let mut rep = Replayer::open("session.rec", ReplayConfig {
    verify_hashes: true,
    ..Default::default()
})?;

while let Some(step) = rep.next_step() {
    my_sim.apply_inputs(step.inputs());
    my_sim.tick();
    rep.after_tick(&probe)?;
}
Enter fullscreen mode Exit fullscreen mode

With verify_hashes on, the replay fails at the first tick where the live hash does not match the recorded one. No second file, no comparison step, no waiting until the end. This is the version worth wiring into CI: record one canonical session, commit it, and replay it on every push. The day someone adds a HashMap iteration to the physics step, the build tells them.

Where the self-check stops

Honesty about limits, because a test that overpromises is worse than no test.

The self-check proves that your simulation is deterministic on one machine, with one build, in one process. That is the floor, not the ceiling. Passing it does not mean two different machines will agree. Cross-platform float behavior, compiler flags, CPU differences, and thread counts all live above this line, and the multi-machine comparison is what covers them.

It also only covers what your hashes cover. If a field is not in light_hash or full_hash, the self-check cannot see it drift. This is why Tickwise reports which hash caught a divergence: when the full hash fires and the light hash saw nothing, you have learned something about your hash coverage, not just about your bug. The repository has a checklist for that question.

So the self-check is not the whole story. It is the cheapest part of the story, and the part almost everyone skips.

Start here

If you are building anything that depends on determinism, whether that is lockstep netcode, rollback, replay files, or a deterministic test suite, run the weak version of the promise first. One machine, one recording, one command.

cargo install tickwise-cli
Enter fullscreen mode Exit fullscreen mode

The repository is at github.com/cosgunhalil/Tickwise, dual licensed MIT or Apache-2.0. The tutorial walks through the full workflow in about fifteen minutes, and the hash coverage checklist answers the question this post keeps circling: what belongs in each hash.

If you run the self-check on your own simulation and it fails, I would genuinely like to hear what caught it. Those stories are how the chaos mode list grows.

Thank you for finding this worth your time.

Top comments (0)