48-Hour Field Notes: The Set Snapshot That Only Agreed With My Laptop

I spent forty-eight hours believing a unit test was deterministic, because it never failed on my laptop. The assertion compared a joined string of required JSON fields against a golden snapshot I had copied once. Why would four short keys shuffle themselves after the process restarted on another box? That question only showed up after I ran the same file away from my local virtualenv.

This is a field notebook, not a traffic postmortem, and I am not attaching fake production metrics. I wanted a tiny, repeatable Python failure that my laptop kept hiding from every rerun. If you iterate a set of strings and freeze that walk into a string, you already know the punchline.

What I was actually asserting

The helper was supposed to prove a token payload contained a required field set. I did not compare two sets, which would have been order-free and honest. I joined the set into a comma-separated snapshot because the failure message looked nicer in the terminal. Does prettier output count as a specification, or is it just a comfort blanket?

REQUIRED = {"user_id", "tenant", "scope", "issued_at"}

def snapshot(fields: set[str]) -> str:
    return ",".join(fields)

def test_required_fields_snapshot():
    got = snapshot(REQUIRED)
    assert got == "issued_at,scope,tenant,user_id"

On my machine that assertion stayed green across evening reruns, editor restarts, and a full night of idle time. Would you have ripped it apart, or would you have trusted the golden string too? I trusted it, and that is the whole amateur hour.

Hours 0–12: what I tried first

I treated the failure as environment drift, because remote boxes love surprising people. Here is the messy list, in the order I actually burned daylight and coffee.

1. I reinstalled the virtualenv and pinned the same CPython minor version I had locally.
2. I dumped locale, sys.platform, and os.getcwd(), hunting a path or encoding ghost.
3. I blamed pytest collection order, then a random plugin I was not even using.
4. I printed list(REQUIRED) once, saw a familiar order, and closed the laptop like a fool.

Did any of that touch hash randomization? Not even close, and I still wrote notes as if networking had betrayed me. The laptop kept agreeing with itself because one process seed can look eternal until you start a different interpreter. Have you ever watched a green suite become a personality test for a single machine?

Hours 12–24: what actually broke

The remote run printed scope,issued_at,user_id,tenant and exploded the snapshot immediately. Same source file, same set literal, same assertion, different Python process. Was the set contents wrong? No, REQUIRED == {"user_id", "tenant", "scope", "issued_at"} still passed when I compared sets directly.

CPython randomizes hashes of str and bytes unless you pin PYTHONHASHSEED. Set iteration follows those hashes, so a join is a hidden order dependency. Dicts keep insertion order since 3.7, but inserting from a set still inherits the set walk. I had been testing hash luck, not the payload contract.

Primary docs live here, and they are not seasonal blog folklore: PYTHONHASHSEED. If the variable is unset or set to random, each process gets a new seed. A numeric value pins the seed, and 0 disables randomization, which is a test knob rather than a product feature.

I added a header so the next remote shell could not gaslight me again.

import os
import sys

print("python", sys.version.split()[0])
print("PYTHONHASHSEED", os.environ.get("PYTHONHASHSEED", "<unset>"))
print("hash(probe)", hash("probe"))

When hash("probe") changes, your set walk can change with it. Why did I spend twelve hours on locale before printing that one line?

Hours 24–36: a matrix I should have run at hour one

I finally stopped guessing and forced the seed from the shell. You can reproduce the whole mess without a test framework if you want the smell to stay small.

# save as hash_order_notes.py — local reproduction, not a production service
REQUIRED = {"user_id", "tenant", "scope", "issued_at"}

def snapshot(fields: set[str]) -> str:
    return ",".join(fields)

def main() -> None:
    print("snapshot=", snapshot(REQUIRED))
    print("set-eq=", REQUIRED == {"issued_at", "scope", "tenant", "user_id"})

if __name__ == "__main__":
    main()

python hash_order_notes.py
PYTHONHASHSEED=0 python hash_order_notes.py
PYTHONHASHSEED=1 python hash_order_notes.py
PYTHONHASHSEED=random python hash_order_notes.py

Run that loop a few times and watch the snapshot line twitch while set-eq stays true. Is your test asserting equality of unordered collections, or is it photographing their current walk? If it is photographing, the other machine is not flaky. Your assertion is.

I also wanted a second interpreter that was not my laptop, because local luck is sticky. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access to ask for a seed matrix and assertions that would not encode walk order. Then I used the free server option to run the same file off my laptop, without inventing hardware details I do not have.

The model did not invent a novel language bug, and it did not need a fake benchmark either. It reminded me to compare sets as sets, and to pin PYTHONHASHSEED only in tests that truly need a stable hash. That was enough to stop me from rewriting JSON parsers at hour thirty. Would a larger model have helped more here, or would it have just narrated the same docs with extra confidence?

A small artifact: order-free tests plus a seed job

Here is the version I would actually keep in a repo. It still prints a snapshot for humans, but the contract no longer depends on hash walk order.

from __future__ import annotations

import os
import unittest

REQUIRED = frozenset({"user_id", "tenant", "scope", "issued_at"})

def missing(fields: set[str]) -> set[str]:
    return set(REQUIRED) - set(fields)

class RequiredFieldsTests(unittest.TestCase):
    def test_required_fields_are_present(self) -> None:
        payload = {"user_id", "tenant", "scope", "issued_at", "extra"}
        self.assertEqual(missing(payload), set())

    def test_missing_field_is_named(self) -> None:
        payload = {"user_id", "tenant", "scope"}
        self.assertEqual(missing(payload), {"issued_at"})

    def test_snapshot_is_for_humans_only(self) -> None:
        # Label: demonstration, not a contract on iteration order.
        text = ",".join(sorted(REQUIRED))
        self.assertEqual(text, "issued_at,scope,tenant,user_id")

    def test_debug_header_does_not_assert_a_hash(self) -> None:
        seed = os.environ.get("PYTHONHASHSEED", "<unset>")
        self.assertTrue(len(seed) > 0)

if __name__ == "__main__":
    unittest.main()

Notice sorted on the human string, and frozenset for the constant. If you truly need hash-stable behavior, pin the seed in the test command, not in application code. The commands below are a decision aid, not a performance study.

PYTHONHASHSEED=0 python -m unittest hash_order_notes_test.py -q
PYTHONHASHSEED=1 python -m unittest hash_order_notes_test.py -q
PYTHONHASHSEED=random python -m unittest hash_order_notes_test.py -q

Decision table I wish I had on the desk

Observation	Likely cause	Do this next	Do not do this
String join of a set changes across processes	str hash randomization	Compare sets, or sorted() before join	Freeze one laptop snapshot
Dict built from a set changes key order	Insertion follows set walk	Build from a list or sort keys	Blame the JSON library
Test fails only on another machine	New interpreter, new seed	Run a PYTHONHASHSEED matrix	Reinstall packages first
hash("x") differs between runs	Expected unless seed is pinned	Pin seed only for hash tests	Export PYTHONHASHSEED=0 in production
Set equality still passes	Contents are fine	Keep equality tests	Rewrite parsers for shuffled keys

Hours 36–48: what I would repeat

I would start with the matrix, not with folklore about Docker layers and mystery locales. I would also refuse golden strings that are not sorted, even when they look pretty in a failure diff. Which of those habits would have saved the first evening?

• Repeat: assert with == on sets, frozensets, or sorted tuples.
• Repeat: print PYTHONHASHSEED and hash("probe") in the debug header.
• Repeat: run at least 0, 1, and random before calling a test deterministic.
• Skip: pinning PYTHONHASHSEED=0 globally to silence a sloppy snapshot.
• Skip: asking a model to fix remote networking when the diff is four keys.

Would I still use a second machine? Yes, because my laptop is a single seed factory with a friendly history. One interpreter family will keep confirming its own luck until you break the spell on purpose.

Limitations, and who should not copy this

This notebook is about test honesty, not about making hashes deterministic in production services. Hash randomization exists to make certain attack classes harder, and turning it off process-wide is a trade you should not make casually. Read the Python docs before you export a fixed seed in a real service, especially if those strings ever came from users.

Do not use this approach if your failing test already compares structured objects without encoding order. Do not paste secrets into a hosted model or a shared server, even when those options are free. Do not treat a model suggestion as a vulnerability write-up when your only evidence is ",".join(some_set).

If your suite already sorts keys, uses set equality, and runs under an explicit seed matrix, you will learn nothing new here. That is a good outcome. Go back to the bug that actually eats data, because this one only eats evenings.

What I am keeping on the sticky note

Unordered collections are not snapshots, and a green laptop is still one process family. Ask whether the assertion would survive PYTHONHASHSEED=random before you blame the remote filesystem, the JSON parser, or the other operator. Forty-eight hours of local agreement is not evidence. It is a sampling error with a comfortable chair.

If you want a second interpreter that is not your laptop, MonkeyCode's free model access and free server option were enough for this matrix, and I stopped there.