DEV Community

Awiros
Awiros

Posted on

Freeze the config: snapshotting settings onto the record they govern

Here's a made-up shift at a plant that inspects welded frames (every name, time and setting in this post is illustrative). An inspector starts an inspection on frame A118 at 11:20 with a handheld camera. The vision model flags a weld gap, the inspector confirms it, and because the org's rules don't ask for a close-up photo yet, they log the finding and move on to the next section.

At 11:40 the quality lead changes an org setting: from now on every finding needs a close-up photo. Reasonable change. They made it for the next batch.

At 11:55 the inspector taps Complete on A118:

11:55:02 POST /inspections/insp-7/complete
11:55:02 422 {"error": "close-up photo required for: weld_gap"}
Enter fullscreen mode Exit fullscreen mode

The weld gap was logged 35 minutes earlier under rules that didn't ask for a photo. The frame has since moved down the line. The inspector now has an inspection they can't close, and the only ways out are walking back to a frame that's gone or asking an admin to flip the setting off for a minute (which changes it for everyone else too).

Nothing here is a bug in the validation. The completion check did exactly what it was told. It read the settings as of 11:55, but the work had started under the 11:20 settings.

Where it actually goes wrong

The "before" code, condensed from the toy in this post:

def complete_reading_live(store, insp):
    s = store.resolve(insp.org, insp.site)       # settings right now
    if s["require_closeup_photo"]:
        missing = [f.label for f in insp.findings if f.photos == 0]
        if missing:
            raise ValidationError(f"close-up photo required for: {', '.join(missing)}")
    insp.status = "done"
Enter fullscreen mode Exit fullscreen mode

Every place in the backend that reads store.resolve(...) during an inspection is a seam where the rules can change under in-flight work. Completion is the one that blows up loudly. The quieter ones are worse: a photo-count limit that tightens halfway through so the next finding is rejected, a rework rule that turns on and sends a half-done inspection down a different path, or a report generated tomorrow that describes yesterday's inspection using today's rules.

Visual inspection is where this comes up for us (aiscope files structured evidence for every inspection, and that evidence has to make sense later), but the same shape shows up for any long-running record: a camera analytics job, a batch annotation task, a scheduled export.

Resolve once, freeze, read from the snapshot

The reference pattern we recommend has three parts.

At creation, resolve the settings once. In the toy that means platform defaults, then org overrides, then site overrides:

def resolve(self, org, site):
    out = dict(DEFAULTS)
    out.update(self.org.get(org, {}))
    out.update(self.site.get((org, site), {}))
    return out
Enter fullscreen mode Exit fullscreen mode

Store the result on the inspection as a snapshot, with a version, a timestamp and a hash:

{
  "id": "insp-7",
  "product": "frame-A118",
  "config_snapshot": {
    "v": 2,
    "resolved_at": "2026-11-09T11:20:04+05:30",
    "hash": "c6dd35f140d5",
    "settings": {
      "require_closeup_photo": false,
      "max_photos_per_finding": 4,
      "rework_on_missing_part": true,
      "allow_parallel_runs": false
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Then every behaviour check on that inspection reads the snapshot, never the live store:

def complete(self, iid):
    insp = self.inspections[iid]
    if insp.flag("require_closeup_photo"):      # frozen at 11:20
        ...
    insp.status = "done"
Enter fullscreen mode Exit fullscreen mode

Same shift, replayed in the toy:

11:20 create insp-7     snapshot c6dd35f140d5 require_closeup_photo=False
11:22 finding weld_gap  photos=0
11:40 org setting       require_closeup_photo -> True
11:45 create insp-8     snapshot require_closeup_photo=True
11:50 finding weld_gap on insp-8 photos=0
11:55 complete insp-7   done
11:58 complete insp-8   422 close-up photo required for: weld_gap
Enter fullscreen mode Exit fullscreen mode

insp-7 closes under its own rules. insp-8, created after the change, gets the new rule. That's what the quality lead meant in the first place.

The UI is the one place that should keep reading live settings, because it's setting up new work. The settings page shows today's values and the "start inspection" form uses them. For an inspection that's already open, show its frozen values with a small line like "rules as of 11:20". Otherwise an inspector looking at the settings page will reasonably ask why their open inspection isn't asking for photos.

In the toy the snapshot is a MappingProxyType and settings is itself a proxy too, so both levels are read-only and snapshot["settings"]["require_closeup_photo"] = True raises TypeError. In a real database you'd enforce the same thing by never writing to that field after create, and having a test that fails if any update path touches it.

Scenario 2: an opt-in flag with inspections already open

A feature request: let two inspectors work on the same product at once (say a large frame split into sections). It's a behaviour change, so it ships as an opt-in flag, allow_parallel_runs, default off, and like every other setting it goes into the snapshot.

The interesting moment is when an org turns it on while single-run inspections are still open. Illustrative timeline:

14:20 create insp-12 on frame-B204   allow_parallel_runs=False (frozen, hash 7e2a91c4b0f3)
15:00 org setting allow_parallel_runs -> True
15:05 create insp-13 on frame-B204
      422 insp-12 was started under single-run rules (snapshot 7e2a91c4b0f3)
15:20 complete insp-12
15:30 create insp-13 on frame-B204   ok
15:31 create insp-14 on frame-B204   ok
Enter fullscreen mode Exit fullscreen mode

The start check looks at the new inspection's resolved flag and at every open inspection's own snapshot:

if open_runs and not resolved["allow_parallel_runs"]:
    raise ValidationError(f"{product} already has an open inspection")
for other in open_runs:
    if not other.flag("allow_parallel_runs"):
        raise ValidationError(f"{other.id} was started under single-run rules ...")
Enter fullscreen mode Exit fullscreen mode

insp-12 was started by someone who was told they had the frame to themselves. Its code paths (who owns which section, when a finding is final) were written for that. Letting insp-13 in at 15:05 would mix rules that were never tested together. Waiting for the old one to close costs a few minutes once per org, at the moment they turn the feature on, and after that the flag behaves as expected. You can pick the other answer and let new runs in. If you do, make it a deliberate decision and write tests for the mixed state, because a check that reads a live value will make that decision for you without anyone noticing.

Scenario 3: the support ticket two weeks later

A ticket comes in: "why did inspection insp-7 close with a finding that has no photo? our rules require photos." Without a snapshot, answering means digging through audit logs for settings changes and lining up timestamps by hand. With one, the toy has a small explain helper that diffs frozen values against today's:

>>> svc.explain("insp-7")
{'require_closeup_photo': (False, True)}
Enter fullscreen mode Exit fullscreen mode

Frozen False, live True. The inspection followed the rules it started with, and the rule changed afterwards. The hash also helps with bulk questions: group inspections by config_snapshot.hash and you can see exactly which rule sets were in use last month, without replaying the settings history.

When a setting has to stay live

Some settings shouldn't be frozen. If a capture source gets disabled because a camera is sending corrupt frames, or a security fix turns something off, you want in-flight work to stop using it right away. The toy keeps an explicit, short list:

LIVE_KEYS = frozenset({"disabled_capture_sources"})
Enter fullscreen mode Exit fullscreen mode

and the check that uses it reads the live store on purpose, with a comment saying so. The risk is that this list grows until the snapshot means nothing, so treat adding a key to it as a review question: what happens to an inspection that's half done when this changes?

Snapshots get old too

Snapshots outlive the code that wrote them. Six months from now there's a new setting, and every inspection created before it has a snapshot without that key. The tempting read is snapshot.get(key, DEFAULTS[key]), which is wrong if today's default isn't how the system behaved back then. The toy versions snapshots and upgrades on read:

def upgrade(snap):
    settings = dict(snap["settings"])
    if snap["v"] < 2:
        settings.setdefault("allow_parallel_runs", False)   # behaviour before v2
    return MappingProxyType(settings)
Enter fullscreen mode Exit fullscreen mode

The stored record is never rewritten. The upgrade only changes how old snapshots are read, and the test for it pins an old v1 snapshot as a fixture.

What it costs

Storage, first. Every record carries a copy of its settings. For a few dozen keys that's small, but a snapshot that grows into a nested config blob makes every inspection heavy. If that happens, store the settings once in a content-addressed table keyed by hash and keep only the hash on the record.

Admins can be surprised. A setting change only affecting new work is what you want, but someone will still ask why it "didn't work". The UI line saying which rules an open inspection uses saves a lot of tickets.

Bugs in a setting also get frozen. If a bad value goes out at 9:00 and is fixed at 9:30, every inspection started in between keeps the bad value. You either live with it, add that key to the live list, or write a one-off, audited fix that patches specific snapshots and logs who did it. Pick one before you need it.

And code has to know the difference. Every flag read needs to go through the record (insp.flag(...)) and not through the settings service. A lint rule or a grep in CI for direct resolve( calls outside the create path catches regressions cheaply.

Running the example

snapshot.py is about 150 lines of standard-library Python: a layered settings store, a frozen snapshot with version and hash, an inspection service, the "before" live-read completion, and an explain helper. test_snapshot.py has 10 tests covering the 11:40 change breaking the live-read version, the snapshot version closing cleanly, new inspections picking up new rules, site overrides beating org overrides, the snapshot rejecting writes, the parallel-run flag waiting for older open inspections, a v1 snapshot reading the pre-v2 default, a live key stopping in-flight work, the support diff, and key order not changing the hash. Run python3 test_snapshot.py, no pytest needed. It's a teaching model, so swap the dicts for your database and keep the tests.

Top comments (0)