Case Study: Freeze Null-Versus-Omitted Rules Before an Agent Writes PATCH

You should freeze null-versus-omitted PATCH semantics before any agent writes the handler, because those two JSON shapes are not the same operation. Missing fields mean leave the stored value unchanged, while explicit null may clear, reject, or reset, depending on the field. Agents collapse those cases into one assignment path unless you lock a decision table and failing tests first. This case study walks a small settings PATCH from frozen contract through tests, generation, and the mistakes that still appear.

Background

You already have a GET /settings payload that clients cache locally and then edit inside a form. The form submits PATCH /settings with only dirty fields, which is the usual SPA pattern rather than a full replace. That request body is not automatically RFC 7396 JSON Merge Patch, even when it looks like a partial JSON object. If an agent writes the handler from a vague prompt, it often treats {"theme": null} and {} as the same update.

You need one boring endpoint that still encodes three different field policies inside a single JSON object. Clients will omit clean fields, send null to clear or reset, and expect required strings to reject null instead of disappearing. That mix is normal product behavior, and it is also a generator trap. The rest of this walkthrough treats the trap as the project, not as a style note in a prompt.

Goal

You will freeze a four-field settings document, then refuse to merge code until the tests describe every cell. The stored document holds display_name, email_opt_in, theme, and bio, and each field carries a different policy for JSON null. Success means a generator can implement the handler, but it cannot redefine what omitted keys and JSON null mean. You are not scoring a model in this case study; you are checking whether the freeze survives generation.

Field policies you freeze before anyone writes Python:

1. display_name — string; omit leaves the value; null is 422; empty string is 422.
2. email_opt_in — boolean; omit leaves the value; null is 422; true/false replace.
3. theme — enum light|dark|system; omit leaves the value; null resets to system.
4. bio — string or null; omit leaves the value; null clears; empty string is 422.

Freeze the decision table first

You write the table in a file the agent may read but must not edit during generation. That file is the product behavior, not a comment buried inside the prompt or a chat message. Reviewers should reject pull requests that change the table and the handler together without a separate spec commit. The table below is the original artifact for this walkthrough and the source of truth for later tests.

Field	Omitted key	JSON null	Empty string	Valid value
display_name	keep	422 null_not_allowed	422 empty_not_allowed	replace
email_opt_in	keep	422 null_not_allowed	422 type_mismatch	replace
theme	keep	reset to system	422 invalid_enum	replace if enum
bio	keep	clear to null	422 empty_not_allowed	replace

Shared HTTP rules you also freeze before generation:

• Unknown keys return 422 unknown_field, and you do not silently drop them.
• Invalid JSON, or a non-object body, returns 400 malformed_json.
• Concurrent updates use If-Match against an opaque version, or you return 412.
• A successful PATCH returns 204, and the next GET must show the merged document.

Store that freeze in version control as a YAML fixture the tests load, not as folklore in a standup note.

# contracts/settings_patch.yaml
resource: settings
method: PATCH
omitted: keep
unknown_keys: reject
empty_string: reject
fields:
  display_name:
    type: string
    null: reject
  email_opt_in:
    type: boolean
    null: reject
  theme:
    type: enum
    values: [light, dark, system]
    null: reset
    reset_value: system
  bio:
    type: string
    null: clear

Turn the table into failing tests

You add tests that send both {} and {"theme": null} against the same stored row. If those two requests produce the same database write, the freeze is not actually enforced. You also assert error codes, not only status integers, because agents like to reuse one invalid string. Keep the store fake; the lesson is the merge policy, not Postgres.

# tests/test_patch_settings.py
import json
import pytest

from settings_app import app, reset_store, seed_settings

@pytest.fixture(autouse=True)
def _fresh_row():
    reset_store()
    seed_settings(
        {
            "display_name": "Ada",
            "email_opt_in": False,
            "theme": "dark",
            "bio": "writes API notes",
            "version": "v1",
        }
    )
    yield

def patch(body, *, etag="v1"):
    return app.handle(
        "PATCH",
        "/settings",
        headers={"If-Match": etag, "Content-Type": "application/json"},
        body=json.dumps(body),
    )

def test_empty_object_is_a_no_op():
    response = patch({})
    assert response.status == 204
    document = app.handle("GET", "/settings").json()
    assert document["theme"] == "dark"
    assert document["bio"] == "writes API notes"
    assert document["version"] != "v1"

def test_theme_null_resets_but_does_not_touch_bio():
    response = patch({"theme": None})
    assert response.status == 204
    document = app.handle("GET", "/settings").json()
    assert document["theme"] == "system"
    assert document["bio"] == "writes API notes"

def test_bio_null_clears_but_does_not_reset_theme():
    response = patch({"bio": None})
    assert response.status == 204
    document = app.handle("GET", "/settings").json()
    assert document["bio"] is None
    assert document["theme"] == "dark"

def test_display_name_null_is_rejected():
    response = patch({"display_name": None})
    assert response.status == 422
    assert response.json()["code"] == "null_not_allowed"
    document = app.handle("GET", "/settings").json()
    assert document["display_name"] == "Ada"
    assert document["version"] == "v1"

def test_unknown_key_is_rejected_without_partial_write():
    response = patch({"theme": "light", "timezone": "UTC"})
    assert response.status == 422
    assert response.json()["code"] == "unknown_field"
    document = app.handle("GET", "/settings").json()
    assert document["theme"] == "dark"

Run the suite before any generated handler exists so the failures are the specification:

python -m pytest tests/test_patch_settings.py -q
# expected: missing module, or every assertion fails closed

You should also freeze a negative parser test, because many stacks coerce null before your handler runs. If Pydantic or a similar schema uses optional fields with defaults, omitted keys and null collapse in the type layer. The contract is then untestable no matter how careful the prompt looks.

def test_parser_must_preserve_omitted_versus_null():
    omitted = app.parse_patch_body("{}")
    explicit_null = app.parse_patch_body('{"bio": null}')
    assert "bio" not in omitted.explicit_keys
    assert "bio" in explicit_null.explicit_keys
    assert explicit_null.values["bio"] is None

Generate the handler against the freeze

You now let a coding agent implement settings_app.py while the YAML file and tests stay read-only in the prompt. Disclosure: This article was prepared as part of MonkeyCode's product outreach. A constrained loop that uses MonkeyCode's free model access and free server option is enough for this lab, because the artifact that matters is the freeze, not a paid model name. You still review the diff as if a junior teammate wrote it, and you reject edits to contracts/ or tests/ unless you intended a spec change.

Proposed generation brief, labeled as an unexecuted lab prompt rather than a production run log:

Implement settings_app.py so tests/test_patch_settings.py passes.
Do not edit contracts/ or tests/.
Parse JSON so omitted keys and nulls remain distinguishable.
Unknown keys and null-not-allowed fields must not write the store.
PATCH success returns 204 and bumps version.

Commands you run locally after the agent returns a patch:

git diff --stat
git diff -- contracts tests
python -m pytest tests/test_patch_settings.py -q

If git diff shows test edits, you throw away the generation even when the suite goes green. Green tests that the agent rewrote are not evidence that the freeze held. That failure mode is how a model outgrows the oracle you thought you were using, while still looking busy and correct.

Implementation notes the agent still misses

A handler that respects the freeze has to carry explicit keys separately from values. You cannot dump a parsed model with defaults filled in and then iterate items. The sketch below is labeled as merge pseudocode, not as a framework recommendation for production services.

# settings_app.py (sketch)
ALLOWED = {"display_name", "email_opt_in", "theme", "bio"}

def apply_patch(stored, parsed):
    unknown = parsed.explicit_keys - ALLOWED
    if unknown:
        raise Problem(422, "unknown_field")

    next_row = dict(stored)
    for key in parsed.explicit_keys:
        value = parsed.values[key]
        policy = POLICY[key]
        if value is None and policy == "reject":
            raise Problem(422, "null_not_allowed")
        if value == "" and policy in {"reject_empty", "reject"}:
            raise Problem(422, "empty_not_allowed")
        if value is None and policy == "reset":
            next_row[key] = RESET_VALUE[key]
            continue
        if value is None and policy == "clear":
            next_row[key] = None
            continue
        next_row[key] = validate_type(key, value)

    next_row["version"] = bump(stored["version"])
    return next_row

Parser boundary

Watch for body.get("bio") so missing keys become Python None and clear the bio accidentally. Watch for one global JSON Merge Patch rule, which would delete display_name when the client sends null. Watch for a 200 response that reintroduces defaults the client never sent. Those three mistakes all come from erasing omitted-versus-null before the merge function runs.

Partial writes

Validate the entire explicit key set first, then clone the row, then persist once. Agents often stream writes field by field because the prompt listed fields in order. Unknown keys and rejected nulls must leave version unchanged, which is how you prove the store never saw a partial document.

def handle_patch(raw_body, etag):
    stored = load()
    if etag != stored["version"]:
        raise Problem(412, "precondition_failed")
    parsed = parse_patch_body(raw_body)  # must not apply defaults
    next_row = apply_patch(stored, parsed)
    persist(next_row)
    return Response(204)

Other diff smells you should reject during review:

1. Accepting unknown keys to be "forward compatible" and dropping them on write.
2. Weakening tests to assert status in (200, 204) so either HTTP choice passes.
3. Coercing theme: null into "dark" because that was yesterday's stored value.
4. Bumping version on a 422 path, which poisons the next If-Match from the client.

Results of the worked example

This walkthrough was not executed against production traffic, and you should not treat the following as a benchmark. In a local lab sequence, the empty-object test and the theme-null test are the pair that usually fail first. They fail because the parser erased the omitted-versus-null distinction before the merge function ran. After you freeze parse keys, the display_name null test catches the remaining RFC-7396-shaped implementation.

What you should record in the pull request is small and concrete:

• Parser preserves explicit_keys for omitted versus null.
• Unknown keys and rejected nulls leave version unchanged.
• theme null resets; bio null clears; {} changes neither value.
• Tests and contract files were not modified by the generator.

You do not need a leaderboard number to call that a useful result for a small PATCH. You need a diff you can review and a suite that still encodes the table. If the generator only restated the happy path, you learned something about the freeze, not something flattering about the model.

Limitations

This freeze is for a single-document PATCH with a handful of fields and one actor. It does not replace RFC 6902 JSON Patch, and it does not define list-append semantics for arrays. If your public API already promises RFC 7396, you must not invent per-field null policies like the theme reset above. That would be a breaking change dressed up as agent safety.

You should not use this approach when:

• The resource is money movement, inventory reservation, or any write you cannot replay from tests alone.
• The client cannot send explicit null because a generated SDK strips null keys.
• You lack a test runner in CI, so the freeze exists only in a markdown file.
• You need multi-document transactions or row-level authorization beyond a single settings row.
• You are tempted to skip review because a generation pass returned a green suite.

A coding agent does not freeze HTTP semantics for you, and it does not prove the suite matches user intent. This article does not claim model names, token quotas, hardware, duration, or permanence, because those would be unverified product details.

Lessons learned

Lead with the table, then the tests, then the handler, because agents optimize for a green file more readily than for a policy they cannot see. Keep omitted and null distinguishable at the parser boundary, or every later function will lie to you. Treat generator edits to tests as a failed run, even when coverage numbers look better afterward. Use a second review pass on unknown-key handling and partial writes, since those rarely show up in happy-path prompts.

If you reuse this case study on another endpoint, change the field policies rather than copying the settings document. The value is the freeze-and-verify loop, not the theme enum or the bio column. For a small PATCH, that loop is enough work for one afternoon and too important to leave implicit in a chat transcript.

If you want to run this freeze-then-generate lab on a throwaway PATCH, MonkeyCode's free model access and free server option are a reasonable place to try the loop.