DEV Community: Anuj Sadani

Valid JSON Is Only Half the Output Contract

Anuj Sadani — Sat, 24 Oct 2026 02:29:27 +0000

A response can be valid JSON and still give a downstream consumer the wrong shape. A Markdown handoff can read fluently while omitting the one section the next engineer needs.

Those are concrete interface failures. They deserve checks that say which field or section broke the agreement.

Output Contracts is a small Python validator for explicit Markdown and JSON requirements. It does not call a model, score the writing, or silently repair the output.

Write down the consumer requirement

Imagine a work handoff that must contain these sections in order:

Summary
Evidence
Unresolved questions
Next steps

A handoff that omits unresolved questions and moves evidence below the next steps might be readable, but it no longer follows the agreed interface. The bundled Markdown contract declares required headings, ordering, and a forbidden completion phrase.

The last rule has a narrow meaning. Banning Everything is complete. can enforce a wording constraint. It cannot determine whether the work is actually complete or whether another phrase makes an equally unsupported claim.

Validate the bad handoff

Use Python 3.11 or newer. There are no third-party runtime dependencies:

git clone https://github.com/asadani/output-contracts.git
cd output-contracts
out_dir="$(mktemp -d)"

python -m output_contracts validate \
  fixtures/handoff-contract.json fixtures/handoff-bad.md \
  --format markdown --out "$out_dir/handoff-review.md"

python -m output_contracts inspect fixtures/handoff-contract.json

Validation exits with status 1 because the fixture violates the contract. The report identifies the missing section, the ordering problem, and the forbidden phrase. inspect validates the contract itself and prints its normalized summary.

A clean validation returns 0. Invalid contracts, unreadable input, and command errors return 2, so a calling script can distinguish a rejected output from a broken validation operation.

Check values as well as syntax

The JSON example requires an identifier, a constrained status, an integer count, and a non-empty items array. The bad fixture includes:

{
  "id": "case-18",
  "status": "accepted",
  "count": true,
  "items": []
}

This parses as JSON. It still violates two declared requirements: true does not satisfy the integer type, and an empty array does not satisfy the minimum item count.

Run both versions:

python -m output_contracts validate \
  fixtures/result-contract.json fixtures/result-bad.json \
  --format json --out "$out_dir/bad-result-review.json"

python -m output_contracts validate \
  fixtures/result-contract.json fixtures/result-good.json \
  --format markdown --out "$out_dir/good-result-review.md"

The adverse example exits with 1; the good example exits with 0. Fields use JSON pointers, and null remains distinct from a missing field.

Keep the contract proportionate

The current implementation is deliberately small. Markdown heading checks recognize ATX headings outside fenced code; they do not treat bold text or HTML as headings. Forbidden patterns scan the full output, including code blocks.

That detail can create false positives. A tutorial quoting a forbidden phrase in an example may need a different contract from a handoff that must never contain it. Patterns come from trusted contract authors, and complex regular expressions are not sandboxed or guaranteed to run in linear time.

For general JSON Schema validation, an established validator may fit better. This project is a compact option for a bounded workflow that needs both report-oriented Markdown checks and a small set of JSON field constraints.

Validate before the next operation

Put structural validation at the point where another person or program consumes the output. Save the report and route violations for correction. Keep content review separate: a required Evidence section can still contain a false statement, and a correctly typed count can still be wrong.

Start with one recurring failure that costs someone time. Add a rule for it, check useful and adverse examples, and watch for false positives before adding more constraints.

The MIT-licensed repository includes the fixture contracts, format reference, and validator. Its useful promise is precise feedback about declared rules.

AI disclosure: This article was drafted with AI assistance using the project's code and fixtures. Passing these checks establishes structural conformance, not factual correctness.

Make Unresolved Decisions Visible Before a Specification Reaches Engineering

Anuj Sadani — Fri, 23 Oct 2026 16:18:36 +0000

A specification can look complete while containing two limits for the same field and an unanswered question about who may approve an action. A reviewer might notice both, but the next revision can make it hard to tell which findings were discussed and which still apply.

Spec Sentry explores a narrow approach: annotate the facts that need mechanical checks, then keep review decisions in a separate ledger.

This is most useful when a team is willing to make important uncertainty explicit. The checker does not promise to find every contradiction in ordinary prose.

Declare the parts that must agree

The fixture contains these declarations:

@quantity preview.max_size = 10 MB
@quantity preview.max_size = 10 GB
@decision sharing-policy | pending | Who may invite guests?

Both quantities use the same key but declare conflicting values or units. The pending decision makes an unanswered product question visible instead of leaving an engineer to invent a default.

Term references can also be explicit:

The initial preview uses [[term:external collaborator]].
@define workspace: A named product collaboration area.

The undefined external collaborator reference becomes a finding. A definition for a different term does not resolve it.

These directives use a small documented grammar. They are easy to inspect because they declare the thing being checked instead of asking the checker to infer the meaning of every sentence.

Check the fixture

Use Python 3.11 or newer. The runtime needs no third-party dependencies:

git clone https://github.com/asadani/spec-sentry.git
cd spec-sentry
out_dir="$(mktemp -d)"

python -m spec_sentry check sharing-spec.md \
  --root fixtures --out "$out_dir/findings.json"

The command exits with status 1 because the example intentionally contains findings. Status 0 means no configured findings were observed; status 2 indicates invalid input or an operational error.

The output locates the conflicting declarations, undefined marked term, and pending decision. It gives the reviewer source evidence to inspect rather than editing the document.

A reviewer decision belongs to a version

A reviewer can accept, dismiss, or defer a finding with a note. The decision ledger is separate from the specification and records the fingerprint associated with the review.

That separation matters after an edit. A dismissal against an older version should not automatically hide a finding in new text. When the relevant evidence changes, the prior disposition becomes stale and needs another look.

The report excludes only fresh dismissed findings from its unresolved result. Accepting a finding records a review decision; it does not erase the underlying issue. This keeps discussion history distinct from a claim that the specification is now correct.

The full walkthrough demonstrates the check, decision, and report lifecycle with a temporary copy of the fixture.

Pay attention to the cost of annotations

Explicit syntax trades broad coverage for reproducible checks. If a limit appears only in prose without a declared quantity, the tool cannot assume that it belongs to a matching key. Markdown headings do not create implicit scopes, and units are compared as written rather than converted.

Those limits affect how a team should use the tool. Annotate the few quantities, terms, and decisions that repeatedly create ambiguity. Do not require a new notation for every sentence unless it produces enough useful findings to justify the authoring burden.

The same principle applies to review ledgers. A ledger that requires more effort than it saves will become another stale artifact.

Try it before a planning handoff

Take one small feature spec with a few consequential limits and unresolved questions. Add the explicit declarations, run the check, and review the findings with the engineer who will implement the feature.

Measure whether the check surfaces something actionable and whether the recorded decisions survive the next revision without concealing new uncertainty. That is a more useful test than counting annotations or declaring the document approved because the report is clean.

The MIT-licensed repository includes the grammar, ledger format, fixtures, and checks. It offers a bounded way to keep selected questions visible through review.

AI disclosure: This article was drafted with AI assistance using the project's implementation and synthetic specification. It does not report a measured reduction in specification defects.