Free Models Love YAML: Add a Schema Diff Before You Merge Their Patches

A model-generated config patch can be syntactically valid, pass a linter, and still break a deploy if the diff changes a type, removes a required key, or widens a default.

Free coding models are good at producing plausible YAML. They are less good at knowing your current schema. The result is a patch that looks clean but shifts meaning. The failure arrives later, in a parser, a pod, or a rollout.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The workflow below uses MonkeyCode's free model access for the proposal step and its free server option for the dry-run. The validation logic is plain Python: schema validation plus a schema-aware diff.

Common failure patterns

• Type drift: retries: 3 becomes retries: "3" and breaks strict parsers.
• Required key removal: the patch drops upstream because the model did not see it in the prompt.
• Extra keys: strict schemas with additionalProperties: false reject new keys.
• Silent default changes: an enum or numeric bound changes while the type stays the same.

A valid YAML file is not the same as a safe change.

What to do before merge

1. Keep the last known-good config and a JSON Schema in version control.
2. Ask the model for a patch, not for a full apply command.
3. Write the model output to a separate file.
4. Validate the new file against the schema.
5. Diff old and new for type changes and removed required keys.
6. Dry-run the apply on a scratch server.

The boundary between step 2 and step 3 matters. If the output only lives in the model's reply, you cannot run the gate independently. Writing to a file gives you a point where validation must happen before anything else can read the change.

The gate below implements steps 4 and 5. It is not a benchmark; run it in a throwaway container against your own repository.

#!/usr/bin/env python3
'''Reject schema-breaking config patches before they reach apply.'''
import sys
import yaml
from jsonschema import validate, ValidationError

def type_of(value):
    if isinstance(value, bool):
        return 'boolean'
    if isinstance(value, int):
        return 'integer'
    if isinstance(value, float):
        return 'number'
    if isinstance(value, list):
        return 'array'
    if isinstance(value, dict):
        return 'object'
    return 'string'

def load(path):
    with open(path, 'r', encoding='utf-8') as handle:
        return yaml.safe_load(handle)

def main():
    if len(sys.argv) != 4:
        print('usage: config_gate.py schema.yaml old.yaml new.yaml', file=sys.stderr)
        sys.exit(2)

    schema = load(sys.argv[1])
    old = load(sys.argv[2])
    new = load(sys.argv[3])

    try:
        validate(instance=new, schema=schema)
    except ValidationError as exc:
        print(f'INVALID: {exc.message}', file=sys.stderr)
        sys.exit(1)

    required = set(schema.get('required', []))
    removed = set(old) - set(new)

    for key in sorted(removed):
        if key in required:
            print(f'BREAKING: required key removed: {key}')
            sys.exit(1)

    for key in sorted(set(old) & set(new)):
        if type_of(old[key]) != type_of(new[key]):
            print(f'BREAKING: type change for {key}: {type_of(old[key])} -> {type_of(new[key])}')
            sys.exit(1)

    print('OK: schema valid and no type/required regressions detected')

if __name__ == '__main__':
    main()

The type_of helper collapses booleans, integers, numbers, arrays, objects, and strings to a comparable form. The script exits non-zero on any missing required key or changed type. It does not inspect nested objects deeply; if you need nested diff, extend the recursive comparison before relying on it.

Save that as config_gate.py. Use a schema that is strict enough to reject unknown keys:

# schema.yaml
type: object
additionalProperties: false
required: [listen, upstream]
properties:
  listen: {type: string}
  upstream: {type: string}
  retries: {type: integer, minimum: 1, maximum: 5}

Run it with:

python config_gate.py schema.yaml config.old.yaml config.new.yaml

If the script exits zero, do one more dry-run before you touch a real environment. For Kubernetes, kubectl apply --dry-run=client -f config.new.yaml is a common check. For plain files, copy to a temp path and run the parser or config test command. Do not let the model response call the apply step directly.

What the gate does not catch

• Same-type value drift: port 8080 becomes 8443 with no type change.
• Business invariants that are not encoded in the schema.
• YAML anchors, aliases, and merge keys that yaml.safe_load may not fully model.
• Hallucinated environment values that happen to pass the schema.
• Network egress from the free server during the dry-run.
• Nondeterministic output between calls.

Schema validation checks shape. A diff checks type and required-key regressions. Neither checks intent.

Who should not use this approach

• If the config is static and low-risk, the overhead may be more trouble than the risk.
• If you already run policy-as-code with a full schema registry and mandatory review, this is redundant.
• If the config contains secrets, scrub them before sending anything to a remote model.
• If you need strict reproducibility, pin the model version, schema version, and input context instead of relying only on a free tier.

This article does not assert specific model names, quotas, hardware, or uptime guarantees. The free access and free server option are operator-supplied availability claims; treat them as a proposal tool, not a production dependency.

Start with a read-only dry-run and a throwaway credential. If you use MonkeyCode's free server option, run the gate as a separate service from the model endpoint so a malformed patch cannot skip the check.