I Let a Free Model Gate Every Migration for 48 Hours. The Vetoes Were the Point.

Would you let a free-tier model veto a production migration before it even reaches the queue? That is the question I asked myself while staring at a backfill that had already burned three developer evenings, so I built a gate that I did not fully trust. For 48 hours, a cron worker asked a free model for a verdict on every pending schema change: run, wait, or investigate. The model never wrote one line of SQL, and that separation turned out to be the entire lesson.

My goal was not to automate migrations, because that would be reckless; my goal was to force a cheap second opinion into a process that usually skips it. The worker lived on MonkeyCode's free server option, the model call went through MonkeyCode's free-model access, and both were made available to me as an operator-invited user for this outreach. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I kept the artifact deliberately small so that anyone could replay the same experiment against their own change queue.

The gate architecture I actually ran

The setup was unglamorous: a Python worker, a database table of pending jobs, and a model endpoint that I treated as a classifier rather than an oracle. Each job went through the same pipeline: summarize the SQL, extract three rule-based signals, ask the model for one word, and then let the rules outrank the model when they disagreed. Here is the decision function that drove the whole experiment:

def decide(job, model_probe, rules):
    summary = summarize(job)
    model_vote = model_probe(summary)   # "run" | "wait" | "investigate"
    rule_vote = rules.score(job)        # 0 = safe, 3 = dangerous

    if rule_vote >= 2:
        return "WAIT", f"rule violation ({rule_vote})"
    if model_vote == "investigate" and rule_vote == 1:
        return "HOLD_FOR_REVIEW", "model wants a human"
    if job.scheduled_at.hour < 6 or job.scheduled_at.hour > 22:
        return "WAIT", "no changes between 22:00 and 06:00"
    return "RUN", f"approved (model={model_vote}, rules={rule_vote})"

The prompt was equally boring, which is why it worked most of the time:

You are a migration gate. Reply with one word only: run, wait, or investigate.
Check for three risks: full-table scans, missing-index filters, and ambiguous WHERE clauses.
If you are not sure, say investigate.

I stored the model vote, the rule score, and the final decision in an audit table because I wanted to separate what the model believed from what actually executed. That audit trail became the most valuable artifact of the 48 hours, even though I initially built it just to cover myself.

If the model says	And the rules say	Gate result
run	safe	run
run	risky	wait, log the mismatch
investigate	anything	hold for review
wait	safe	wait, model veto outranks safety
wait	risky	wait and alert the channel

The veto that was actually right

On the second evening, the model said wait on a migration that my rules had scored as completely safe. The SQL added a column with a default value, which is normally harmless, but the model complained about a missing-index filter on the WHERE clause of a related cleanup query. I checked the table, and the cleanup query used a column without any supporting index, so the backfill would have touched a much larger row range than the planner estimated.

That veto was the whole experiment justifying itself, and it came from a place I did not expect: the model read the migration description, noticed the JOIN, and guessed that the cleanup phase would scan. My rules never looked at the JOIN because I had written them for column adds, not for cascade deletions. The model was not smarter than my rules; it was just checking a different boundary that I had not encoded.

Where the model dragged me into a rabbit hole

Day three gave me the opposite lesson. The model voted investigate on a simple index rename, and the investigation text mentioned log buffer growth and a server that I had never seen in our deployment history. I spent two hours chasing that phantom because I assumed the model had spotted something from the fleet metrics, but it had simply patterns-matched a phrase inside the job description.

The prompt allowed investigate too liberally, and the model interpreted caution as a free pass to invent supporting evidence. I learned to treat model-generated justifications as untrusted commentary, because the vote still had value even when the explanation was nonsense. From that point on, I ignored the reasons and tracked only the one-word vote, which ironically made the gate more reliable.

What I would repeat without hesitation

If I ran this experiment again tomorrow, I would keep four practices exactly the way they were, because they survived contact with reality:

• Store the model vote separately from the rule score, because conflating them makes the audit trail useless.
• Lock the output format to one word, because free-form responses turn into misleading prose that humans start to trust.
• Run the gate in shadow mode for at least a week, logging decisions without executing them, because that gives you a golden set of vetoes before any real risk.
• Never let a model veto become an automatic block, because a stubborn free-model vote sent me into a two-hour investigation that a human could have dismissed in seconds.

The golden set that kept me honest

Before the run, I wrote a small set of synthetic jobs with expected outcomes, which acted as a sanity check whenever the worker changed behavior. It was not a benchmark and it was definitely not a guarantee, but it caught the moment when I accidentally inverted the rule priority and the gate started approving everything with a rule score of three. That regression would have been ugly in production, so I am glad the test suite was ugly enough to fail loudly.

GOLDEN = [
    (job_with_missing_index, "WAIT"),
    (job_with_full_table_scan, "WAIT"),
    (job_safe_at_11am, "RUN"),
    (job_with_ambiguous_where, "HOLD_FOR_REVIEW"),
]

Limitations and who should not use this

This gate is not a migration safety system, and I would not ship it as the final word on any schema change that affects customer data. The free-model access that I used comes with rate limits and variable latency, so the worker filed a few wait votes simply because the request timed out mid-call. The free server option also restarts without warning, which taught me to make the worker idempotent instead of pretending the infra was durable.

You should not copy this pattern if your migration has irreversible consequences, if your rollback plan is empty, or if you want the model to write the actual migration code. The gate is only useful as a cheap, noisy second opinion that sits between the queue and the human reviewer. If you keep a queue of your own pending schema changes, point an offline copy of this gate at it for one weekend and count how often the veto is actually right; that number will tell you more than any prompt tweak ever will.