A migration that looks correct in review can still corrupt data because review sees intent, not edge cases. You read the SQL, it says split the name into first_name and last_name and backfill from the existing column, and you approve it. Then the one-word customer names become empty first names, the three-word names drop their middle names, and the unique constraint on email stays intact only because the bad rows never got tested. The fix is not to stop generating migrations; it is to force every generated migration to prove a handful of invariants against disposable data before it reaches your branch.
I use MonkeyCode's free model access and free server option for this because the check needs to be cheap enough to run on a hunch. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The harness does not depend on that provider, but a free tier removes the excuse not to run it.
Treat the migration as a claim, not a script. A script is a sequence of statements; a claim is something you can falsify. Before you run anything, write down the few properties that must remain true after the migration. For a name-split migration, the claims are:
precondition: every row in users has a non-null name
invariant: after the migration, every original user still exists and no email is duplicated
rollback: the old name values can be reconstructed
The free model's job is not to write the production migration for you. It is to turn that contract into seed rows and assertion queries. That division matters because the model is often too confident about the migration itself, but it is reasonably good at inventing malformed rows when you ask it to attack the contract.
The simplest gate runs in SQLite against an in-memory database. SQLite will not catch PostgreSQL locking or type behavior, but it is a fast way to find logic errors before you pay for a heavier check. The script below is deliberately flawed on purpose, so the assertion should catch it.
import sqlite3
schema = """
CREATE TABLE users (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
email TEXT NOT NULL UNIQUE
);
"""
seed = """
INSERT INTO users (name, email) VALUES
('Ada', 'ada@example.com'),
('Grace Hopper', 'grace@example.com'),
('Augusta Ada King', 'augusta@example.com'),
('', 'blank@example.com'),
(' ', 'spaces@example.com'),
('José Alvarez de Toledo', 'jose@example.com');
"""
migration = """
ALTER TABLE users ADD COLUMN first_name TEXT;
ALTER TABLE users ADD COLUMN last_name TEXT;
UPDATE users SET
first_name = trim(substr(name, 1, instr(name, ' '))),
last_name = trim(substr(name, instr(name, ' ') + 1));
"""
assertions = [
"SELECT id FROM users WHERE first_name IS NULL OR first_name = ''",
"SELECT id FROM users WHERE last_name IS NULL OR last_name = ''",
"SELECT a.id FROM users a JOIN users b ON a.email = b.email AND a.id < b.id",
"SELECT id FROM users WHERE trim(name) != trim(first_name || ' ' || last_name)",
]
con = sqlite3.connect(":memory:")
con.executescript(schema + seed)
try:
for statement in migration.strip().split(";"):
if statement.strip():
con.execute(statement)
failures = []
for query in assertions:
rows = con.execute(query).fetchall()
if rows:
failures.append((query, rows))
con.rollback()
print("OK" if not failures else failures)
except Exception as exc:
con.rollback()
raise
Run that as a plain Python file and the first assertion should return a row for Ada. The original migration used instr(name, ' '), so a one-word name produces an empty first name because there is no space to split on. That is exactly the kind of plausible mistake that survives a quick code review: the SQL looks like it should split names, and it does work for the common two-word case, but it breaks on the oldest and simplest form a name can take.
The free server option matters for the second pass. Once the logic holds up in SQLite, point the same contract at a disposable PostgreSQL instance, because that is where the migration will actually run. The only change is the connection string and the migration dialect; the seed rows and assertions stay the same. This catches issues like a missing IF NOT EXISTS, a column type that coerces differently, or a statement that takes a lock you did not plan for. The point is not to copy production data. It is to make the contract witness a dialect-specific failure before your teammates spend review time on it.
The workflow has clear limits. A migration can pass on six synthetic rows and still fail on a ten-million-row table because the test data does not reproduce distribution, locking, or resource pressure. The model can also miss an invariant you never told it to check, so the contract is only as good as the properties you choose. And if the same model writes both the migration and the assertions, it is grading its own homework; keep the invariant list in your own words and only ask the model to expand it into malformed seeds. Do not use this as a replacement for backups, a human review, or a real migration tool. If you already have a mature framework with your own migration tests, this adds little; it is most useful in the early, exploratory stage before a schema change earns a place in the branch.
The value is not in finding every bug. The value is in finding the plausible bug before it is merged, while the failure is still cheap. A ten-minute disposable run turns a hunch about a migration into evidence you can show someone else. If you already have a free model tier and an idle server slot, run one migration through the gate today.
Top comments (0)