Opinion: An AI Patch That Can't Undo Itself Isn't Done

#ai #database #testing #devops

Every AI patch is tested for the path it takes, never for the path it leaves behind. CI pipelines apply the patch, run the probes, and call it done, while the undo path stays untested until the moment it is needed. That asymmetry is the most expensive blind spot in AI-assisted development, because the model never saw the rollback requirement in its training context. A disposable server is the cheapest place to prove that a patch can undo itself before it touches real data.

Why AI patches are structurally bad at rolling back

AI models generate forward patches far better than they generate rollbacks. Training data is dominated by pull requests that add features, fix bugs, and refactor code, while rollback commits are rare and usually urgent. A model that has seen ten thousand examples of "add a column" has seen maybe a hundred examples of "drop a column safely". The result is a systematic bias: AI-generated changes tend to be additive, because addition is the pattern the model knows best.

Additive changes look safe at merge time and become dangerous at rollback time. Adding a column is a one-line migration, but dropping it after new data arrived is a data-loss decision. Adding a config key is harmless until the old binary reads a config file that contains an unknown key. The model generates the forward path with confidence, and the rollback path with a mechanical inverse that ignores the data written in between.

The rollback contract has three clauses

Treat the rollback as a contract with three clauses, each of which must be verified separately. Structural convergence means the schema, configuration, and dependency versions match the baseline after the undo runs. Data convergence means the rows, files, and state markers match the baseline, or differ only in ways the old code can read. Behavioral convergence means the system passes the same probes after rollback that it passed before the patch was applied.

Most teams verify only the first clause, because that is what "the rollback script ran without errors" actually proves. The second clause is where AI patches fail, because the model cannot predict what data will be written between the forward apply and the rollback. The third clause is where hidden dependencies surface, because the old code may read the new data format without crashing, and corrupt it silently.

The round-trip workflow

Run the round-trip on a disposable server, not on staging. The point is not to simulate production; the point is to give the patch a chance to fail cheaply. MonkeyCode's free model access can generate the patch, and you can ask it for a rollback draft in the same session. Its free server option gives you a disposable place to run the round-trip. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Step 1: Capture a baseline. Dump the schema and record the row counts for every table. This is the reference state that the rollback must restore.

Step 2: Apply the forward patch. Run the migration, then verify the forward behavior with a probe query. If the forward path fails, the round-trip stops here.

Step 3: Simulate post-patch writes. Insert rows, update records, change configuration — whatever your application would realistically do after the patch ships. This is the step everyone skips, and it is the entire reason the rollback exists.

Step 4: Execute the rollback. Run the rollback script exactly as it would run in an incident.

Step 5: Compare against the baseline. Check structural convergence first, then data convergence. A rollback that restores the schema but loses rows has failed its contract.

The verification script

The script below is a template, not a production tool. It uses SQLite because it is available on any disposable server, and it makes the convergence check explicit.

#!/usr/bin/env bash
# roundtrip_contract.sh — verify an AI patch's undo path on a disposable server
# Usage: ./roundtrip_contract.sh <db_file> <migration.sql> <rollback.sql> <probe.sql> [writes.sql]

set -euo pipefail

DB="${1:?database file required}"
MIGRATION="${2:?migration script required}"
ROLLBACK="${3:?rollback script required}"
PROBE="${4:?probe script required}"
WRITES="${5:-}"
BASELINE="baseline_$(date +%Y%m%d_%H%M%S)"

# Step 1: capture baseline schema and row counts
sqlite3 "$DB" ".schema" > "${BASELINE}.schema"
for table in $(sqlite3 "$DB" "SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%';"); do
  echo "$table $(sqlite3 "$DB" "SELECT count(*) FROM $table;")" >> "${BASELINE}.counts"
done
echo "baseline captured"

# Step 2: apply the forward patch
sqlite3 "$DB" < "$MIGRATION"
echo "forward patch applied"

# Step 3: verify forward behavior
sqlite3 "$DB" < "$PROBE"
echo "forward probe passed"

# Step 4: simulate post-patch writes
if [ -n "$WRITES" ]; then
  sqlite3 "$DB" < "$WRITES"
  echo "post-patch writes simulated"
fi

# Step 5: execute the rollback
sqlite3 "$DB" < "$ROLLBACK"
echo "rollback executed"

# Step 6: compare schema convergence
if diff -u "${BASELINE}.schema" <(sqlite3 "$DB" ".schema") > /dev/null; then
  echo "structural=converged"
else
  echo "structural=diverged"
  diff -u "${BASELINE}.schema" <(sqlite3 "$DB" ".schema") | head -30
  exit 1
fi

# Step 7: compare row counts
for table in $(sqlite3 "$DB" "SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%';"); do
  count=$(sqlite3 "$DB" "SELECT count(*) FROM $table;")
  baseline_count=$(grep "^$table " "${BASELINE}.counts" | awk '{print $2}')
  if [ "$count" != "$baseline_count" ]; then
    echo "data=diverged table=$table baseline=$baseline_count current=$count"
    exit 1
  fi
done
echo "data=converged"
echo "verdict=rollback_contract_holds"

A concrete failure case makes the value obvious. Suppose the migration adds a status column, the writes file inserts a row with status='active', and the rollback drops the column. The script reports data=diverged because the row count differs from the baseline. The fix is not to skip the write simulation; the fix is to make the rollback preserve the data, for example by copying the column to a shadow table before dropping it.

Reading the verdict

Convergence result	Meaning	Action
Structural + data + behavioral	The undo path is honest	Merge, keep the rollback script
Structural only	Rollback restores shape, not content	Redesign the rollback before merge
Neither	The rollback is fiction	Do not merge until the undo path is real
Behavioral failure	Old code cannot read the restored state	Add a compatibility layer or abandon rollback

Limitations and who should skip this

A disposable server cannot reproduce production write patterns, so the simulated writes are only as good as your imagination. The script compares row counts, not row content, so silent data corruption can slip through. Teams with append-only data stores, teams that never roll back, and teams with trivial rebuild-from-source workflows should skip this, because the rollback contract is not where their risk lives.

AI-generated rollbacks deserve the same skepticism as AI-generated patches. The model can produce a rollback script that looks correct and still miss the data written between apply and undo. The round-trip is not a substitute for a human who understands the data lifecycle; it is a cheap way to force that human to think about the undo path before the incident does.

MonkeyCode's free server option is a reasonable place to run this round-trip if you do not have a disposable environment handy. The same free model session that generated the patch can draft the rollback, but only the round-trip proves the contract.

DEV Community