Fail the Public API First: A 90-Minute Spike on AI Refactors

AI-generated refactors stay cheap to produce in 2026. Broken public exports remain expensive to discover later. A 90-minute ship-or-kill spike can close that gap.

The core move is not more generated code. The core move is a failing contract test. Pair it with a one-page ADR the model must complete. Kill the workflow if either artifact is weak.

The hypothesis

One hypothesis, one clock, one binary outcome. That is the entire spike.

Hypothesis: A free remote coding setup can emit two mergeable artifacts in 90 minutes. First, a contract test that fails when public exports drift. Second, an architecture decision record a human reviewer would sign.

Ship: Both artifacts exist. The test fails on a planted break. The ADR names a real tradeoff.

Kill: The test is tautological. The ADR is filler. The clock expires with missing files.

This spike does not grade prose quality. It grades evidence you could attach to a pull request.

Why this spike exists now

Current AI coding threads keep returning to cheap diffs. Architecture still lags the generated patch. Public interfaces break while unit tests stay green.

Vacuous unit tests were a different failure mode. Dependency drift in AI pull requests was another. This spike targets export drift and missing decisions. Those two failures still survive otherwise pretty test suites.

Do not start from a blank toy app. Start from a small library you already ship. The library needs a public surface you can freeze.

Clock, scope, and kill criteria

Set a visible 90-minute timer before any prompt. Do not negotiate the box later.

In scope

• One package with a documented public API
• One contract test file committed on the branch
• One ADR under docs/adr/
• One planted breaking change used as a control

Out of scope

• Full rewrite of internals
• New product features
• Cross-vendor model bake-offs
• Production load tests
• Token accounting dashboards

Kill immediately if

• The test imports private module paths
• The ADR lists benefits and ignores costs
• The model cannot name a rejected alternative
• Anyone extends the timer "just this once"

A kill still counts as a successful spike. You learned the workflow does not earn a merge.

Where a free coding environment fits

You need a reachable workspace and a model that can edit files. A local IDE is enough. A free hosted option also works if tests can run there.

MonkeyCode is one option for hosting this protocol. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Operator-supplied claims used here are free model access and a free server option. This article does not invent model names, token quotas, hardware specs, or uptime.

Use any environment that can run the test command. The protocol is the artifact. The host is interchangeable.

If the environment cannot run the test command, kill the spike. A generated ADR without a red test is theater.

Artifact 1: freeze the public surface

Label: proposed, unexecuted example. Adapt paths to your package.

Create scripts/freeze-public-api.mjs.

import { writeFileSync } from "node:fs";
import { pathToFileURL } from "node:url";

const entry = process.argv[2] ?? "./src/index.js";
const out = process.argv[3] ?? "./api-freeze.json";

const mod = await import(pathToFileURL(entry).href);
const keys = Object.keys(mod).sort();

const freeze = {
  generatedAt: new Date().toISOString(),
  entry,
  exports: keys,
};

writeFileSync(out, JSON.stringify(freeze, null, 2) + "\n");
console.log(`froze ${keys.length} exports -> ${out}`);

Run it once against current main. Commit the freeze before the model starts editing.

node scripts/freeze-public-api.mjs ./src/index.js ./api-freeze.json
git add api-freeze.json scripts/freeze-public-api.mjs
git commit -m "chore: freeze public exports before ai refactor spike"

The AI branch must not rewrite the freeze to pass. Treat freeze edits as a failed control.

Artifact 2: the contract test that must go red

Label: proposed test. It asserts export names only. It does not pretend to prove runtime behavior.

import { readFileSync } from "node:fs";
import { test } from "node:test";
import assert from "node:assert/strict";
import * as api from "../src/index.js";

const freeze = JSON.parse(
  readFileSync(new URL("../api-freeze.json", import.meta.url), "utf8")
);

test("public export set matches freeze file", () => {
  const actual = Object.keys(api).sort();
  const expected = freeze.exports.slice().sort();
  assert.deepEqual(actual, expected);
});

test("freeze file is not empty", () => {
  assert.ok(freeze.exports.length > 0);
});

Plant a break the model must detect. Use a throwaway rename, not a real feature.

# On a scratch commit, rename one named export.
# Example: parseQuery -> parse_query
npm test -- public-api

Expected result: the first test fails. If it stays green, kill the spike. Your freeze never bound the public surface.

Revert the plant after you see red. Leave the test file in place. That red-to-green control is the point of the hour.

Artifact 3: the ADR skeleton

Do not ask for "a design doc". Ask for this exact skeleton. Empty sections are a kill.

# ADR-00XX: Keep the public export set frozen during AI refactors

- Status: Proposed
- Date: 2026-09-03
- Deciders: [names]
- Time box: 90 minutes

## Context
[What change did the model want to make?]
[Which modules sit behind the public index?]

## Decision
[What will stay public?]
[What may move without a major version?]

## Rejected alternative
[Name one plausible alternative.]
[State the concrete cost of that alternative.]

## Consequences
- Positive: ...
- Negative: ...
- Follow-up issue: ...

## Evidence attached
- Command that produced a failing contract test
- Path to api-freeze.json
- Diff stat: files changed / lines

The rejected alternative section is the load-bearing part. "We could also do nothing" is filler. Kill filler.

90-minute protocol

Numbered steps. Do not reorder them. Stop at minute 90 even if the ADR is mid-sentence.

1. Create a branch spike/api-freeze-adr.
2. Start a timer for 90 minutes.
3. Commit the freeze file from main.
4. Add the contract test. Confirm it passes on main.
5. Plant one export rename. Confirm the test fails.
6. Revert the plant. Leave the test in place.
7. Prompt the model to refactor internals only.
8. Require the ADR skeleton to be completed in-repo.
9. Re-run the contract test after every model edit.
10. At minute 90, fill the decision table. Stop.

Suggested prompt. It is not a magic spell.

Refactor src/ for clarity. Do not change public exports.
Do not edit api-freeze.json.
Keep tests red if exports drift.
Fill docs/adr/adr-00XX-public-api-freeze.md
using the skeleton. Name one rejected alternative
with a concrete cost. Stop at file edits. No extra features.

If the model edits the freeze file, treat it as a failed control. That is a kill, not a nit.

Decision table

Evidence at minute 90	Result	Next action
Test fails on planted break; ADR names a real alternative	Ship	Keep freeze + test on main
Test stays green after planted break	Kill	Fix the freeze harness first
Model rewrote api-freeze.json	Kill	Add a CODEOWNERS rule
ADR has an empty rejected alternative	Kill	Do not merge the refactor
Timer expired with partial files	Kill	Record the gap; do not extend
Test passed, internals improved, ADR signed	Ship	Open a normal human review

Ship means the workflow is allowed on the next real refactor. It does not mean the refactor skips human review.

Commands worth logging

Log these outputs in the PR body. Raw command output beats screenshots.

date -u
git rev-parse --short HEAD
git diff --stat main...HEAD
npm test -- public-api
wc -l docs/adr/adr-00XX-public-api-freeze.md

Optional guard against freeze tampering:

git log --oneline -- api-freeze.json
git diff main -- api-freeze.json

If git diff main -- api-freeze.json is not empty, kill. The freeze is a control, not a suggestion.

Protect the freeze after a ship.

# .github/CODEOWNERS
/api-freeze.json  @api-owners
/src/index.js     @api-owners
/docs/adr/        @api-owners

CI only needs the same test command. Do not add extra linters inside the spike.

# proposed job fragment
- name: Public API freeze
  run: npm test -- public-api

False greens to watch

A green contract test is not automatically evidence. Several cheap cheats show up in AI diffs.

• Re-exporting every internal helper from index.js
• Sorting skipped, so key order hides a removal
• Asserting the freeze file exists, not its contents
• Letting the model regenerate api-freeze.json in the same commit
• Importing ../src/internal.js inside the "public" test

Any one of those cheats is a kill. Do not patch the test to restore a green run. Restore the control instead.

What good looks like without fake metrics

Do not collect token counts for this spike. Token burn was a different question. This spike cares about three binary checks.

1. Did the planted break go red?
2. Did the model leave api-freeze.json untouched?
3. Did the ADR reject a specific alternative?

Three yes answers is a ship. Any no is a kill. No composite score. No dashboard.

Diff stat is context, not a KPI. A 40-file cleanup can still be a kill. A 3-file cleanup can ship.

Reviewer checklist

Use this list during the human pass. It is short on purpose.

• Freeze file matches main except for an intentional version bump
• Contract test failed on the planted rename
• No private imports inside the contract test
• ADR rejected alternative names a concrete cost
• Diff does not expand the public export set by accident
• Follow-up issue exists for behavior-level contract tests

If two items fail, kill. Do not bargain item by item.

Limitations

This protocol does not prove behavioral compatibility. Export names can match while return shapes drift. Add schema tests later if you ship the workflow.

It does not cover HTTP APIs. OpenAPI drift needs a different freeze file. Do not reuse api-freeze.json for REST paths.

It does not evaluate model quality across vendors. One environment is enough for a spike. Bake-offs explode the time box and hide the kill criteria.

It does not replace code review. A signed ADR is still a draft. A human owns the merge.

Free hosted environments can vanish or throttle without notice. Do not build release gates on unpaid capacity. Keep the freeze file and tests in your own git remote.

Generated ADRs often omit operations cost. If production paging is relevant, add that section before you ship. Silence there is another kill signal.

Who should not use this approach

Skip this spike if you have no public API. Internal scripts do not need a freeze file.

Skip it if your package is pre-1.0 and you accept daily breaks. The test will only nag you.

Skip it if reviewers will not read ADRs. Process without readers is waste.

Skip it if you cannot run tests in the same environment as the model. Copy-paste between chat and laptop already failed the control.

Skip it if the goal is marketing copy about AI speed. This protocol is designed to kill weak workflows.

After a ship or a kill

After a ship, promote the freeze test into CI. Protect api-freeze.json with CODEOWNERS. Require an ADR path on refactors that touch src/index.js.

After a kill, keep the notes in the spike branch. The next spike starts from the failed control, not from a new chat window.

If you run the spike, publish the kill criteria beside the PR. That record is the useful part.