Free Model Explanations Are Just Another Build Input. So I Gave Them a Replay Cache.

Finley stared at the same C++ build failure for the third time. The template error had not changed. The free model summary above it had. The first run called it a recursive concept constraint. The third run called it an ambiguous partial specialization. Both guesses were plausible. Neither was the build artifact the team needed. The build log was deterministic. The model output was not. The bug was not the model. The bug was treating a nondeterministic external call as if it were part of the build graph.

The team had a free model endpoint and a free server option available through MonkeyCode's outreach. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The endpoint was useful for turning a 400-line template storm into a short first clue. But it should not be re-queried on every retry. It should be captured, versioned, and replayed. This workflow turns a free model response into a reproducible build input. It can be used with any HTTP model endpoint. Remove every MonkeyCode mention and the cache still works.

The failure mode

A CI job reruns when a developer pushes again. The same error appears. The same model request goes out. The answer changes because sampling is not fixed, or the endpoint silently changes its output. The team pays in quota, time, and trust. Even a fast free call is still a hidden network edge in a build step. A build should be repeatable. A summary that wobbles between runs breaks that contract.

The fix is not to ask for temperature zero. Temperature zero is a request, not a guarantee. The fix is a local replay cache. A fingerprint identifies the logical input. The first fresh call becomes the recorded response. Later runs replay it without network.

The replay contract

Each model interaction becomes an append-only JSON Lines event:

{"fingerprint":"a1b2c3d4e5f60718","endpoint":"http://example.invalid/model","version":"contract-1","response_sha256":"ab12...","response":"recursive concept constraint","created_at_utc":"2026-08-14T00:00:00Z"}

The fingerprint is built from three fields only: endpoint, model version, normalized prompt. The normalizer removes volatile text.

sed -E -e 's#/home/[^/]+#/build#g' -e 's#[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9:.]+Z##g' build/error.txt > build/error.normalized.txt

If a build log contains a timestamp, a path, or a machine name, that text must not change the cache key. Otherwise the cache never hits.

The core Python tool has operations for replay, store, and drift. It uses only the standard library.

#!/usr/bin/env python3
import hashlib, json, pathlib, subprocess, sys, time

ROOT = pathlib.Path('model_replays')

def normalize(prompt: str) -> str:
    return '\n'.join(line.rstrip() for line in prompt.splitlines()).strip()

def fingerprint(endpoint: str, version: str, prompt: str) -> str:
    raw = '\0'.join([endpoint, version, normalize(prompt)])
    return hashlib.sha256(raw.encode()).hexdigest()[:16]

def load(fp: str):
    path = ROOT / f'{fp}.jsonl'
    if not path.exists():
        return None
    events = [json.loads(line) for line in path.read_text().splitlines() if line.strip()]
    return events[-1] if events else None

def save(fp: str, event: dict) -> None:
    ROOT.mkdir(parents=True, exist_ok=True)
    with (ROOT / f'{fp}.jsonl').open('a') as f:
        f.write(json.dumps(event) + '\n')

The replay path checks the cache before calling the network. On a miss, it calls the endpoint with a short timeout. If the endpoint fails, it writes a sentinel and exits with code 3. The explanatory step can fail closed while the real build failure remains visible.

def cmd_replay(endpoint, version, prompt_file, out_file, refresh=False):
    prompt = pathlib.Path(prompt_file).read_text()
    fp = fingerprint(endpoint, version, prompt)
    if not refresh:
        cached = load(fp)
        if cached:
            pathlib.Path(out_file).write_text(cached['response'])
            print(f'replay {fp}')
            return 0
    request = {'prompt': prompt, 'format': 'text'}
    proc = subprocess.run(
        ['curl', '-sS', '--max-time', '20', endpoint,
         '-H', 'content-type: application/json',
         '--data', json.dumps(request)],
        text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
    )
    if proc.returncode != 0:
        pathlib.Path(out_file).write_text('model_unavailable\n')
        return 3
    response = proc.stdout
    event = {
        'fingerprint': fp,
        'endpoint': endpoint,
        'version': version,
        'response_sha256': hashlib.sha256(response.encode()).hexdigest(),
        'response': response,
        'created_at_utc': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()),
    }
    save(fp, event)
    pathlib.Path(out_file).write_text(response)
    print(f'fresh {fp}')
    return 0

A Makefile target integrates the wrapper into a C++ project. If the free server option is available, set MODEL_ENDPOINT to that endpoint. The contract stays the same.

MODEL_ENDPOINT ?= http://127.0.0.1:8080/model
MODEL_VERSION ?= contract-1

explain:
    python3 replay_cli.py replay --endpoint "$(MODEL_ENDPOINT)" --version "$(MODEL_VERSION)" --prompt-file build/error.normalized.txt --out-file build/model_summary.txt || exit 3
    test -s build/model_summary.txt

Drift check

The cache guarantees repeatability, but it also freezes the first answer. A wrong first answer will stay wrong. To catch silent drift, run a refresh job on a schedule, not on every CI run.

def cmd_drift(fp: str) -> int:
    path = ROOT / f'{fp}.jsonl'
    events = [json.loads(line) for line in path.read_text().splitlines() if line.strip()]
    seen = {event['response_sha256'] for event in events}
    if len(seen) > 1:
        print(f'drift {fp}: {len(seen)} distinct responses')
        return 1
    print(f'stable {fp}')
    return 0

In a scheduled job, pass --refresh. The wrapper records the new response as another event. The drift check compares previous hashes. If the hash changes, a human reviews the new summary before it becomes the new cached response. If it matches, the endpoint is stable for that logical input.

Situation	Cache behavior	Build result
Same normalized prompt, previous response exists	Replay	Build step stays deterministic
New prompt, endpoint responds with valid output	Store as new event	First fresh snapshot
Endpoint times out or returns invalid output	Write sentinel	Exit 3, keep the original build failure
Forced refresh, same response hash	Append event, mark stable	No review needed
Forced refresh, different response hash	Append event, mark drift	Human review before replacing cached response
No cache and no endpoint	No summary	Skip explanation, never block the real error

Where this does not help

The replay cache is only useful when the prompt space is small and repetitive. Compiler error summaries, fixture generators, and API error explainers fit. Open-ended coding assistance does not. A changing repository pushes different diffs, so fingerprints change and the cache hit rate falls.

Do not use this to hide a degraded endpoint. The cache should make failures visible, not mask them. The sentinel and exit code matter. Every cache hit should be logged with the fingerprint and the original request hash. If an incident happens, the team can inspect model_replays/ as an audit trail.

For the C++ team, the summary stepped out of the build graph. The build itself stayed red. The explanation stayed identical on rerun. That was enough to turn a free model from a surprise into an input. The artifact is small: normalize the prompt, fingerprint it, replay before you call, store the response, and check for drift before you trust the refresh. That order matters.