Input Drift Is the New Model Drift: A Free Prompt Audit Pipeline

Nobody edited the prompt. That was the problem.

Last week a teammate pasted a refactor request into the coding assistant. The tool declined politely. It kept quoting an internal rule: never use native arrays. Nobody remembered writing that rule. We checked the template history. One quiet commit had added it days ago. The model was fine. The instructions were rotting.

Model drift gets all the attention. Input drift is the ignored sibling. Your model stays pinned. Your prompt templates quietly accumulate contradictions. This pattern shows up in every DEV thread about AI reviewers: teams measure output quality, but never audit the instructions that produce it.

This article gives you a one-file audit script, a decision table, and a cron job. It runs for free. It catches the next silent rule before it sabotages your work.

The failure mode you cannot see

Prompt templates look stable. They are not. A teammate "improves" one line. A doc gets copied into context. An old example survives a redesign. Each change is harmless alone. Together they create a prompt that fights itself.

Your model has no way to know the rule is wrong. It knows how to follow instructions. It does not know which instruction is current. You need a separate reviewer for the instructions themselves.

A human can do this. Humans get bored by the fourth template. A free model does not get bored, and a cron job does not forget.

A one-file audit script

Save this as prompt_audit.py. It scans a directory of prompt templates for mechanical risk signals. Each warning is a candidate for the model review step.

#!/usr/bin/env python3
"""Scan prompt templates for mechanical risk signals."""
import re
import sys
from pathlib import Path

# Absolute words hide conflicts. Collect them first.
ABSOLUTE_WORDS = [
    "always", "never", "must not",
    "do not", "always use", "never use",
]

# Stale dates and deprecation notes are visible rot.
STALE_PATTERNS = [
    r"20(1[0-9]|2[0-4])[-/.]?[0-9]{1,2}",
    r"deprecated as of",
    r"as of last release",
]


def scan(path: Path) -> list:
    findings = []
    text = path.read_text(errors="ignore")
    for line_no, line in enumerate(text.splitlines(), 1):
        lowered = line.lower()
        for word in ABSOLUTE_WORDS:
            if word in lowered:
                findings.append((line_no, f"absolute word: {word}", line.strip()[:80]))
        for pattern in STALE_PATTERNS:
            if re.search(pattern, line):
                findings.append((line_no, f"stale marker: {pattern}", line.strip()[:80]))
    return findings


if __name__ == "__main__":
    root = Path(sys.argv[1])
    for template in root.rglob("*.md"):
        hits = scan(template)
        if hits:
            print(f"## {template}")
            for line_no, kind, snippet in hits[:10]:
                print(f"- {kind} at line {line_no}: {snippet}")

The script is deliberately dumb. It finds suspicious lines. It does not judge them. The judging happens in the next step.

Send the findings to a free model

Mechanical rules catch obvious rot. They miss contradictions. Two absolute rules can conflict. Only a language model sees that conflict.

This is the model review step. It is pseudocode because SDKs change. The shape of the call matters more than the package name.

# model_review.py (pseudocode)
# Send each suspicious line plus the whole template to a model.
# Ask for a short JSON verdict.

review_prompt = """You audit a prompt template for internal conflict.
Context is provided. Report contradictions, dead rules, and ambiguity.
Output only JSON: {"conflicts": [], "dead_rules": [], "confidence": 0.0}
"""

findings = scan(Path("templates/"))
for file, line_no, kind, snippet in findings:
    response = chat_model(
        model="free-tier-model",        # pick any free model you trust
        messages=[
            {"role": "system", "content": review_prompt},
            {"role": "user", "content": f"{file}:{line_no} -> {snippet}"},
        ],
    )
    print(response)

Use the cheapest free model for this job. The task is small. The output is throwaway. You want a signal, not a platform.

Decide before the model does

Write the decision rules first. Otherwise the audit output becomes a long email nobody reads. This table maps the signal to an action.

Signal	Likely cause	Action	Escalate
Absolute words increased	Someone hardened a rule	Read the diff	No
Two conflicting absolutes	Copy-paste context merge	Delete newer rule	Yes
Stale date or version	Old example survived	Update or remove	No
Model reports dead rule	Rule no longer matches code	File issue	Yes
Confidence below 0.3	Template is ambiguous	Rewrite, do not patch	Yes

Escalation means open an issue with owner and deadline. Low-confidence findings should block new AI work until rewritten.

The default action is never "change the model." The model was fine. The instruction was wrong.

Run it every night on a free server

A nightly cron job turns this script into a real process. You need a place that runs code without your laptop being awake.

This is where MonkeyCode fits. It is an open-source coding assistant, and its free tier includes free models plus a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Check the repository for current quotas before you rely on them; free limits change without ceremony.

A minimal cron line looks like this:

0 3 * * * cd /srv/prompt-audit && python prompt_audit.py templates/ >> audit.log 2>&1

Add the model review step as a second command. Keep both in one shell script. A two-line cron file is easier to debug than a workflow engine.

What the audit cannot catch

The script sees text, not intent. It cannot know why a rule exists. It only knows the rule looks risky.

The model review step adds judgment, but free models can be sycophantic. They agree with the user's framing. They rarely say "delete this rule entirely." Treat the model output as a triage, not a verdict.

The audit also misses structural rot. A template that is too long passes every check. A prompt with five examples passes. Size and clarity are invisible to this pipeline. Add a line count to the script if you need that signal.

Who should skip this approach

Teams with two prompt templates do not need a cron job. A weekly human read is enough. Teams in regulated environments need audit trails a free server cannot provide. And if your team treats every free tier as a production SLA, stop, because this pipeline inherits that risk.

For everyone else, this is a thirty-minute setup with a one-week payoff. The next silent rule will arrive. The question is whether you find it before your assistant starts refusing your own refactors. Clone a free-tier stack, point the script at your templates, and read the first report before you trust the second.