I got tired of LLM drift breaking my apps, so I built an open-source fix

Gary Zavaleta — Fri, 21 Aug 2026 04:06:53 +0000

If you have been building with Large Language Models over the last year, you have probably hit the same wall I did: LLM Drift. The agent reinvents your codebase, renames your files, and apologizes in bullet points. The AI model or coding tool did not get worse. Your context did.

I stopped fighting drift with smarter prompts. I fight it with five small files and a loop. This post shows the whole system.

Direct beats meta

Abstract instructions drift. Concrete instructions do not. Compare:

Meta: "You are an expert Python developer. Write clean, Pythonic, robust code. Follow PEP 8 and best practices. Handle errors gracefully."

Direct: "Use Python 3.12 and uv. Type-hint all public functions. Validate inputs with Pydantic models in src/schemas.py. Copy the repository pattern from src/db/user_repo.py. Name tests tests/test_*.py. Run ruff check . && uv run pytest -q before each commit."

Every direct line is checkable. The code has type hints or it does not. Nothing in "clean and robust" is checkable. After twenty turns, the AI's idea of "clean" is not yours anymore. That gap is the drift.

The files

BRD.md and PRD.md: you fill these from templates. You, not the AI (or ask AI to use the templates based on your initial writing). They force you to know what you want.
generate-agents.md: a prompt. It reads your PRD and generates hierarchical AGENTS.md files: a small root file, detailed sub-folder files. It also generates TASKS.md: milestones as sections, tasks as checkboxes.
content-in/ and content-out/: inputs you write, outputs the AI writes. Every artifact has a fixed place and a fixed name.

The task loop

One task = one checkbox line in TASKS.md = one branch = one PR. If a task is too big for one PR, split the task list, not the PR.

For each task:

1. Plan (plan mode):

Ultrathink the requirement @content-in/req.md. Ask as many questions as you need.
Save the plan in content-out. Use the name YYYYMMDD-TOPIC-PLAN.md.

2. Implement (auto mode):

Ultracode the plan @content-out/YYYYMMDD-TOPIC-PLAN.md.
Create a branch and a PR for this task.

3. Record:

Write an implementation record of all the work you did. Save it in content-out.
Use the name YYYYMMDD-TOPIC-RECORD.md. Include the record in the PR.

4. Review in another model:

Review the implementation in PR #XX. The requirement is @content-in/req.md.
The plan is @content-out/...-PLAN.md. The record is @content-out/...-RECORD.md.
Review the code as well.

Tick the task. Take the next one. The dated plan and record give you a paper trail: requirement → plan → code → record → review. When something drifts, you can see where.

The rules that start arguments

These guidelines can be added to your AGENTS.md or CLAUDE.md as well.

Banned words: "delve", "seamless", "leverage", "robust", "comprehensive", "crucial". If the AI writes them, the AI is padding.
I write prompts like aircraft manuals. ASD-STE100 is the Simplified Technical English standard from aerospace: sentences of 20 words or less, active voice, one instruction per sentence. Aviation solved unambiguous instructions decades ago. Borrow it.
Do not preserve backward compatibility. Delete the old path when you replace it. Dead code is drift fuel.
Never let a model grade its own homework. The review step runs in a different model. Same-model review confirms; cross-model review finds.

Receipts

I ran the framework on itself. The latest improvement went requirement → plan → implementation → record, and the record sits in the PR description: PR #1.

garyzava / avoid-llm-drift

Avoiding LLM Drift When Using AI Coding Assistants

The way we code with AI assistants has evolved with the rise of agentic coding. Direct instructions beat meta (abstract) ones and using multiple lean instructions helps avoid LLM drift (this means when AI stops listening to you).

For example:

Meta: "You are an expert Python developer. Write clean, Pythonic, robust code. Follow PEP 8 and best practices. Handle errors gracefully."
Direct: "Use Python 3.12 and uv. Type-hint all public functions. Validate inputs with Pydantic models in src/schemas.py. Copy the repository pattern from src/db/user_repo.py. Name tests tests/test_*.py. Run ruff check . && uv run pytest -q before each commit."

However, you still need a controlled way to provide context and maintain a shared state across sessions.

Coding agents perform better when instructions explicitly cover:

Problem statement, target users & JTBD (Jobs to Be Done), scope vs non-scope
Acceptance criteria (what “done” means)
…

View on GitHub

Copy the templates. Bend the folder structure to your stack; a Python repo and a JavaScript repo need different folders. If it helps, star it.

What is your rule for keeping agents on track? Tell me the one thing that actually stopped drift for you.

DEV Community: Gary Zavaleta