I use AI tools a lot. I'm not going to pretend otherwise. My Whiteboard.ai project, for example, was built with Claude Code under written engineering rules and a plan-then-approve loop, and that's stated plainly in its case study.
But I've had to set rules for myself, because I noticed a specific failure mode: I can ship something I don't understand. That's the real risk of AI tools, and it isn't about quality of output. It's about what happens to me if the tool does all the thinking.
Here are the rules, where AI helps, where it quietly hurts, and one lesson from a different angle: building products on top of models.
The failure mode
Without AI, if I don't understand something, I can't write it. The gap between "I need this" and "I have this" forces learning.
With AI, that gap can vanish. The tool writes the code; it compiles; it passes a quick manual check; I move on. The code is in my repo, but the understanding isn't in my head. Weeks later something breaks and I'm debugging code that's effectively someone else's, in a codebase I'm supposed to own.
So I set rules.
My rules
1. Explain it back
If I can't explain a block of generated code in my own words, I don't commit it. "It works" isn't enough. I need to be able to say why it works and what would break it.
2. I choose the architecture
The assistant fills in; it doesn't decide where logic lives. The decision to put CardBridge's rules in Postgres (Day 2) was mine, with reasons I could defend. That's the kind of decision I don't hand over, because the consequences last the whole life of the project.
3. Read the diff. Always.
Generated changes can touch files I didn't expect. Reading the diff before accepting is the cheapest quality control there is. If a change surprises me, that's a signal to stop and ask why.
4. Verify in the running app
"It compiles" isn't "it works". Type checks and unit tests are necessary, not sufficient. For anything user-facing, I run the actual app and use the feature.
5. Write the rules down
For Whiteboard.ai, I wrote engineering rules in a file and had the assistant plan first, with me approving the plan before code. A written rule survives a long session in a way that a mental intention doesn't.
Where AI genuinely helps me
- Boilerplate and repetitive refactors. Renaming across forty files, adapting a pattern to a new case.
- Rubber-ducking a bug when nobody's awake to ask. Explaining the problem clearly is half the fix, and a responsive duck helps.
- Unfamiliar APIs, where I need a starting example to adapt rather than a blank page.
- Test scaffolding, once I know what to test.
Where it quietly hurts
- It makes the first draft so easy that I skip the thinking. The first design is rarely the best one. When drafting is free, it's tempting to stop at draft one.
- Confident wrong answers. A fluent explanation isn't evidence that it's right. I've learned to treat fluency as zero information about correctness.
- Silent scope creep. Assistants happily "improve" things you didn't ask about.
A lesson from building on top of models
AI coding tools are one side. The other is building products that call models. My Knot.ai project, a spoken interview-practice app, taught me something I now think of as a rule of thumb:
Instruction is not control.
In early testing, I handed the whole interview to an LLM and asked it nicely to follow rules: adjust difficulty by one level, cover three topic areas. Against a simulated candidate, the interviewer model tunnel-visioned on one topic and escalated difficulty freely. It ignored "adjust by one level" and "cover three areas" the moment they competed with its own instincts.
The fix was to split the job. The model does what only a model can: understand an answer, judge it, phrase the next question. Everything countable became a state machine on the server: difficulty is a streak-based ladder clamped to 1–5, topic coverage rotates on a counter. The model supplies judgement and phrasing; the server decides what happens.
That's the same principle as my AI-tool rules, from the opposite direction. Don't rely on a prompt to enforce what code can enforce. And don't rely on an assistant to enforce good judgement that only you can supply.
The test I use
Before I commit anything an AI wrote, I ask:
Could I rebuild this from scratch if the tool vanished tomorrow?
If yes, I'm using it as a power tool. If no, I'm renting understanding, and the rent comes due at the worst time.
What I'm still figuring out
I don't have this solved. A few open questions I wrestle with:
- How much unfamiliar code is "okay to accept" while I'm learning versus building something to ship?
- When does reviewing a diff become a rubber stamp because I'm tired?
- How do I keep my own coding muscles in shape when the tool is always there?
If you have answers, I genuinely want them.
A review checklist for AI-generated changes
Run through this before accepting a diff.
- Scope. Does it touch only files related to the request?
- Surprises. Anything deleted, renamed or reformatted that I didn't ask for?
- Understanding. Can I explain each hunk aloud?
- Edge cases. What happens on empty input, errors, concurrency?
- Security. Any secrets, unsafe string-building, permissive access?
- Dependencies. Did it add a package? Is it needed and trustworthy?
- Tests. Does a test fail without this change and pass with it?
- Running app. Did I actually use the feature?
Prompts that help me understand, not just copy
How you ask changes what you learn.
- "Explain this function line by line as if I'm a second-year student."
- "What are three ways this could fail in production?"
- "Give me two alternative designs and the trade-offs of each. Don't pick one."
- "What would you check before trusting this?"
- "Write the failing test first, then the minimal fix."
Notice the pattern: these ask the tool to expose reasoning, which keeps me in the loop.
A before/after example
Weak request: "Add rate limiting to my API."
Better request: "I have a Vercel function at api/chat.ts. I want to limit each IP to 20 requests per minute. List two approaches that fit a serverless host, explain the trade-offs, and wait for me to choose before writing code."
The second one keeps the architecture decision with me and gives me something to learn from.
The plan-then-approve loop
For bigger tasks, I use a loop with written rules:
- State the goal and constraints.
- The assistant proposes a plan, with no code yet.
- I read it, push back, and approve or change it.
- It implements one step at a time.
- I review each diff before moving on.
Written engineering rules (naming, testing, "don't change files outside the plan") make the loop reliable, because rules written once apply to the whole session.
A mini-exercise
Pick a function you wrote with AI help last month. Without looking at the code, write down what it does, what inputs it can't handle and how you'd test it. Then compare with the code. The gaps you find are exactly the "understanding debt" this post is about.
Signs I'm over-relying on it
- I accept changes without opening the files.
- I can't remember how a feature I "built" works.
- I ask the tool what to build, not how to build it.
- Debugging starts with pasting the error rather than reading it.
If two or more are true, I slow down for a few days and write something from scratch.
How do you draw the line between using AI as a tool and letting it replace understanding?
Top comments (0)