Coding with AI agents carries a quiet trap: the illusion of competence.
Tasks close in record time, architecture moves forward, and everything builds cleanly. But at the end of the day, an uncomfortable question remain: Did I actually learn any of this, or was I just acting as a prompt supervisor?
In an intend to prevent AI from turning into a mindless "code vending machine," I designed an integrated learning system where the agent doesn't just write software—it documents my learning journey, maps concepts, and actively tests my comprehension.
Here is how the system works:
The Core Rule: The Vault Stores Comprehension, Not Code
I maintain an Obsidian vault (Engineering/) directly connected to my AI agent sessions via automated hooks and skills.
The foundational rule is strict: the vault never duplicates repository docs or generic boilerplate. It only captures:
- Mental models and architectural analogies.
- Real trade-offs and discarded alternatives.
- Current Architecture Decision Records (ADRs) and their change history.
- Especific markers distinguishing verified facts
[V]from assumptions[A].
System Architecture: 3 Key Layers
Architecture of the learning pipeline: from session classification to milestone exams.
1. Contextual Project Modes (learn, log, off)
Whenever an agent session starts inside any repository, it checks the configured mode:
-
learn: Full pedagogical depth. Generates architecture notes, links concepts, and triggers milestone exams. -
log: Lightweight mode. Only tracks architectural decisions (ADRs) and concept links. -
off: Zero overhead. No notes written to the vault.
2. Background Concept Distillation
Whenever I ask a conceptual question during a session—such as "What is backpressure?" or "Why prefer this concurrency pattern here?"—the agent answers in chat and quietly reports:
Saved "Backpressure" to Concepts.
Concepts start small: a 3–5 line Term (definition, quick example, and the context where it surfaced). As the same pattern appears across different projects, it matures in place into a deeper Concept. Links are embedded organically in the body text, weaving a knowledge web over time.
3. Milestone Exams (Active Recall)
Passive note-taking doesn't build long-term retention. To counter this, whenever a milestone is reached in a learn project, the agent offers an exam:
- 10 to 12 open-ended questions (no multiple choice—multiple choice tests recognition, not recall).
- Grounded strictly in the actual repository code, decisions made, and real bugs encountered during development.
- Covers real code reading, failure scenarios, and change propagation ("If we swap dependency X tomorrow, what could break and why?").
- Supports an oral mode where questions are evaluated through speech and voice transcription.
If I cannot defend and explain why an architectural path was chosen, the task is not considered truly learned.
What Changed in My Daily Routine
- True Retention: The codebase doesn't feel foreign when returning to it two weeks later.
- Cross-Project Synergy: A concept learned while fixing a bug in Rust or Go naturally connects to an architectural decision made months later in a TypeScript API.
- AI as a Socratic Partner: The agent transitions from an autocomplete utility into a technical partner that demands conceptual clarity before and after shipping code.
If you are using AI agents to write software, consider designing your workflow to amplify your own understanding, not replace it.
Do you have a system for retaining what you learn while coding with AI? Let's discuss your setup in the comments below.
Top comments (0)