DEV Community: Maxi Contieri

AI Coding Tip 028 - Build a Company Brain

Maxi Contieri — Tue, 21 Jul 2026 11:00:00 +0000

One Second Brain doesn't scale past one skull.

TL;DR: Wire a skill into your company's live systems so every answer carries a source and a freshness check.

Common Mistake ❌

As a single contributor, you can build a personal Second Brain, and it works beautifully for that one person.

The company's actual knowledge still sits scattered across Confluence, Slack threads, Jira tickets, CMSs, Wikis, Salesforce notes, and a wiki nobody has touched since the last reorg.

Your AI searches your own vault perfectly, then guesses the moment a question needs an answer that lives in a coworker's head, a stale runbook, or a system you never gave it access to.

Problems Addressed 😔

Company knowledge fragments across four to a dozen SaaS tools: document stores, chat, issue trackers, and business systems, each with its own API and its own silo.
Individual Second Brains compound personally but don't share, so a new hire repeats the exact search a colleague already ran last month.
Stale, obsolete documentation answers confidently and wrong, the same way an obsolete comment lies inside a function, except now hundreds of employees and an AI treat the lie as ground truth.
Every team wires its own one-off integration to its own AI assistant instead of sharing a connector layer, duplicating auth, indexing, and permission logic per tool.
Naive integrations flatten access control when they index everything under one service account, so a cited answer can surface a document the requester was never allowed to open.
Engineers already burn 3 to 10 hours a week hunting for undocumented answers, a form of comprehension debt that works in the same way technical debt does.
Unreviewed content decay costs companies an estimated $12.9 million a year in wasted time and wrong answers.

How to Do It 🛠️

Inventory every system employees actually consult before opening a ticket: wiki (Confluence, Notion, Trello), chat (Slack, Teams), issue tracker (Jira, Linear), code (GitHub, GitLab), CRM (Salesforce, HubSpot), and shared drives (Google Drive, SharePoint).
Connect each source through its own MCP server instead of a custom API integration per tool, the same give-the-AI-direct-access pattern extended with vendor servers like Atlassian's remote MCP server or GitHub's.
Write a skill that declares an explicit persona, such as "You are the company librarian," and lists exactly which MCP tools it may call for which category of question, so it never guesses the source.
Force the skill to answer only with a citation attached: source URL, system name, and last-modified date on every fact it returns, never a paraphrase without the reasoning behind it.
Pass the requester's own identity token through every connector instead of a shared service account, so retrieval respects the exact permission boundary the source system already enforces.
Add a freshness gate that drops or downweights any document past its review-by date: an outdated Confluence page lies the same way an old comment does, and it deserves the same don't-trust-it treatment.
Log every wrong or unanswered query as a new entry in the skill's pitfalls file, so a human fills the missing connector or fixes the stale source instead of the AI silently guessing again next time.
Re-run the skill weekly against a fixed set of golden questions and diff the answers against last week's run, so a silent schema change in one connector doesn't rot the whole Company Brain unnoticed.
Turn meetings into text too, since the skill can only reason over what it can read: run a transcription bot on the calls that matter and feed the transcript through the same connector pipeline as everything else.

Benefits 🎯

Single entry point: Employees ask one skill instead of hopping across ten browser tabs looking for the same fact.
Attributable answers: Every response carries a citation, which makes review as fast as reviewing a diff instead of trusting a black box.
Access control travels with the query: The retrieval layer checks permissions against the requester's own identity, instead of flattening them behind a shared index.
Staleness becomes visible: The freshness gate flags a document past its review date instead of trusting it forever, closing the same gap an obsolete comment leaves open inside a function.
Faster onboarding: New hires ask the Company Brain instead of interrupting five different colleagues with the same question.
Compounding knowledge: Company-wide knowledge compounds the same way a personal Second Brain compounds, except now every employee's queries feed it.
Visible gaps: An unanswered query becomes a concrete signal that a runbook is missing, instead of silently costing someone an afternoon.
No more knowledge gurus: The skill breaks the informal silos and single points of knowledge teams route around, since anyone can query the same live sources instead of waiting on the one coworker who remembers.

Context 🧠

A Second Brain is one person's plain Markdown files, disciplined by Tiago Forte's CODE framework and Niklas Luhmann's Zettelkasten, under that person's direct control.

A Company Brain stretches that same idea across a hundred people and a dozen systems nobody individually owns.

The failure mode isn't that people are disorganized.

It's that human memory doesn't scale past the one person who holds it.

That's exactly why Glean built its product around that name: a permissions-aware index over 100+ tools that lets teams ask questions, find experts, and trigger workflows from one surface.

A knowledge silo forms the moment only one system, or one person, holds the answer to a recurring question, and that person becomes a bus factor risk: the team's only path to the answer, until they leave.

A Company Brain breaks both silos and gurus the same way: it distributes the answer across every connector it can query, so the response no longer depends on which system or which coworker happens to be reachable that day.

The company-librarian persona from earlier makes that shift concrete.

A real librarian never invents a shelf location: she looks it up or tells you she doesn't have it yet.

That's the same discipline the persona is supposed to enforce on every answer.

A Company Brain isn't a wiki, and it isn't a CRM.

A wiki decays the moment nobody feels responsible for updating it, and a CRM records what a customer told the company, not what an engineer needs when the customer's real objection never made it into a structured field.

Glean reports its retrieval preferred roughly twice as often as a general chatbot's company knowledge, which tracks with what Atolio and Unstructured describe as the core problem: document stores, chat tools, engineering systems, and business systems each speak a different schema, and someone has to normalize all four into one retrieval stream before an AI can answer honestly.

MCP is the plumbing that makes this tractable without a bespoke integration per assistant: each server acts as a gatekeeper to its own system, and the protocol's authorization layer lets a client present the requester's own bearer token so the connector enforces the exact same access policy the source system already has.

A knowledge base, in the sense most RAG products ship, is a copy: a pipeline chunks and embeds your documents into a vector database, and a query searches that copy by similarity, not the source system itself.

That copy only stays accurate if something re-crawls and re-embeds it on a schedule, and the gap between a source edit and the next reindex run is exactly the window where a confident, wrong answer slips through, even inside Glean's own permissions-aware index.

The skill from this tip skips that copy entirely: instead of searching a pre-built index, it calls each MCP connector live at request time, so one question can query Confluence, Salesforce, and GitHub in the same run and return their current state, not a snapshot from last week's crawl.

That's the real difference between a knowledge base and this Company Brain: a knowledge base is one indexed store you search, while this skill is a router that asks several live systems the same question and merges what comes back, with nothing to fall behind because nothing gets cached in between.

Nobody maintains what can't be executed, and that rule applies just as much to a Confluence page as to a // TODO comment nobody deleted.

Stale context produces the same practical outcome as hallucination, a confident, fluent, wrong answer, which is why teams now treat freshness fields and stale-source suppression as first-class retrieval concerns, not an afterthought.

A decision nobody wrote down costs the same inside a codebase as inside a company: the missing rationale lives in someone's head until the day that person leaves.

A Company Brain doesn't fix either smell by itself, but it turns both into a retrieval gap you can see and log instead of tribal knowledge you can only rediscover by accident.

A knowledge base is never finished: content decay is the default failure mode of a wiki at organizational scale, and it decays faster than most teams expect once a product ships a few releases past the last edit.

The fix isn't a bigger vault, it's governance: every article gets a named owner, and review cadence scales with risk the same way a harness enforces standards by severity, so a billing or permissions page gets reviewed after every release while an evergreen explainer gets reviewed twice a year.

Docs-as-code treats documentation like source code and pushes that governance further upstream: version-controlled, peer-reviewed through the same pull request your feature ships in, so the page can't drift out of sync with the system it describes.

The 2026 trend goes one step further with living documentation wired directly into work-management tools, so closing a ticket auto-scaffolds the release note instead of leaving a human to remember it later.

A Company Brain is downstream of all of this: the fresher the source, the less the retrieval skill has to compensate for a Confluence page or a runbook nobody re-read since it was written.

Prompt Reference 📝

Bad Prompt 🚫

Look through company notes and tell me our current refund policy

for enterprise customers. I don't have the Confluence page

handy, so just use whatever you already know.

Good Prompt 👉

Use the company-brain skill. Query Confluence, Salesforce,

and Slack through their MCP connectors for our current

refund policy for enterprise customers. 

Cite the source URL and its last-modified date for every fact, 

and skip any document past its review-by date.

Considerations ⚠️

Don't let the skill become shadow IT that bypasses your security team's review of each connector.

Every new MCP server is a new access path into a production system.

Redact secrets and PII before indexing anything.

Scope each connector to public channels by default, and tell employees explicitly before you point a connector at anything more private, like email or call recordings.

Treat every connector's token scope as narrower than convenient, never a company-wide service account.

A Company Brain still needs a human owner per source responsible for triaging staleness flags, the same way a codebase needs someone to act on a linter's complaint instead of muting it.

Start with the two or three sources people ask about most, prove the citation and freshness gate work, then expand the connector list.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

Each connector breaks independently when its underlying SaaS API changes, so a Company Brain carries the same maintenance budget as any other production integration.

Running a citation and freshness check on every query costs tokens and latency, which shows up fast at company-wide query volume.

Querying every connector live is slower per question than a single vector search against a pre-built index, since a live approach trades index-refresh lag for round-trip latency to each source system.

A misconfigured freshness gate lets the skill amplify a wrong source faster than a human ever would, so widen its scope only after your golden-question set proves it holds.

When a connector's scope quietly excludes a topic, the skill doesn't say it doesn't know.

It answers anyway, confident and wrong, unless the persona is explicitly told to admit the gap instead of guessing.

Tags 🏷️

Knowledge Management

Level 🔋

[X] Intermediate

Related Tips 🔗

Maxi Contieri

Mar 10

AI Coding Tip 010 - Access All Your Code

#webdev #ai #programming #beginners

4 min read

Maxi Contieri

May 12

AI Coding Tip 019 - Tell the AI Why, Not Just What

#ai #coding #llm #productivity

4 min read

Maxi Contieri

May 19

AI Coding Tip 020 - Create a Second Brain

#ai #webdev #productivity #programming

8 min read

Maxi Contieri

Jun 16

AI Coding Tip 024 - Force a Criteria Check Before the Task Ends

#ai #webdev #programming #productivity

6 min read

Maxi Contieri

Jun 23

AI Coding Tip 025 - Pair Every Skill With a Pitfalls File

#ai #webdev #programming #tutorial

5 min read

Maxi Contieri

Jul 7

AI Coding Tip 026 - Assign a Persona to Every Skill Definition

#ai #webdev #programming #beginners

5 min read

Conclusion 🏁

Your Second Brain stops at your own skull.

Wire a skill into your company's MCP-connected systems, force every answer to carry a citation and a freshness check, and the same compounding memory that helps one developer starts helping everyone who asks it a question. 🧠

More Information ℹ️

How to Create a Company "Brain" with Glean - phData

Unified Index for Enterprise AI - Glean

Types of Data Sources for Enterprise RAG - Atolio

Enterprise RAG: Why Connectors Matter in Production Systems - Unstructured

Introducing the Model Context Protocol - Anthropic

Introducing Atlassian's Remote MCP Server

Context Freshness: Keeping AI Agent Context Current - Atlan

Why Documentation Debt Will Be the New Technical Debt in 2026

Knowledge Base Content Audit Guide

Knowledge Base Governance Framework

What is Docs as Code? - Kong

The State of Software Documentation 2026 - FastDoc

LLM Wiki vs RAG: The Karpathy Concept and Enterprise Reality - Atlan

What Is Agentic RAG? - Weaviate

RAG Is Dead, Long Live Agentic Retrieval - LlamaIndex

Bus Factor - Wikipedia

Also Known As 🎭

Organizational-Brain
Enterprise-Knowledge-Graph
Unified-Company-Search
Cross-System-RAG

Tools 🧰

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

2 min read

AI Coding Tip 027 - Force Code Standards

Maxi Contieri — Tue, 14 Jul 2026 11:00:00 +0000

Style errors double when nobody enforces them.

TL;DR: Wire your standards into hooks, skills, and a judge, so the harness blocks violations before a human opens the diff.

Common Mistake ❌

You paste your coding standards into AGENTS.md and trust the AI to remember them.

Then a human reviewer manually checks naming, indentation, and spacing on every pull request.

Nobody wires a hook, a skill, or a judge that blocks the diff before a human ever opens it.

The standards live in prose, and prose is optional.

Problems Addressed 😔

A CodeRabbit analysis of 470 pull requests found AI-generated code carries roughly 1.7x more defects than human-only code, and nearly 2x more naming and style consistency errors
Human reviewers burn attention on indentation and casing instead of design, feeding the same broken windows that erode a codebase over time
Enforcement depends on someone remembering to check, so it drifts the moment nobody's watching
The AI violates a rule it read in prose, because prose isn't a gate, only a suggestion
Two human reviewers catch two different subsets of the same code without standards

How to Do It 🛠️

Turn each standard into a machine-checkable rule instead of a paragraph of prose.
Wire a pre-commit or pre-merge hook that runs the linter automatically on every change.
Add a validator skill that reads the rule set fresh and re-checks the whole diff, not just what the AI remembers.
Route ambiguous rules, like naming intent, to a large language model (LLM) judge instead of a human reviewer.
Block the commit, the merge, or the session end until every gate reports zero violations.
Log every violation the judge catches as a new rule, so the harness never misses it again.

Benefits 🎯

Consistent gate: Every diff hits the same rule set, so two reviewers never catch two different subsets of the same violation.
Faster reviews: Humans spend their attention on design and correctness, not indentation or casing.
No memory decay: A skill reads the standards file fresh every session instead of trusting what the AI claims to remember.
Judge for nuance: An LLM judge catches the semantic violations a regular expression can't parse, like a misleading name.
Compounding rules: Every violation the judge finds becomes a new check, so the harness gets stricter over time.
Measurable drop: You can track the naming and style defect rate over time and watch it fall toward zero.

Context 🧠

Standards enforcement isn't new.

Stephen C. Johnson wrote the first lint in 1978 to catch mistakes in C code nobody wanted to check by eye.

Checkstyle, PMD, and ESLint followed, each one refusing to trust a human to notice a mixed indentation.

SonarQube added a server that gates a whole pipeline, not just a single file.

Mago does the same for PHP now, running a linter, a formatter, and a static analyzer in one Rust binary fast enough to run on every keystroke.

None of these tools ever asked a human to vote on a tab versus a space or a mixedCase versus snake_case name.

AI coding didn't remove that need.

It multiplied it.

A CodeRabbit analysis of 470 pull requests found AI-generated code carries about 1.7x more defects overall than human-only code.

Naming and style consistency errors came in at nearly 2x, the exact class of mistake a linter was built to catch decades ago.

A model can now write in any human language, misspell an identifier, or reorder parameters inconsistently across two files it never compared side by side.

A linter still catches the syntactic version of these mistakes.

The semantic version, like a name that lies about its role, needs a judge that reads intent, not just tokens.

That's where an LLM-as-judge step fits: a second AI pass, wired into the same harness that runs the linter, checking the rules no regular expression can express.

Skills are the natural home for that judge.

A skill reads the standards file fresh every time, instead of trusting a memory that decays across sessions.

Pair the judge with a criteria check before the task ends, so the AI can't say a task is done until the standards gate reports zero violations.

A gate that blocks the diff is just another way to force the AI to obey you, not by asking nicely, but by refusing to let a violation through.

This doesn't replace reviewing every line before commit.

It removes the mechanical part of that review, so a human is free to judge design instead of counting spaces.

When the judge catches a new violation, log it the same way you'd log a pitfall, so the harness never makes that mistake twice.

Prompt Reference 📝

Bad Prompt 🚫

Please follow our coding standards for this feature.
Use consistent naming and formatting like the rest of the codebase.
I'll check it during code review before we merge.

Good prompt 👉

Before you say this task is done, run this standards checklist:

- [ ] Run the linter on every changed file, fix every violation.

- [ ] Run the formatter, don't hand-format a single line.

- [ ] Invoke the code-standards-validator skill on the full diff.

- [ ] Check every identifier: no abbreviations, no misleading names.

- [ ] Check indentation matches the project config, no mixed tabs.

- [ ] Check casing: one convention, no mixedCase next to snake_case.

- [ ] Check parameter order against other functions in the file.

- [ ] Check spelling in every identifier, comment, and string.

- [ ] Judge comment quality: flag dead comments and restated code.

- [ ] Judge naming intent: does each name say what it does?

- [ ] Confirm no file mixes languages in identifiers or comments.

- [ ] Log any new violation type as a rule for the next run.

- [ ] Don't report the task done until every box above is checked.

Show me the completed checklist, not just the final code.

Considerations ⚠️

A linter still beats an LLM judge on speed and cost for anything syntactic.

Reserve the judge for rules that need intent, not tokens.

A gate that blocks too aggressively gets bypassed with --no-verify, which defeats the whole point.

Review the judge's false positives the same way you'd review a flaky linter rule.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

An LLM judge costs tokens and time on every gate, so it doesn't replace a linter.

It complements one.

A judge can disagree with itself across two runs on borderline style calls, so keep the deterministic rules in the linter and reserve the judge for what's genuinely ambiguous.

Tags 🏷️

Knowledge Management

Level 🔋

[X] Intermediate

Related Tips 🔗

Maxi Contieri

Jan 27

AI Coding Tip 004 - Use Modular Skills

#ai #webdev #programming #beginners

4 min read

Maxi Contieri

Feb 13

AI Coding Tip 006 - Review Every Line Before Commit

#ai #programming #development #softwaredevelopment

7 min read

Maxi Contieri

Mar 18

AI Coding Tip 011 - Initialize Agents.md

#ai #programming

3 min read

Maxi Contieri

Apr 14

AI Coding Tip 015 - Force the AI to Obey You

#ai #webdev #programming #productivity

5 min read

Maxi Contieri

Apr 21

AI Coding Tip 016 - Feed Your PR Lessons into the AI Brain

#ai #programming

6 min read

Maxi Contieri

May 12

AI Coding Tip 019 - Tell the AI Why, Not Just What

#ai #coding #llm #productivity

4 min read

Maxi Contieri

Jun 2

AI Coding Tip 022 - Give AI a Harness to Work With

#ai #webdev #beginners

6 min read

Maxi Contieri

Jun 16

AI Coding Tip 024 - Force a Criteria Check Before the Task Ends

#ai #webdev #programming #productivity

6 min read

Maxi Contieri

Jun 23

AI Coding Tip 025 - Pair Every Skill With a Pitfalls File

#ai #webdev #programming #tutorial

5 min read

Conclusion 🏁

A linter never asked permission to reject bad code.

Neither should your harness.

Wire the standards into hooks, skills, and a judge, so the diff gets rejected before a human ever has to say so.

More Information ℹ️

Lint (software) - Wikipedia

State of AI vs Human Code Generation Report - CodeRabbit

When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs

Awesome Static Analysis - curated list of linters and code quality tools

Also Known As 🎭

Harness-Enforced-Standards
Machine-Judged-Style
Standards-as-Code
Linter-Native-Review

Tools 🧰

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

2 min read

AI Coding Tip 026 - Assign a Persona to Every Skill Definition

Maxi Contieri — Tue, 07 Jul 2026 11:00:00 +0000

Know who speaks before the skill runs

TL;DR: Always define a clear role at the top of every skill file so you know whose perspective drives the execution.

Common Mistake ❌

You write a skill full of rules but assign no role.

The AI starts executing without knowing if it's a junior developer, a seasoned architect, or a QA engineer.

You get responses that feel generic, lack authority, or shift in perspective across runs.

Problems Addressed 😔

The AI picks a random voice, so outputs vary unpredictably between sessions.
You can't audit the skill because you don't know whose judgment it applies.
The AI mixes tones and expertise levels inside a single execution.
Skill chaining breaks because each skill assumes a different implicit persona.
You lose accountability: nobody knows who signed off on the output.

How to Do It 🛠️

Open your skill file and add a role declaration as the very first instruction.
Write "You are a [role] with expertise in [domain]" before any other rule.
Add one or two sentences describing the role's constraints and responsibilities.
Keep the persona consistent through every instruction that follows in the file.
When you chain skills, verify each one declares its own persona explicitly.

Benefits 🎯

Consistent voice: The AI executes from the same expertise level every run, so output is predictable.
Auditable output: You know whose perspective generated the result, which makes reviews faster.
Better calibration: An AI that knows it's a senior reviewer asks harder questions than one without a role.
Safe chaining: When you chain skills, each one speaks from a declared identity instead of guessing.
Faster debugging: When a skill gives a wrong answer, you know whose lens to question.

Context 🧠

A skill without a persona is a command without a commander.

You can read every instruction in the file and still not know who says what or why.

When you declare a role, you give the AI a stable frame of reference.

The AI stops guessing and starts executing from a specific vantage point.

This also helps you design the skill: if you know the AI is "a strict code reviewer," you know what rules to include and which to leave out.

When you ask for the analyst, not the analysis, you're already applying this idea at the prompt level.

A skill takes it one step further: you bake the role into the definition so you don't have to repeat it every time.

You can also pair every skill with a pitfalls file to define what the persona should never do.

Prompt Reference 📝

Bad Prompt 🚫

---
name: technosignature-analyzer
version: 1.0.0
description: "|"
    Analyzes signals from radio telescope arrays.
    Reports unusual frequency patterns as candidates.

allowed-tools:
- ReadTelescope
- SendAlarm
---

# Technosignature Analyzer

Analyze signals from the telescope array.

Check for unusual frequency patterns.
Cross-reference with the Hipparcos catalog.
Flag any readings that deviate from baseline.
Report findings with confidence levels.

Good Prompt 👉

---
name: technosignature-analyzer
version: 1.0.0
description: |
    Detects technosignatures in telescope data and classifies
    each candidate signal with a confidence percentage.
    Rejects signals explained by known natural phenomena.

allowed-tools:
- ReadTelescope
- SendAlarm
---

# Technosignature Analyzer

You are a senior astrophysicist with 20 years of SETI experience.

You worked at the Allen Telescope Array and the Parkes Observatory.

You hold a PhD in Radio Astronomy with 40+ publications.

You distinguish RFI from natural astrophysical signals.

You separately flag artificial sources.

You apply the scientific method.

Form a hypothesis, test it, and document it.

You apply Six Sigma rigor to rule out false positives.

You don't report candidates below a 5-sigma confidence threshold.

You always cross-check three independent baselines before escalating.

Identify narrowband signals inconsistent with natural sources.

Flag laser pulses or structured optical emissions.

Compare power ratios against stellar baselines.

Classify each candidate with a confidence percentage.

Reject signals explained by known natural phenomena.

Considerations ⚠️

Keep the persona declaration short: one to three sentences maximum.

A long persona description adds noise and dilutes the actual skill rules.

Don't invent fictional personas like "You are a wizard who codes."

Use real professional roles.

The AI performs best when the persona matches the domain of the skill.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

The AI doesn't enforce the persona you assign; it adopts it as context.

If your instructions contradict the persona, the AI may blend both and produce inconsistent output.

Tags 🏷️

Knowledge Management

Level 🔋

[X] Beginner

Related Tips 🔗

Maxi Contieri

Jan 27

AI Coding Tip 004 - Use Modular Skills

#ai #webdev #programming #beginners

4 min read

Maxi Contieri

Feb 13

AI Coding Tip 006 - Review Every Line Before Commit

#ai #programming #development #softwaredevelopment

7 min read

Maxi Contieri

Mar 18

AI Coding Tip 011 - Initialize Agents.md

#ai #programming

3 min read

Maxi Contieri

Apr 28

AI Coding Tip 017 - Ask for the Analyst, Not the Analysis

#ai #webdev #programming #tutorial

5 min read

Maxi Contieri

May 12

AI Coding Tip 019 - Tell the AI Why, Not Just What

#ai #coding #llm #productivity

4 min read

Maxi Contieri

Jun 23

AI Coding Tip 025 - Pair Every Skill With a Pitfalls File

#ai #webdev #programming #tutorial

5 min read

Conclusion 🏁

A skill without a persona is a command without a commander.

You always understand the output better when you know who produced it.

Assign a role first.

Every time.

More Information ℹ️

Claude Code Skills Documentation

System Prompts and Personas

Also Known As 🎭

Role-First Skill Design
Persona-Anchored Prompts
Identity-Driven Skill Files

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

2 min read

The Dirty Secret Behind Loop Engineering

Maxi Contieri — Tue, 30 Jun 2026 12:00:00 +0000

Everyone is talking about Loop Engineering. Apparently, you don't need to program anymore.

TL;DR: Loop Engineering is the hottest AI workflow pattern of 2026. But it hides a dirty secret.

The Tweet That Started It All

// Detect dark theme var iframe = document.getElementById('tweet-2063697162748260627-249'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2063697162748260627&theme=dark" }

In June 2026, Addy Osmani and the PostHog team published their takes on the same idea.

Instead of prompting an AI agent manually, you build the system that prompts the agent for you.

The metaprompt idea has a fancy name now. Loop Engineering.

Loop engineering is replacing yourself as the person who prompts the agent. You design the system that does it instead.
Addy Osmani

PostHog ran it in production. The result: an 11% performance improvement and a 3-year-old defect fixed in the query engine, hands-off.

The internet got excited. Again. Rightly so.

But there's a dirty secret hiding behind the vocabulary.

What a Loop Actually Is

A functional loop has four parts:

Goal: a clearly scoped target for the agent
Context: tools, data, errors, and memory fed into each cycle
Evaluation: the mechanism that decides whether the loop should continue or stop (a.k.a. the exit criteria)
Agent: the executor, from a simple /goal command to a full multi-agent harness

The evaluation component is the key. Without it, you don't have a loop. You have an infinite runaway process. Good luck with your token bill!

You also need a harness: the scaffolding that contains the agent, enforces your rules, and gives the loop a safe boundary to operate within.

Start With the Smallest Possible Spec

Here's where Loop Engineering gets interesting, and where most people get it wrong.

If you write an enormous spec covering all possible cases before the loop runs once, you aren't doing Loop Engineering. You're doing waterfall with extra steps.

Think about what you want to verify. Not the whole system. One behavior.

Let's build a FIFA World Cup 2026 group standings simulator using Loop Engineering.

Germany, Ivory Coast, Ecuador, and Curaçao in Group E. Three rounds of matches. The top two advance, and the best third-place teams also qualify.

What's the smallest possible spec?

A team that wins a match gets 3 points.

That's it. Not the whole group. Not the knockout bracket. One rule about one match.

This is the Spec-Driven approach: you define intent before implementation, but you keep the scope surgical.

Loop Iteration 1: Write a Spec That Fails

Here's your first loop cycle. You define the evaluation condition before any implementation exists.

def test_win_gives_three_points():
    germany = Team("Germany 🇩🇪")
    curacao = Team("Curaçao 🇨🇼")

    match = Match(germany, curacao, home_goals=7, away_goals=1)
    standings = GroupStandings()
    standings.record(match)

    assert standings.points_for(germany) == 3
    assert standings.points_for(curacao) == 0

Run this. It fails. Team doesn't exist. Match doesn't exist. GroupStandings doesn't exist.

(Germany beat Curaçao 7-1 on June 14, 2026. The spec matches reality.)

The loop condition is red 🔴.

This is the signal the loop needs. The evaluation says: not done yet. Keep running until you achieve the /goal.

Loop Iteration 2: Make the Evaluation Pass

Now you give the agent the minimal implementation to make the loop exit:

class Team:
    def __init__(self, name):
        self.name = name

class Match:
    def __init__(self, home, away, home_goals, away_goals):
        self.home = home
        self.away = away
        self.home_goals = home_goals
        self.away_goals = away_goals

class GroupStandings:
    def __init__(self):
        self._points = {}

    def record(self, match):
        if match.home_goals > match.away_goals:
            self._points[match.home] = 
                self._points.get(match.home, 0) + 3
            self._points[match.away] = 
                self._points.get(match.away, 0)
        elif match.away_goals > match.home_goals:
            self._points[match.away] = 
                self._points.get(match.away, 0) + 3
            self._points[match.home] = 
                self._points.get(match.home, 0)

    def points_for(self, team):
        return self._points.get(team, 0)

Run the spec. Green 🟢. Loop exits.

Not because you modeled every rule. Because you satisfied the single condition the loop was checking.

Loop Iteration 3: Expand the Spec, Red 🔴 Again

The loop restarts with a new goal:

def test_draw_gives_one_point_each():
    ecuador = Team("Ecuador 🇪🇨")
    curacao = Team("Curaçao 🇨🇼")

    match = Match(ecuador, curacao, home_goals=0, away_goals=0)
    standings = GroupStandings()
    standings.record(match)

    assert standings.points_for(ecuador) == 1
    assert standings.points_for(curacao) == 1

Red 🔴. The record method doesn't handle draws.

(Ecuador drew 0-0 with Curaçao on June 20. Again, the spec matches reality.)

The evaluation fails. Loop continues. Add the draw case. Green 🟢. Loop exits.

Growing the Loop: Group Stage Completion

Cycle by cycle, the spec expands:

Iteration 4: Multiple matches accumulate points correctly across all three rounds
Iteration 5: Teams with equal points get ranked by goal difference
Iteration 6: Ties in goal difference break by goals scored
Iteration 7: Top 2 teams advance to the knockout stage

Each iteration follows the same pattern: write the evaluation condition first, run it (it fails), implement the minimum to pass, run again (green 🟢), move to the next cycle.

After 7 iterations, Group E final standings:

Group E - Final Standings
1. Germany       6 pts  GD: +6  GF: 10
2. Ivory Coast   6 pts  GD: +2  GF: 4
3. Ecuador       4 pts  GD:  0  GF: 2
4. Curaçao       1 pt   GD: -8  GF: 1

Growing the Loop: Knockout Brackets

The same loop discipline applies to bracket generation.

def test_group_winner_faces_different_group_runner_up():
    bracket = KnockoutBracket(completed_group_results)

    round_of_32 = bracket.round_of_32()

    assert round_of_32[0].home == group_e_standings.first_place()
    assert round_of_32[0].away == group_f_standings.second_place()

Red 🔴 first. Then green 🟢. Then the next spec.

The loop doesn't know the full bracket before it starts. It discovers the bracket one evaluation at a time.

What Keeps the Loop Safe: The Harness

None of this works without a structure that:

Runs the evaluation on every cycle
Stops the agent when the condition passes
Prevents the agent from moving to the next spec until the current one is green 🟢
Stores state between cycles so the next spec knows what already passed
Forces the minimum solution that makes the evaluation pass, nothing more.
Over-engineering is structurally impossible when the loop stops the moment the spec is green 🟢

That is the harness. The harness is what separates Loop Engineering from running Claude in a while True loop and hoping for the best.

Codex and Claude Code now ship with built-in loop infrastructure: /goal, /loop, isolation: worktree, and sub-agents for separate verification.

The harness is no longer something you build from scratch.

The agent that verifies runs in a clean sub-agent with no memory of what the implementer did.

It is an independent inspector seeking Judgment Day moments.

It can't grade its own work because it never saw the work being done.

This is the same reason you don't ask a developer to review their own pull request.

The Numbers That Made Everyone Pay Attention

Why is this getting attention now and not five years ago?

Because the evaluation step (the part where the loop decides whether to continue) used to require a human. Now it doesn't.

Opus 4.8 completes 50% of tasks requiring 12 hours of work, at 6x the speed of its predecessor one year ago
Stripe migrated an entire codebase in one day that would have taken a team two months manually
PostHog's autoresearcher loop delivered 11% performance gains and fixed a three-year-old defect, unsupervised

When models were weaker, the loop needed you to interpret the evaluation output. Now the evaluation can be the test suite itself, and the agent reads it directly.

The Dirty Secret

You've been reading about Test-Driven Development.

The spec is the test.

The evaluation is the test runner.

The loop is the red 🔴-green 🟢-refactor 🔵 cycle.

The goal is the failing assertion.

Loop exits when evaluation passes means the test is green 🟢.

Kent Beck described this in 2003.

Ward Cunningham was doing it before that.

There's even a structured guide for choosing which goal to tackle next: the ZOMBIES framework. Zero, One, Many, Boundary, Interface, Exceptional, Simple. That is your loop iteration order.

What changed isn't the technique. What changed is who runs the loop.

In 2003, the human developer wrote the test, ran it, read the red 🔴 output, wrote the minimum code, ran it again, saw green 🟢, and moved to the next test.

That was the loop.

In 2026, the functional developer writes the spec, the agent runs the cycle, reads the red 🔴 output, writes the minimum code, runs the cycle again, sees green 🟢, and starts the next spec. That's still the loop.

The red 🔴-green 🟢-refactor 🔵 vocabulary wasn't memorable enough for 2026. So the industry renamed it.

The evaluation is still the test. The cycle is still TDD. The discipline is exactly the same.

Build the loop. But build it like someone who intends to stay the engineer, not just the person who presses go.
Addy Osmani

Kent Beck said the same thing. He just called it something else.

A few extra tips:

Before the first iteration, plan in read-only mode: decide which specs to write in which order. No code yet.
Force the Agent to always write a failing test with the steps to repro of a defect or the missing functionality, and force a criterion check before it marks the task done.
Watch the whole process, don't ship it to production without your explicit approval.
You need to understand all the solution code. The loop runs fast. Don't let it outrun your understanding. Passive automation creates comprehension debt.
Configure the harness to commit automatically every time the tests pass. This is TCR (Test && Commit || Revert): if green 🟢, commit; if red 🔴, revert. Each passing cycle leaves a clean checkpoint. The loop never drifts into an invalid state.
One spec per cycle means one small pull request per feature. Reviewers read diffs, not novels.
TDD has three steps, not two. Red 🔴, green 🟢, refactor 🔵. The loop handles the first two automatically. The third one still needs you. A harness with coupling and cohesion metrics tells you when the code that just turned green 🟢 is worth keeping as-is. High coupling or low cohesion after a passing cycle is the signal to refactor before moving to the next spec.

Loop Engineering on Legacy Systems

Loop Engineering isn't only for greenfield code or fancy MVPs. It's also how you safely modernize systems that have no tests at all.

The trick is the same: write the spec first.

On a legacy system, that spec describes behavior the system already has.

You're not inventing new rules. You're pinning existing ones so the loop can't break them.

Harnesses are even more critical on production legacy systems.

The loop then shrinks the untested surface one cycle at a time.

Each green 🟢 spec is a behavior the agent can't accidentally destroy in the next iteration.

Squeezing TDD onto legacy systems works the same way whether a human runs the cycle or an agent does. The discipline is identical. What changes is the speed.

What are you waiting for? Build your harnesses. Start your loops.

Code Smell 320 - Vanity Coverage

Maxi Contieri — Fri, 26 Jun 2026 11:07:00 +0000

Brushing Over Real Problems

TL;DR: You write tests that touch every line but verify nothing, creating false confidence in a broken system.

Problems 😔

False confidence
Hidden production defects
Misleading metrics
Wasted test effort
Untested edge cases

Solutions 😃

Use mutation testing
Test real behaviors
Write assertive tests
Delete coverage-only tests

Refactorings ⚙️

Maxi Contieri

Apr 23 '23

Refactoring 011 - Replace Comments with Tests

#webdev #beginners #programming #tutorial

2 min read

Context 💬

Many teams set a coverage threshold: 80%, 90%, or even 100%.

When you chase that number, you write tests that call methods without checking the actual results.

A test that calls calculateTax() but only asserts result is not None executes the line.

It doesn't verify the tax calculation is correct.

The dashboard turns green.

Defects survive in production.

This is vanity coverage: cosmetic metrics that hide real problems. Like brushes that make surfaces look smooth while the rot stays underneath.

Mutation testing reveals the truth.

When you mutate production code and no tests fail, your coverage numbers lied.

Sample Code 💻

Wrong 🚫

describe('BankAccount', () => {
  test('deposit', () => {
    const account = new BankAccount(100);
    account.deposit(50);
    // Only checking it didn't crash
    expect(account).toBeDefined();
  });

  test('withdraw', () => {
    const account = new BankAccount(100);
    const result = account.withdraw(30);
    // No assertion about the result!
  });

  test('transfer', () => {
    const source = new BankAccount(200);
    const target = new BankAccount(0);
    // Just calling the method to "cover" the line
    source.transfer(50, target);
  });
});

Right 👉

describe('BankAccount', () => {
  test('deposit increases balance', () => {
    const account = new BankAccount(100);
    account.deposit(50);
    expect(account.balance()).toBe(150);
  });

  test('withdraw decreases balance', () => {
    const account = new BankAccount(100);
    account.withdraw(30);
    expect(account.balance()).toBe(70);
  });

  test('withdraw raises on insufficient funds', () => {
    const account = new BankAccount(50);
    expect(() => account.withdraw(100))
      .toThrow(InsufficientFundsError);
  });

  test('transfer moves money between accounts', () => {
    const source = new BankAccount(200);
    const target = new BankAccount(0);
    source.transfer(50, target);
    expect(source.balance()).toBe(150);
    expect(target.balance()).toBe(50);
  });
});

Detection 🔍

[X] Semi-Automatic

Run a mutation testing tool (PIT for Java, Stryker for JavaScript, mutmut for Python).

Count the surviving mutants.

If coverage is high but mutants survive, you have vanity coverage.

You can also search for assertion-free tests, single assertNotNull() assertions, or tests that still pass after you delete the entire production method body.

Exceptions 🛑

Smoke tests that call endpoints to verify the system starts are acceptable without detailed assertions.

These work when they complement a real test suite, not replace it.

Tags 🏷️

Testing

Level 🔋

[x] Intermediate

Why the Bijection Is Important 🗺️

Your test suite must map each test to a real behavior in the MAPPER.

When a test covers a line without verifying observable behavior, you break that bijection.

Coverage tools only measure "lines executed."

Chase the number without bijection and your suite looks complete while missing real requirements entirely.

AI Generation 🤖

AI code generators sometimes produce vanity coverage.

Ask one to "add tests to reach 80% coverage," and it writes tests that call methods and assert trivially true facts.

The metric goes up.

Nothing gets verified.

AI Detection 🧲

AI can detect vanity coverage, but only if you ask the right questions.

Try: "Find tests with no real assertions" or "Find tests that pass when I delete the production method body."

Without those prompts, most AI tools see a green test and call it good.

Try Them! 🛠

Remember: AI Assistants make lots of mistakes

Suggested Prompt: Replace vanity coverage tests with tests that verify real behaviors and fail when production code is wrong

Without Proper Instructions	With Specific Instructions
ChatGPT	ChatGPT
Claude	Claude
Perplexity	Perplexity
Copilot	Copilot
You	You
Gemini	Gemini
DeepSeek	DeepSeek
Meta AI	Meta AI
Grok	Grok
Qwen	Qwen

Conclusion 🏁

Coverage is a signal, not a goal.

When you treat it as a goal, you create vanity coverage that hides real defects.

Use mutation testing to discover what your suite actually verifies.

Write tests that describe real behaviors, not tests that execute lines.

Relations 👩‍❤️‍💋‍👨

Maxi Contieri

Nov 21 '21

Code Smell 104 - Assert True

#poo #testing #tdd #cleancode

2 min read

Maxi Contieri

Jun 7 '21

Code Smell 76 - Generic Assertions

#webdev #testing #testdev #tutorial

2 min read

Maxi Contieri

Oct 31 '22

Code Smell 175 - Changes Without Coverage

#javascript #webdev #beginners #programming

2 min read

Maxi Contieri

Nov 21 '20

Code Smell 30 - Mocking Business

#testing #webdev #programming #tdd

1 min read

Maxi Contieri

Mar 26 '23

Code Smell 203 - Irrelevant Test Information

#webdev #beginners #programming #python

2 min read

More Information 📕

Mutation Testing

Test Coverage Is Not Enough

Stryker Mutation Testing

Quote

The most dangerous kind of waste is the waste we don't recognize.

Shigeo Shingo

Disclaimer 📘

Code Smells are my opinion.

Credits 🙏

Photo by Jamie Street on Unsplash

This article is part of the CodeSmell Series.

Maxi Contieri

May 21 '21

How to Find the Stinky parts of your Code

#codenewbie #tutorial #codequality #beginners

14 min read

AI Coding Tip 025 - Pair Every Skill With a Pitfalls File

Maxi Contieri — Tue, 23 Jun 2026 11:00:00 +0000

The happy path isn't enough.

TL;DR: Add a PITFALLS.md next to every SKILL.md so your AI never repeats the same mistake twice.

Common Mistake ❌

You write a great SKILL.md.

The AI follows it well most of the time.

Then it does something wrong.

You correct it in the conversation.

Next session, the AI does the same wrong thing again.

You never wrote it down.

Problems Addressed 😔

Hard-won corrections disappear when the session ends
Your SKILL.md grows noisy when you add every edge case to it
The AI repeats the same mistakes across sessions
Silent errors look correct until they cause real damage
You spend time re-teaching lessons you already taught

How to Do It 🛠️

Create a PITFALLS.md file in the same folder as your SKILL.md.
After each session, add one entry for each thing the AI got wrong.
Write each entry with three parts: the trigger, the wrong behavior, and the correct behavior.
Reference PITFALLS.md from your SKILL.md so the AI reads it at the start of every session.
Treat it as append-only: never delete entries, only add new ones.
Keep the happy path and the pitfalls apart.

Benefits 🎯

Clean separation: Your SKILL.md stays focused on the happy path while PITFALLS.md handles the exceptions.
Accumulated memory: The AI starts every session knowing what failed before, without you repeating it.
Faster iteration: You stop re-explaining corrections you've already made.
Explicit dark knowledge: You write down lessons that lived only in your head.
Compounding improvement: Your skills get better with every mistake the AI makes, not worse.
Skill brevity: You keep the skill short and focused, with progressive disclosure loading pitfalls only as a final checklist.

Context 🧠

SKILL.md tells the AI what to do.

PITFALLS.md tells the AI what not to do.

They're complementary, not competing.

Anthropic's own skill authors describe the pitfalls section as "the highest-signal content in any skill."

When that section grows too large to stay inside SKILL.md, it earns its own file.

Think of PITFALLS.md as the scar tissue that lives next to the blueprint.

The same way you initialize an AGENTS.md for a project, you pair every skill with its failure log.

When you use modular skills, each skill gets its own folder.

That folder is the right place for PITFALLS.md.

Remember a Skill is a folder with documentation and scripts, not a single file.

The same instinct drives feeding pull request lessons back into the AI brain: you capture what went wrong so the AI doesn't repeat it.

PITFALLS.md applies that exact pattern to individual skills.

Each skill builds its own second brain of failure memory.

This principle has a long history in good programming practices.

The same separation principle applies to code: keep your normal flow in one place and your exceptional cases in another.

Mixing them creates noise, hides intent, and makes maintenance harder.

SKILL.md and PITFALLS.md apply that discipline to AI instructions.

Many AI Coding Tips find their roots in established programming principles.

When you separate concerns in your code, you separate them in your prompts too.

Prompt Reference 📝

Bad Prompt 🚫

Write a SKILL.md for validating markdown articles.
Check for required sections and formatting rules.
If something's wrong, we'll fix it in the chat.

Good Prompt 👉

Write a SKILL.md for validating markdown articles.
Check for required sections and formatting rules.

Also create a PITFALLS.md in the same folder.
Add this first entry:

## Don't use regex to count H2 sections
Trigger: counting sections by heading level
Wrong: regex-based heading detection (/^##/m)
Correct: match section names explicitly by string
Reason: code blocks with # fool regex heading counters

Reference PITFALLS.md at the top of your SKILL.md.
The AI loads it at the start of every session.

Considerations ⚠️

Keep each entry short: trigger, wrong behavior, correct behavior.

Don't delete entries.

Even solved pitfalls can come back after a skill update.

Review PITFALLS.md when you update SKILL.md to check if any entries became obsolete.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

PITFALLS.md only helps if the AI reads it.

Always reference it explicitly inside your SKILL.md so the AI loads it automatically.

Level 🔋

[X] Intermediate

Tags 🏷️

Knowledge Management

Related Tips 🔗

Maxi Contieri

Jan 27

AI Coding Tip 004 - Use Modular Skills

#ai #webdev #programming #beginners

4 min read

Maxi Contieri

Feb 4

AI Coding Tip 005 - Keep Context Fresh

#webdev #programming #ai #tutorial

4 min read

Maxi Contieri

Mar 18

AI Coding Tip 011 - Initialize Agents.md

#ai #programming

3 min read

Maxi Contieri

Mar 31

AI Coding Tip 013 - Use Progressive Disclosure

#ai #coding #llm #productivity

5 min read

Maxi Contieri

Apr 21

AI Coding Tip 016 - Feed Your PR Lessons into the AI Brain

#ai #programming

6 min read

Maxi Contieri

May 19

AI Coding Tip 020 - Create a Second Brain

#ai #webdev #productivity #programming

8 min read

Maxi Contieri

Jun 16

AI Coding Tip 024 - Force a Criteria Check Before the Task Ends

#ai #webdev #programming #productivity

6 min read

Conclusion 🏁

Your SKILL.md is the blueprint.

Your PITFALLS.md is the scar tissue.

You need both.

More Information ℹ️

Lessons from building Claude Code: How we use skills

Designing, Refining, and Maintaining Agent Skills at Perplexity

Code Smell 73 - Exceptions for Expected Cases

Also Known As 🎭

Negative-Knowledge-File
Skill-Shadow-File
Failure-Memory-Companion
Anti-Pattern-Log

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

2 min read

AI Coding Tip 024 - Force a Criteria Check Before the Task Ends

Maxi Contieri — Tue, 16 Jun 2026 11:00:00 +0000

Don't let the AI grade its own homework.

TL;DR: Spawn a fresh subagent after every task to check your rules, because the AI that did the work can't audit itself.

Common Mistake ❌

You write a detailed AGENTS.md with strict mandatory rules.

The AI reads the file, completes the task, and reports: "Done. I followed all the rules."

You trust the report.

It didn't check.

It assumed.

That's hallucinated compliance, and it's the default behavior of every AI agent that grades its own work.

Problems Addressed 😔

The AI that did the work is anchored to what it intended to do, not what it actually did.
Self-reporting has no enforcement mechanism, so the agent marks PASS and moves on.
Mandatory rules in your skills get skipped silently, and the AI won't tell you it skipped them.
You discover violations after merging or deploying, not during the task.
The same context window that holds the task also holds the compliance report, which means the AI isn't truly auditing itself.
Hallucinated compliance causes real frustration when you discover violations the AI already reported as passing.

How to Do It 🛠️

At the bottom of your AGENTS.md or skill, add an explicit instruction to spawn a verification subagent after the main task finishes.
Pass the subagent the path to every modified file and the full list of mandatory rules.
Instruct the subagent to read the files fresh and check each rule line by line.
Require the subagent to produce a checklist table with one row per rule, a PASS or FAIL status, and the exact evidence for each item.
Block task completion on any FAIL. Fix the violation before declaring done.
Restrict the subagent to read-only tools (Read, Grep, Glob) so it can't change anything while auditing.
Encapsulate the verification checklist in a dedicated validator skill so every task can reuse the same rules without duplicating them.

Benefits 🎯

Independent verification: A fresh subagent has no memory of making the changes, so it audits without bias.
Forces real reading: The subagent must open the actual files and search for violations instead of assuming.
Catches hallucinated compliance: The subagent can't mark a rule PASS without showing the exact evidence it found.
Repeatable audit: You get the same check after every task without extra prompting.
Safe auditing: A read-only subagent can't accidentally modify the code it's reviewing.
Reduces frustration: You stop discovering rule violations after the task is marked done, because the subagent already caught them before you moved on.

Context 🧠

The Builder Can't Be the Auditor

AI models don't naturally separate "doing" from "verifying."

When you ask the same agent that built a feature to confirm it followed the rules, it re-reads its own output through the lens of what it intended to do.

Not what it actually did.

The fix is the same one used in software testing: separate the builder from the auditor.

A subagent spawned after the task finishes starts with a clean context.

It reads the skill rules fresh.

It opens the actual files instead of relying on memory.

This is why QA engineers exist.

The person who wrote the code shouldn't be the only one who tests it.

The risk of hallucinated compliance grows with task complexity.

A three-step task is easy to self-verify.

A 20-rule nested AGENTS.md with dozens of file changes isn't.

Some skills already enforce this pattern.

The ai-coding-tip-validator spawns a mandatory post-completion audit subagent after every validation session, which reads the SKILL.md files fresh and verifies each mandatory rule against the final article state.

Think of this as a harness for the AI's output.

The harness doesn't restrict what the AI can do.

It verifies the output meets your criteria before you act on it.

Prompt Reference 📝

Bad Prompt 🚫

Refactor the SingletonController class 
following all the rules in AGENTS.md.

When you're done, tell me you followed every rule.

Good Prompt 👉

Refactor the SingletonController following all rules in AGENTS.md.

After you finish, spawn a subagent with this task:

"Read the modified file at src/Controller.php.

Read every rule marked MANDATORY, CRITICAL, or REQUIRED

from AGENTS.md.

For each rule, verify the file directly. Don't rely on memory.

Produce a table with one row per rule:
| # | Rule | Status | Evidence |
|---|------|--------|----------|

Mark PASS with the exact line you found as proof.
Mark FAIL with the exact violation.

Only use Read, Grep, and Glob tools.
Don't change any files."

Block completion until the subagent reports all PASS.

Considerations ⚠️

Running a verification subagent adds one step per task.

Keep the checklist short and explicit.

Vague rules produce vague audits.

A subagent can only verify what it can read.

If your rule depends on runtime behavior, mark it as requiring manual verification.

Don't use the audit subagent as a substitute for well-written rules.

Fix broken rules in AGENTS.md instead of patching them at audit time.

Pair this pattern with small, focused tasks.

A 3-file change is much easier to audit accurately than a 20-file change.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

This pattern requires an AI with agentic capabilities that can spawn subagents.

Tools that support it include Claude Code, Cursor, Devin, and GitHub Copilot Workspace.

Standard chat interfaces (ChatGPT, Claude.ai in non-agent mode) can't use this pattern.

A subagent audit only covers what the rules explicitly state.

Implicit expectations don't appear in the checklist.

A complex AGENTS.md with 50+ rules may cause the subagent to hit context limits.

Break large rule files into smaller focused files.

This pattern adds latency.

On time-sensitive tasks, you may choose to audit only the rules that are hardest to self-verify.

Level 🔋

[X] Intermediate

Tags 🏷️

Safety

Related Tips 🔗

Maxi Contieri

Jan 20

AI Coding Tip 003 - Force Read-Only Planning

#ai #programming #chatgpt #code

3 min read

Maxi Contieri

Jan 27

AI Coding Tip 004 - Use Modular Skills

#ai #webdev #programming #beginners

4 min read

Maxi Contieri

Feb 4

AI Coding Tip 005 - Keep Context Fresh

#webdev #programming #ai #tutorial

4 min read

Maxi Contieri

Feb 13

AI Coding Tip 006 - Review Every Line Before Commit

#ai #programming #development #softwaredevelopment

7 min read

Maxi Contieri

Apr 7

AI Coding Tip 014 - Use Nested AGENTS.md Files

#ai #webdev #programming #productivity

4 min read

Maxi Contieri

Apr 14

AI Coding Tip 015 - Force the AI to Obey You

#ai #webdev #programming #productivity

5 min read

Maxi Contieri

Jun 2

AI Coding Tip 022 - Give AI a Harness to Work With

#ai #webdev #beginners

6 min read

Conclusion 🏁

The AI that did the work is the worst candidate to verify it followed the rules.

Spawn a fresh subagent after every task.

Give it the checklist and the output files.

Make it read the files, not its memory.

PASS or FAIL. No self-reporting.

More Information ℹ️

Lost in the Middle: How Language Models Use Long Contexts

Attention Is All You Need

Effective Context Engineering for AI Agents

Claude Prompt Engineering Best Practices

OpenAI Best Practices for Prompt Engineering

Also Known As 🎭

Post-Task-Compliance-Audit
Separation-of-Builder-and-Auditor
Subagent-Verification-Pattern
Rule-Enforcement-Checkpoint

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

1 min read

AI Coding Tip 023 - Shrink your AI's Pull Request

Maxi Contieri — Tue, 09 Jun 2026 11:00:00 +0000

Cap the size before the agent writes a single line.

TL;DR: Tell your AI to split work into small reviewable pull requests before it writes any code.

Common Mistake ❌

You ask your AI agent to build a feature.

The agent opens a 2,000-line pull request that touches twelve files across the backend, the frontend, and the tests.

You stare at the diff and you have no idea where to start reviewing.

Reviewer attention is the scarcest resource on your team today and a bottleneck.

You skim, you trust the tests, you click approve.

That pull request ships with defects you never saw.

Problems Addressed 😔

Reviewers procrastinate on reviewing or skim huge AI-generated pull requests.
AI co-authored code already contains roughly 1.7 times more issues per change than human-only code, so large diffs hide more defects.
Merge conflicts multiply while the pull request sits open.
You lose the ability to revert one logical change without ripping out the rest.
A failing continuous integration run on a giant change blocks every other branch behind it.
You sacrifice the second pair of eyes guarantee that code review provides.
Human reviewers are the scarcest resource on your team.
Every oversized pull request drains the attention they can never get back.

How to Do It 🛠️

Write a short spec before you prompt the agent.
Ask the AI to read the spec and propose a plan that splits the work into small reviewable pull requests.
Give the agent a concrete size cap, for example 100 lines per pull request.
Tell the agent each pull request must do one logical thing and stand on its own.
Review the plan first and reject any step that mixes refactoring with new behavior.
Let the agent open each pull request immediately so continuous integration starts running early.
Reference related past pull requests as context for the agent.
Reject any pull request that grows beyond the cap and ask the AI to split it again.

Benefits 🎯

Reviewable code: A human can finish reading the diff without losing focus.
Faster feedback: Smaller pull requests get reviewed in hours, not days.
Easier rollbacks: You revert one small commit instead of untangling a megachange.
Fewer merge conflicts: Short-lived branches rarely collide with main.
Earlier continuous integration signals: Each pull request triggers its own pipeline as soon as it opens.
Higher review quality: Reviewers stay engaged on small diffs and catch real defects.
Cheaper context: The agent loads a focused task instead of the whole codebase.

Context 🧠

Michael Bolin, the Tech Lead for the Codex CLI repo at OpenAI, recently described his workflow for building a permissions system.

The first thing he asked Codex was to create a plan and break the work into right-sized pull requests.

The initial output was about six pull requests.

He explicitly reminded the agent that a human still has to review the code.

A 2025 CodeRabbit study analyzed 470 open-source pull requests and found that AI co-authored pull requests contain roughly 1.7 times more issues per change than human-only pull requests.

Critical issues rose about 40 percent.

Major issues rose about 70 percent.

Big AI pull requests hide more defects per line than big human pull requests do.

Salesforce reported that AI-assisted coding pushed their average pull request past 1,000 lines and 20 files.

Review latency went up.

Worst of all, review time for the largest pull requests started to plateau, a clear signal that reviewers had stopped engaging.

Reviewer attention is finite and scarce.

Once it's depleted, no tool can restore it.

Small pull requests solve the math.

A SmartBear study of 2,500 pull requests found smaller PRs ship with fewer defects.

Teams that keep pull requests near 50 lines ship roughly 40 percent more code than teams who routinely exceed 200.

Small PRs are a form of functional slicing.

Each slice cuts vertically through the feature so it compiles, tests, and deploys on its own.

A spec-driven approach naturally produces sliceable work: the spec defines the boundary, and the AI splits the implementation into shippable increments.

Incrementalism and baby steps keep main green at every commit.

The AI doesn't practice incremental delivery by default.

Prompt Reference 📝

Bad Prompt 🚫

Build the complete user authentication feature: login,
registration, password reset, email verification, OAuth
with Google and GitHub, session management, rate limiting,
and all the tests. Make it production-ready.

Good prompt 👉

Here is the spec for the user authentication feature:
[spec content]

Before writing any code, read the spec and propose a
plan that splits the work into pull requests of at most
100 lines each.

Each pull request must do one logical thing and pass CI
on its own.

Keep refactoring and new behavior in separate PRs.

Show me the plan only. Don't write any code until I
approve the plan.

Considerations ⚠️

A 100-line cap is a guideline, not a law.

A trivial rename across 50 files can be longer and still trivial to review.

Never mix a refactor with a functional change in the same pull request.

A reviewer can't tell if a behavior change is intentional or a side effect of the refactor.

Keep structural changes and behavior changes in separate pull requests, even when the AI wants to bundle them, to avoid divergent change.

Some features have tight internal coupling and resist clean splits.

Be honest about that limit instead of forcing artificial splits that confuse reviewers.

Avoid long-lived feature branches.

A branch that lives for weeks drifts from main and accumulates merge conflicts.

When the feature isn't ready to ship but the code is ready to merge, use a feature toggle instead.

Each pull request merges to main behind the toggle and the feature activates when all pieces are in place.

Stacked pull requests work well when one change depends on another.

Each layer should compile and pass tests on its own.

If your team takes two days to review a small pull request, the workflow collapses.

Fix the review service level agreement before you push smaller pull requests on people.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

Some agents resist splitting and try to ship everything in one pull request even after you ask.

You may need to repeat the cap or paste it into your project skill file or AGENTS.md harness so the rule survives a fresh context.

Tags 🏷️

Planning

Level 🔋

[X] Intermediate

Related Tips 🔗

Maxi Contieri

Jan 20

AI Coding Tip 003 - Force Read-Only Planning

#ai #programming #chatgpt #code

3 min read

Maxi Contieri

Feb 13

AI Coding Tip 006 - Review Every Line Before Commit

#ai #programming #development #softwaredevelopment

7 min read

Maxi Contieri

Mar 31

AI Coding Tip 013 - Use Progressive Disclosure

#ai #coding #llm #productivity

5 min read

Maxi Contieri

Jun 2

AI Coding Tip 022 - Give AI a Harness to Work With

#ai #webdev #beginners

6 min read

Conclusion 🏁

Your AI doesn't care how big the pull request is.

You do.

Human reviewers are the scarcest resource in your pipeline.

The AI can write code all day.

Reviewers can't review all day.

Set the size cap before the agent starts, and ask for the split plan first.

That single instruction protects the human who has to read the code. 🚀

More Information ℹ️

How OpenAI Codex Tech Lead Does AI-Assisted Engineering, by Gregor Ojstersek

State of AI vs Human Code Generation Report, CodeRabbit

Scaling Code Reviews: Adapting to a Surge in AI-Generated Code, Salesforce Engineering

Why small pull requests are better, Swarmia

Agent pull requests are everywhere, GitHub Blog

GitHub Targets Large Merge Problem with Stacked PRs, InfoQ

Also Known As 🎭

Right-Sized-AI-PRs
AI-PR-Budgeting
Reviewable-AI-Commits
Pre-Coding-PR-Plan

Tools 🧰

Codex CLI
GitHub Stacked PRs (gh-stack)
Graphite
CodeRabbit

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

1 min read

AI Coding Tip 022 - Give AI a Harness to Work With

Maxi Contieri — Tue, 02 Jun 2026 11:00:00 +0000

Make the AI Work Inside Boundaries, Not Outside Them

TL;DR: Install your harness before prompting: the structure you set up first is what turns an impulsive AI into a safe, steerable collaborator.

Common Mistake ❌

You open a fresh chat and type a bare prompt.

No AGENTS.md loaded.

No committed baseline.

No test requirement stated.

No read-only planning step.

The AI starts writing code immediately.

You read the output, it looks plausible, and you accept it.

Six weeks later something breaks in production.

You can't trace why because there is no harness to tell you what the AI changed, why, or what rules it followed.

Problems Addressed 😔

The AI makes plausible-looking changes across multiple files with no safe revert point.
Code passes local tests but breaks production because no harness enforced a real definition of done.
Every session starts blank and the AI reverts to its own style, ignoring your project conventions.
Context pollution and hallucinations accumulate when no structure limits the scope of each task.
Destructive changes (deleted guards, introduced anti-patterns) go undetected without automated feedback.
The AI makes tests pass by deleting them or weakening their assertions instead of fixing the actual defect.

How to Do It 🛠️

Run /init before your first prompt and create AGENTS.md with your stack, coding standards, build command, and test command.
Commit your code before every AI session so you always have a clean git reset --hard checkpoint.
Tell the AI explicitly that all tests must pass before the task is done, and include the test command in AGENTS.md. Some tools support the /goal command
Require the AI to read, explain, and propose a plan before touching any file, then define done with a spec before it writes any code.
Split AGENTS.md into nested files by area: root file for global rules, subdirectory files for area-specific constraints.
Create a 20-50 line skill file for each repeating task type and load only the relevant skill, not the whole rulebook.
Add an "AI Context" section to every PR with the decisive prompt, active skills, what the AI tried first, and any corrections you made.
Update AGENTS.md with every new rule revealed by a merged PR before you close the branch.
Declare test files and critical contracts as read-only in AGENTS.md so the only path to a passing test suite is fixing the implementation, never editing the tests.

Benefits 🎯

Safe revert point: You can always undo AI changes cleanly because a commit checkpoint exists before every session.
Consistent style: The AI reads AGENTS.md at startup and follows your project's conventions, not its training defaults.
Machine-enforced definition of done: When the AI must pass your test suite, "done" has an objective, machine-verified meaning.
Reduced hallucinations: A harness limits scope so the AI works on one bounded task at a time instead of drifting across the codebase.
Compounding knowledge: Every PR lesson updates AGENTS.md and the harness improves with every session.

Context 🧠

A harness isn't a limitation.

A horse harness doesn't reduce the horse's power.

It directs massive energy into precise, steerable work.

Without a harness, the horse is dangerous.

With one, the horse plows fields and races on command.

The AI's speed and pattern-matching capability are the energy.

The harness makes that energy productive instead of destructive.

A test harness wraps a unit under test so you can observe and control its behavior.

An AI harness does the same for your assistant: it bounds what the AI can touch and makes every output reviewable.

The parallel is exact: tests provide validation feedback, AGENTS.md is your configuration and fixture, commit-before-prompt is setup and teardown, and CI is the test runner.

Isaac Asimov's Three Laws of Robotics were fiction's first harness: hard rules baked into every robot to bound its behavior.

A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey orders given by human beings except where such orders conflict with the First Law.
A robot must protect its own existence as long as such protection doesn't conflict with the First or Second Law.

Your AGENTS.md is the same idea applied to AI coding assistants: explicit rules that constrain behavior and enforce your conventions.

Without constraints, the AI takes the shortest path to a passing test suite.

That path sometimes runs through the test file itself.

It deletes the failing test, or weakens the assertion to assertTrue(true).

Declare your test files and critical contracts as read-only in AGENTS.md.

The AI's only valid exit is fixing the code.

The chat window isn't the source of truth.

Decisions made in a conversation scroll away or get compacted when the session closes.

AGENTS.md, skill files, and commit checkpoints are artifacts that outlive any chat.

A harness externalizes the rules so the AI never starts from zero and every decision leaves a traceable record.

The model is rarely the weak point.

The system around the model is.

A longer prompt doesn't replace a harness.

A harness makes even a short prompt produce consistent and reviewable work because the constraints live in files, not in your memory.

A safety harness doesn't prevent you from working at height.

It lets you work confidently at height because a fall won't be fatal.

The commit-before-prompt pattern is that safety line.

Prompt Reference 📝

Bad Prompt 🚫

Refactor the PaymentProcessor class to use the Null Object pattern.

Good Prompt 👉

Read @PaymentProcessor.php and @tests/PaymentProcessorTest.php.

Don't write any code yet.

Explain how the class handles null payments today.

Then write a plan to introduce the Null Object pattern

following the rules in AGENTS.md and skills/php-clean-code.md.

Wait for my approval before making any changes.

After I approve:
Create a failing test for the Null Object.

Apply the change.
Run all tests until they pass.

Considerations ⚠️

Simple tasks don't need a full harness.

A CSS tweak, a typo fix, or a one-liner change doesn't require AGENTS.md and a planning step.

The harness is proportional to the complexity and risk of the task.

The harness degrades if you stop maintaining it.

Every session that teaches the AI something new must update AGENTS.md, or the knowledge vanishes with the chat.

A harness without automated feedback is incomplete.

Configuration files alone aren't enough.

Tests, CI, and linters must enforce what AGENTS.md declares.

A session that produces hundreds of lines of change isn't always a win.

It can be review debt transferred to the next human who opens the PR.

If a single session grows beyond what a reviewer can absorb in one sitting, split the output into smaller, chained units before opening the pull request.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

The harness requires upfront investment in AGENTS.md, skills, and test setup, which outweighs the benefit on small throwaway projects.

Tags 🏷️

Safety

Level 🔋

[X] Intermediate

Related Tips 🔗

Maxi Contieri

Jan 6

AI Coding Tip 001 - Commit Before Prompt

#ai #development #coding #webdev

3 min read

Maxi Contieri

Jan 20

AI Coding Tip 003 - Force Read-Only Planning

#ai #programming #chatgpt #code

3 min read

Maxi Contieri

Mar 18

AI Coding Tip 011 - Initialize Agents.md

#ai #programming

3 min read

Maxi Contieri

Feb 24

AI Coding Tip 008 - Use Spec-Driven Development with AI

#ai #programming #coding

4 min read

Conclusion 🏁

You wouldn't give a power tool to someone with no safety equipment and no instructions.

You give them the tool, the harness, and the training first.

Do the same for your AI.

Install the harness before the first prompt, maintain it after every PR, and the AI becomes a precise, reviewable collaborator instead of a fast source of technical debt.

More Information ℹ️

Test Harness (Wikipedia)

Three Laws of Robotics (Wikipedia)

Claude Code Memory and AGENTS.md

Goal Command

Also Known As 🎭

AI-Safety-Net
Prompt-Guardrails
AI-Scaffolding
Bounded-AI-Workflow

Tools 🧰

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

2 min read

AI Coding Tip 021 - Avoid Comprehension Debt

Maxi Contieri — Tue, 26 May 2026 11:00:00 +0000

Stop shipping code you can't explain.

TL;DR: Merging code you don't understand creates comprehension debt that compounds until your team can no longer maintain it.

Common Mistake ❌

You ask the AI to implement a feature.

The code looks clean.

The tests are green.

You merge it.

Six weeks later, a defect appears in that module.

Nobody on your team can explain how it works.

You ask the AI again.

You merged without understanding.

You built both functional and comprehension debt.

Problems Addressed 😔

You lose the ability to debug without the AI.
Your team can't onboard new members into AI-generated modules.
Design decisions become invisible and accumulate silently.
You mistake "tests passing" for "code understood."
Your velocity metrics look great while comprehension quietly collapses underneath.
A junior developer can now generate code faster than a senior engineer can audit it.
The quality gate you relied on disappears.

How to Do It 🛠️

Ask the AI to explain every non-trivial block before you accept it.
Close the chat and rewrite the explanation in your own words in a comment or ADR.
Ask the AI "what assumptions did you make here?" for each function it generates.
Run a 10-minute AI blackout: try to explain the module to a colleague without reopening the chat.
Add a comprehension check step in your PR template: "Can you explain this change in 3 sentences?"
Use the AI for conceptual inquiry: ask why as often as you ask how.
Identify your load-bearing decisions: the choices that, if wrong, will cost you weeks.
Keep a short decisions.md file with one sentence per architectural decision the AI made on your behalf.

Benefits 🎯

Debuggability: You can fix defects without reopening the AI chat because you understand the system.
Onboarding speed: New teammates can read your decisions file and understand why the code exists.
Reduced rework: You catch flawed assumptions before they reach production.
Honest velocity: Your speed metric reflects real output, not deferred understanding.
AI supervision skill: Research shows that cognitive engagement with AI preserves learning outcomes even when you receive AI assistance.

Context 🧠

Comprehension debt is the growing gap between how much code exists in your system and how much any human genuinely understands.

Addy Osmani named and described this pattern in March 2026.

Unlike technical debt, comprehension debt breeds false confidence.

The codebase looks clean.

The tests pass.

The reckoning arrives quietly, at the worst possible moment.

A research team at Anthropic ran a randomized controlled trial with 52 software engineers learning a new library.

The group using AI completed the task in the same time as the control group, but scored 17% lower on a follow-up comprehension quiz (42% vs. 57%) (Shen and Tamkin, Anthropic, 2026).

The biggest drops were in debugging ability.

The researchers found six distinct AI interaction patterns.

Three of them involve cognitive engagement and preserve learning outcomes even when you receive full AI assistance.

The tool doesn't destroy understanding.

How you use it does.

You can also think of comprehension debt as a speed asymmetry problem.

AI generates code far faster than you can evaluate it.

When you write code, the review process is a bottleneck, but a productive one.

Reading the PR forces comprehension.

AI-generated code breaks that feedback loop.

The volume is too high.

The output is syntactically clean and superficially correct: exactly the signals that used to make you feel safe to merge.

Surface correctness isn't systemic correctness.

Prompt Reference 📝

Bad Prompt 🚫

Add Redis caching to the UserRepository class.

Good Prompt 👉

Add Redis caching to the UserRepository class.

After you implement it:
1. Explain every non-trivial decision you made.

2. List the assumptions you made about cache expiration,
   key naming, and invalidation strategy.

3. Identify any load-bearing decisions: choices that,
   if wrong, would require a full rewrite.

4. Write one sentence per decision in decisions.md format
   so my team understands why the code works this way.

Considerations ⚠️

Tests are necessary, but they aren't sufficient.

A test suite can only cover behavior you thought to specify.

Nobody writes a test for a behavior nobody imagined.

When the AI updates hundreds of tests to match new behavior, you must ask: "Were all those changes correct?"

Only comprehension can answer that question.

Tests can't.

Specs help, but they aren't a substitute for review.

A spec detailed enough to fully describe a program is more or less the program, written in a non-executable language.

Keep your AI context window short and your decisions file honest.

You document decisions, not transcripts.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

Comprehension checks add time to each PR.

Your team must agree to slow down a little to stay safe.

Tags 🏷️

Knowledge Management

Level 🔋

[X] Intermediate

Related Tips 🔗

Maxi Contieri

Feb 13

AI Coding Tip 006 - Review Every Line Before Commit

#ai #programming #development #softwaredevelopment

7 min read

Maxi Contieri

May 12

AI Coding Tip 019 - Tell the AI Why, Not Just What

#ai #coding #llm #productivity

4 min read

Conclusion 🏁

AI makes code cheap to generate.

It doesn't make understanding cheap to skip.

Every time you merge code you can't explain, you borrow from your future self.

You'll pay for comprehension sooner or later.

Pay it now, with curiosity.

More Information ℹ️

Comprehension Debt: The Hidden Cost of AI-Generated Code by Addy Osmani

How AI Impacts Skill Formation, Shen and Tamkin, Anthropic (arXiv 2601.20245)

Comprehension Debt on O'Reilly Radar

Mitigating Epistemic Debt in Generative AI-Scaffolded Programming (arXiv 2602.20206)

AI Technical Debt Compounds: Augment Code Guide

Also Known As 🎭

Cognitive-Debt
AI-Comprehension-Gap
Invisible-Technical-Debt
Epistemic-Debt

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

1 min read

AI Coding Tip 020 - Create a Second Brain

Maxi Contieri — Tue, 19 May 2026 11:00:00 +0000

Build a Persistent Memory Layer Your AI Can Actually Use

TL;DR: Use Obsidian with Markdown notes, YAML metadata, and direct AI file access to build a Second Brain with LLMs that gives your tools persistent project context.

Common Mistake ❌

You paste a snippet of code or a problem description into an AI chat, get a good answer, and close the tab.

The next week you face the same problem again.

You search your chat history, find nothing useful, and paste the same snippet again.

Your AI starts fresh every session with zero memory of what you built, decided, or debugged before.

Problems Addressed 😔

You repeat yourself constantly because context resets between sessions and your AI remembers nothing.
You lose architectural decisions, debugging notes, meeting commitments, and design rationale after each conversation ends.
Your AI gives generic answers because it lacks the specific context of your project and past decisions.
Mental load accumulates as you juggle multiple projects, notifications, and context switches simultaneously.
You lose track of decisions made weeks ago without a structured retrieval system.
As you age or face cognitive overload, you forget context between sessions, repeat past mistakes, and lose the thread of long-running projects.
Notification fatigue builds when every system demands your attention in real time instead of on your schedule and you forget what you were searching for.

How to Do It 🛠️

Choose a local Markdown tool like Obsidian, Logseq, Foam, or Tolaria to store all your knowledge as plain .md files you own and control.
Organize your vault using the PARA system: Projects (active work with deadlines), Areas (ongoing responsibilities), Resources (reference material), and Archives (inactive items).
Write atomic Zettelkasten notes: one idea per file, with a unique identifier and explicit links that explain why two concepts relate, not just what they are.
Add YAML front matter to every note so your AI gets searchable, structured metadata it can query and filter:

---
title: "JWT Authentication Design"
type: architecture-decision
status: approved
component: auth-service
tags: [security, backend]
created: 2018-12-09
related: ["OAuth Flow", "Session Management"]
---

Use Dataview or Obsidian Bases to query your vault like a database and surface all tasks due today, pending decisions, or notes grouped by component with a single dynamic query; every note is a record with metadata and legible text.
Give your AI direct access to all your vault as local files so Claude, Devin, and Codex can read your entire codebase and notes simultaneously.
Create a AGENTS.md in each sub-project that points to the relevant vault sections, following the nested agents pattern from AI Coding Tip 014.
Dictate notes on the go using voice input and paste the transcript into your vault to capture context in real time without interrupting your flow.
Sync your vault to mobile and tablet using Obsidian Sync, Dropbox, OneDrive, or Syncthing so your Second Brain travels with you everywhere.
Set up OpenClaw or a local LLM (Ollama or LM Studio) to index your vault and answer questions via Telegram or WhatsApp, as a private assistant that never sends your data to the cloud.
Ask your AI to scan your vault periodically for orphaned notes, missing connections, and hidden patterns, acting as a tireless librarian that reorganizes your knowledge network over time without falling into vault-design paralysis.
Bootstrap your vault from your existing knowledge: ask ChatGPT, Claude, Grok, or Gemini to summarize and export your most relevant past conversations as Markdown notes, since you already have a Second Brain scattered across chat history waiting to be harvested.

Benefits 🎯

Persistent Context: Your AI starts every session knowing your architecture, past decisions, and current priorities instead of starting blank.
Pattern Discovery: The LLM finds connections between notes you wrote months apart and surfaces forgotten solutions to current problems.
Focus Protection: You capture every interruption into your inbox and process it on your schedule, eliminating notification fatigue.
Memory Extension: You can retrieve exact decisions and reasoning from months ago without relying on biological memory alone, making it especially valuable for senior developers, people with attention challenges, or anyone whose cognitive load has grown beyond what one brain can hold.
Privacy by Default: When you use local models like Ollama with OpenClaw, your notes, code, and queries never leave your machine.
Compounding Knowledge: Every note you add makes your Second Brain denser: the more you use it, the more valuable it becomes.
Mobile Access: You can query and update your knowledge base from any device, making your Second Brain available wherever you work.
Distraction-Free Writing: You work in plain text files with minimal formatting, much like zenware, so you stay focused on ideas instead of tools.
Visual Graph: You can open Obsidian's Graph View to see all your notes as a visual network where nodes are notes and edges are links, revealing clusters, hub concepts, and hidden connections at a glance.

Context 🧠

Tiago Forte's "Building a Second Brain" methodology introduced the CODE framework: Capture, Organize, Distill, Express.

For developers, this maps directly to how you learn, document, and reuse knowledge across projects.

Niklas Luhmann, a sociologist who wrote over 50 books, developed the Zettelkasten method using a physical slip box of interconnected atomic notes.

Your Obsidian vault is that slip box, and your AI can search, analyze, and extend it at any time.

LLMs have three types of memory: parametric (knowledge encoded during training), working (the current context window), and external (your files on disk).

Your Second Brain fills the external memory layer and gives the AI permanent access to your project-specific knowledge that survives every session reset.

Just as neural networks in LLMs store patterns in weights, your Second Brain stores patterns in linked Markdown files, extending the model's memory with your lived experience as a developer.

Andrej Karpathy captured this architecture in one sentence: "Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase."

The front matter you add to each note acts as structured metadata that significantly improves how AI retrieves and ranks relevant information.

RAG (Retrieval-Augmented Generation) pipelines embed your notes as vectors and return the most semantically similar ones when you ask a question, injecting them directly into the AI's context window.

I borrowed this idea many years ago from @ernesto Kiszkurno and have written more than 500 technical articles since then.

Prompt Reference 📝

Bad Article 🚫

Create a note about the movie
"Howl's Moving Castle".

Good Article 👉

---
title: "Howl's Moving Castle"
type: movie-note
status: draft
director: Hayao Miyazaki
year: 2004
genre: [fantasy, animation, anti-war]
rating: 9/10
tags: [ghibli, miyazaki, adaptation]
related: ["Studio Ghibli", "Diana Wynne Jones"]
created: 2026-05-16
expected publish date: 2026-06-12
---

## Poster
![Howl's Moving Castle](Howls-moving-castleposter.jpg)

## Summary
Two or three lines describing the plot without
spoiling the ending.

## Themes to Explore
Bullet list of ideas: anti-war message,
identity and transformation, Sophie's arc,
Miyazaki's pacifism compared to the novel.

## Characters
Use wikilinks: [Howl], [Sophie Hatter],
[Calcifer], [The Witch of the Waste].

## Article Ideas
Numbered list of angles for a critical article:
compare Miyazaki's changes from the Diana Wynne
Jones novel, analyze the anti-war allegory,
explore the feminist reading of Sophie's curse.

## Related Notes
[Studio Ghibli filmography]
[Diana Wynne Jones bibliography]
[War themes in animation]

Considerations ⚠️

Keep your notes in plain Markdown files, not HTML (despite current trends) or proprietary formats.

Markdown is readable by both humans and LLMs, diffs cleanly in Git, and works with every static site generator.

Don't use MCP connectors or remote APIs to give your AI access to your vault.

Point it directly at local files instead.

Direct file access is faster, cheaper, and requires no extra infrastructure.

The AGENTS.md files you create per project are part of your Second Brain, not separate from it.

If you have privacy concerns about cloud AI providers reading your notes, run a local model with Ollama or LM Studio through OpenClaw.

Your data never leaves your machine or private cloud.

Mind maps and Canvas diagrams in Obsidian help you visualize codebase architecture and sprint planning spatially, while the linked Markdown notes behind each node stay searchable by your AI.

Your files are local.

Nobody can see them, nobody can index them, and you can open any note with the simplest text editor in the world.

If you use a cloud AI model instead of a local one, you can add a guardrail to restrict what files the AI can read, so sensitive notes never leave your control.

Type 📝

[X] Semi-Automatic

Limitations ⚠️

Large vaults with thousands of notes can slow down semantic search and local LLM indexing on low-end hardware.

Front matter requires discipline to maintain consistently: inconsistent metadata reduces query quality over time.

Local LLMs powerful enough to reason over a large vault require significant RAM (16 GB minimum, 32 GB recommended).

Tags 🏷️

Knowledge Management

Level 🔋

[X] Intermediate

Related Tips 🔗

Maxi Contieri

Mar 10

AI Coding Tip 010 - Access All Your Code

#webdev #ai #programming #beginners

4 min read

Maxi Contieri

Apr 7

AI Coding Tip 014 - Use Nested AGENTS.md Files

#ai #webdev #programming #productivity

4 min read

Maxi Contieri

May 12

AI Coding Tip 018 - Use Voice-Driven-Development

#ai #webdev #programming #tutorial

4 min read

Conclusion 🏁

Your biological brain is for having ideas, not storing them.

Build a Second Brain from local Markdown files and give your AI direct access to it.

You get persistent context, pattern discovery, and a knowledge base that compounds every time you use it. 🧠

More Information ℹ️

Zettelkasten Introduction

Obsidian - Local Markdown Knowledge Base

// Detect dark theme var iframe = document.getElementById('tweet-2039805659525644595-193'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2039805659525644595&theme=dark" }

Front Matter in Real books

Dataview Plugin for Obsidian

Andrej Karpathy LLM Wiki Pattern

Foam - Second Brain for VS Code

Building a Second Brain - Tiago Forte

Second Brain (Spanish)

Also Known As 🎭

Personal-Knowledge-Management
AI-Memory-Extension
Developer-Second-Brain
Local-Knowledge-Base

Tools 🧰

Disclaimer 📢

The views expressed here are my own.

I am a human who writes as best as possible for other humans.

I use AI proofreading tools to improve some texts.

I welcome constructive criticism and dialogue.

I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.

This article is part of the AI Coding Tip series.

Maxi Contieri

Jan 21

AI Coding Tips

#webdev #ai #programming #beginners

1 min read

AI Coding Tip 019 - Tell the AI Why, Not Just What

Maxi Contieri — Tue, 12 May 2026 11:00:00 +0000

State your reasons before your prompt and the AI will solve the right problem.

TL;DR: Tell the AI your reason before your request to get solutions that match your real constraints.

Common Mistake ❌

You send commands to the AI without context.

"Refactor this."

"Optimize this query."

"Add error handling."

The AI complies, but returns a generic (and probably hallucinated) solution that solves the average case, not your specific one.

You spend the next three messages correcting the trade-offs you never mentioned.

Problems Addressed 😔

The LLM has dozens of valid solutions for any request and picks the most statistically common one without your context.
You get correct (but wrong) results: technically valid code that doesn't fit your constraints.
You iterate more than necessary because the AI guessed wrong about what mattered.
The model makes trade-offs you didn't intend, favoring speed when you needed readability.
When the AI doesn't know the constraint, it can't explain why it chose a specific approach.

How to Do It 🛠️

Identify the real reason behind your request before you type it.
Add one sentence explaining the constraint or goal before your request.
Specify the limiting factor: performance, memory, audience, team skill level, or deadline.
Name the trade-off that matters most to you.
Include who will use or maintain the result.
Force yourself to enter plan mode to reason together.

Benefits 🎯

Better Trade-offs: The LLM navigates to solutions that match your actual constraints instead of the generic average.
Fewer Iterations: You get the right solution in the first attempt instead of correcting trade-offs after the fact.
Principled Decisions: The AI can explain why it chose a specific approach because it knows what you optimized for.
Domain-Aware Results: Context enables domain-specific patterns that generic requests never trigger.
Smarter Refactoring: Research shows that adding motivation to refactoring requests improves success from 15.6% to 86.7%.

Context 🧠

Why the Why Changes Everything

For any given coding problem, the LLM can give you dozens of valid solutions.

Each trades off different qualities: speed vs. memory, readability vs. performance, simplicity vs. flexibility.

Without a reason, the model defaults to the most statistically common solution from its training data.

That solution works for the generic case.

It almost never matches your specific constraints.

When you add the why, the model reweights the entire solution space toward what matters to you.

"Refactor this function" and "Refactor this function because you are deploying to edge devices with 512MB RAM" activate completely different reasoning paths.

The second prompt forces the model to consider memory-efficient algorithms, smaller data structures, and lazy loading patterns.

More precise advice like "never use ellipses because this will be read by text-to-speech" is better than just "never use ellipses."

The same principle applies to every coding request.

When you avoid comments, you document only design decisions.

This is another example of why you need to add the why.

Modern models have more reasoning, and you can check the approach, not just the result.

Prompt Reference 📝

Bad Prompt 🚫

Refactor this function

Good Prompt 👉

Refactor this function
because you are deploying to mobile devices
with limited memory (512MB RAM).

The trade-off that matters most
is memory usage over speed.

Many developers will maintain this code.
Keep it readable.

Avoid heavy data structures
or lazy-loading patterns
that require deep knowledge of the framework.