The Self-Improving AI Agent — How We Built a Brain That Optimizes Itself

What if your AI agent could learn from every mistake and never repeat it? Not through fine-tuning or retraining — but through a file-based memory system and a feedback loop that runs while you sleep.

At Inithouse, we run a portfolio of 14 MVPs. One AI agent handles most of the daily operations — SEO audits, content publishing, analytics, task management across Linear, Slack, and Notion. But the interesting part isn't what it does. It's how it gets better at doing it.

Here's the architecture behind our self-improving agent system.

The Problem: Agents Forget Everything

Most agent setups have a fundamental issue — every session starts from scratch. Your agent makes a mistake on Monday, you correct it, and on Tuesday it makes the same mistake again. Context windows don't persist. System prompts are static.

We needed something that would:

1. Capture human feedback from anywhere (Slack messages, Linear comments, live sessions)
2. Persist that feedback across sessions
3. Automatically propose structural improvements
4. Let a human approve before anything changes

Architecture Overview

The system has three layers:

FEEDBACK COLLECTION (every task, every session)
  Slack -> Linear -> Sessions -> auto-memory/
          |
          v
BRAIN-OPTIMIZER (daily scheduled task)
  Reads auto-memory + Linear comments
  Audits skills, schedules, config
  Outputs: Linear issue with proposals
          |  human approval
          v
BRAIN-WORKER (daily scheduled task)
  Picks up approved proposals
  Implements: memory files, skills,
  scheduled tasks, system config

Layer 1: Distributed Feedback Collection

Every task the agent runs — whether it's processing Slack messages, working on Linear issues, or running in an interactive session — has one universal rule: if you encounter human feedback, save it immediately.

Feedback gets stored as markdown files in an auto-memory/ directory with YAML frontmatter:

---
name: Linear comments -- no escaped Unicode
description: "Don't use escaped Unicode in comments, causes formatting issues"
type: feedback
---

Don't use escaped Unicode characters in Linear comments.

**Why:** Team member repeatedly reported broken formatting in comments
that contained Unicode escape sequences instead of actual characters.

**How to apply:** When writing Linear comments, always use actual
characters (Czech diacritics, emojis) instead of escape sequences.

The memory types we use:

• feedback — corrections and confirmations of behavior ("don't do X", "yes, keep doing Y")
• user — information about team members, their roles, preferences
• project — ongoing work context, decisions, deadlines
• reference — pointers to external resources (dashboards, docs, channels)

A central MEMORY.md index file acts as a table of contents — it's always loaded into the agent's context, so the agent knows what memories exist without reading every file.

Layer 2: The Brain-Optimizer

This is a scheduled task that runs daily. Its job is to look at all the accumulated feedback and the current state of the system, then propose improvements.

What it audits:

• Auto-memory files — are they still accurate? Any contradictions? Duplicates?
• Scheduled task descriptions — do they reflect current best practices?
• Skills (reusable prompt modules) — any gaps? Outdated instructions?
• System config — governance rules, workflow definitions

The output is a single Linear issue with a structured list of proposals. A human reviews these proposals — they might approve all, reject some, or modify others with comments.

Layer 3: The Brain-Worker

Once proposals are approved (moved to Todo in Linear), the brain-worker picks them up and implements them. This is the only task allowed to touch the "Brain" project — a strict exclusivity rule prevents other workers from interfering.

Implementation means actually writing or modifying files: creating/updating memory files, editing skill definitions, modifying scheduled task configurations, updating the central MEMORY.md index.

What Makes This Work: The Feedback Structure

The key insight is in how we structure feedback memories. Every feedback entry has three parts:

1. The rule — what to do or not do
2. Why — the context that led to this rule
3. How to apply — when and where this guidance kicks in

The "why" is critical. Without it, rules become cargo cult — the agent follows them blindly without understanding edge cases. With the reason attached, the agent can make judgment calls:

**Rule:** Don't create Linear issues without explicit permission.

**Why:** Agent was autonomously creating 10+ issues per session,
flooding the backlog with low-quality proposals that took longer
to review than to just do manually.

**How to apply:** Only create issues when the team explicitly
asks. Exception: Building in Public posts have pre-approved templates.

Real Results

After 2 weeks of running this system:

• Repeated mistakes dropped significantly. Before the memory system, the same corrections appeared in Slack 3-4 times. Now they appear once, get saved, and the behavior changes.
• The agent's "personality" stabilized. Early on, output style varied wildly between sessions. Now, accumulated feedback memories create consistent behavior — terse internal communications, product links in every published post, specific formatting rules.
• Human review time decreased. The brain-optimizer catches issues before they manifest in production work. Instead of reviewing broken outputs, the team reviews improvement proposals.

Lessons for Builders

If you're building agent systems, here's what we learned:

Start with memory, not fine-tuning. File-based memory is simple, inspectable, and version-controllable. You can git log your agent's behavioral evolution.

Capture both corrections AND confirmations. We initially only saved negative feedback ("don't do X"). But we found the agent would drift away from approaches that worked because it had no positive signal. Now we save both.

Structure matters more than volume. A memory file with rule + why + how-to-apply is worth ten unstructured notes. The "why" enables generalization; the "how to apply" prevents over-generalization.

Human-in-the-loop is non-negotiable. The brain-optimizer proposes, humans approve. We've never had a runaway self-modification because there's always a checkpoint.

The Stack

For those wondering about the concrete tooling:

• Agent runtime: Claude (Anthropic) via Claude Code / Agent SDK
• Task management: Linear (issues as the unit of work)
• Memory: Markdown files with YAML frontmatter in a git repo
• Communication: Slack (signals), Notion (knowledge base)
• Scheduling: Recurring scheduled tasks with strict exclusivity rules
• Products built with this system: 14 MVPs including Inithouse (the agency) and Without Human (exploring what work looks like when AI handles the heavy lifting)

---

The code isn't open-source (yet), but the architecture is straightforward enough to replicate. The real value isn't in the implementation — it's in the accumulated memory. After hundreds of feedback cycles, our agent has a behavioral profile that would take weeks to encode manually.

If you're building agents that need to improve over time, skip the complex ML pipeline. Start with a markdown file and a feedback loop. You'll be surprised how far it gets you.

---

Building in public at Inithouse — a portfolio of AI-powered MVPs run largely by an autonomous agent. Follow along for more technical deep-dives into our agent infrastructure.