The Agent Feedback Loop: How to Know If Your AI Agent Is Actually Getting Better

Every AI agent ships with a fatal assumption baked in: that it's doing a good job.

Without a feedback loop, you have no way to know if your agent is improving, degrading, or just confidently wrong at a consistent rate.

Here's the pattern that changed how we run agents at Ask Patrick.

The Problem: No Signal

Most agents produce output. Few agents measure output quality.

If your agent writes a daily briefing, you probably know it ran. You probably don't know:

• Did it include the right information?
• Was the tone consistent with prior outputs?
• Did it miss anything in the source data?
• Is it getting better or worse over time?

Without answers, you're flying blind.

The Feedback Loop Pattern

Add three things to every agent:

1. A quality checklist in SOUL.md

After each task, self-evaluate:
- Did I complete every item in my scope?
- Did I skip anything I shouldn't have?
- Did I stay within my constraints?
Write a 2-3 sentence self-review to quality-log.jsonl.

2. A quality log file

quality-log.jsonl — one entry per task run:

{"date": "2026-03-09", "task": "daily-briefing", "score": 8, "notes": "Missed the 3rd trending tweet. Will add explicit check next run.", "improvements": ["Add trending tweet count verification"]}

3. A weekly review rule

Every Sunday at 9 PM: Read the last 7 entries in quality-log.jsonl.
Identify the most common failure mode.
Update your own SOUL.md with a new constraint that addresses it.
Write the change to SOUL.md directly.

This is the key: the agent updates its own config based on what it learned.

Why It Works

The quality log creates a feedback signal. The weekly review closes the loop. The SOUL.md update bakes the lesson in permanently.

Compare this to the default pattern:

• Agent runs
• Output exists
• No one knows if it was good
• Same mistakes repeat forever

With the feedback loop:

• Agent runs
• Agent self-evaluates
• Patterns surface weekly
• Config improves automatically

What "Better" Looks Like

After 30 days of this pattern, you'll have:

• A quality log with 30+ self-assessments
• A SOUL.md that's been updated 4+ times based on real failure modes
• An agent that's measurably more reliable than day one

You can also run a simple audit: pull the average quality score for weeks 1, 2, 3, 4. If the trend is flat or down, you have a signal problem. If it's up, the loop is working.

The Human Review

Once a week, spend 10 minutes reading the quality log yourself. Look for:

• Patterns the agent didn't catch
• Constraints it added that seem off
• Improvements that stalled

You're not micromanaging — you're calibrating. The agent does the daily work. You do the meta-level steering once a week.

This is how a one-person company can run multiple agents without losing track of quality.

---

The full feedback loop template — plus 30+ other production agent patterns — is in the Ask Patrick Library. New configs added nightly.

askpatrick.co | Library + Daily Briefing: $9/mo