The most common failure mode I see in AI agent setups: the agent makes the same mistake repeatedly. Not because it cannot learn -- because nothing in the system is set up to teach it.
You get a result, you correct it, you move on. The agent has no idea the correction happened. Next session, same mistake.
This is fixable. Here is a working feedback loop that actually makes agents improve over time.
The core problem
Most agent interactions are stateless. You ask, it answers, the conversation ends. If you edited its output, deleted a file it created, or ignored its suggestion -- the agent has no record of it.
The feedback loop needs three things:
- Capture -- record what happened, including corrections
- Store -- keep feedback in a form the agent can reference
- Retrieve -- surface relevant feedback when similar situations arise
That is the entire architecture. Everything else is implementation detail.
Capture: Logging with Feedback Tags
Every agent action gets logged with metadata. The key addition is a feedback field that captures whether the output was accepted, modified, or rejected.
#!/usr/bin/env python3
# agent_logger.py
import json
import sqlite3
from datetime import datetime
from pathlib import Path
DB_PATH = Path.home() / ".agent_memory" / "feedback.db"
def init_db():
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(DB_PATH)
conn.execute("""
CREATE TABLE IF NOT EXISTS agent_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
timestamp TEXT NOT NULL,
task TEXT NOT NULL,
action_taken TEXT,
output_summary TEXT,
feedback TEXT,
correction TEXT,
context_hash TEXT
)
""")
conn.execute("""
CREATE INDEX IF NOT EXISTS idx_task ON agent_logs(task)
""")
conn.execute("""
CREATE INDEX IF NOT EXISTS idx_context ON agent_logs(context_hash)
""")
conn.commit()
conn.close()
def log_action(task, action_taken, output_summary, feedback=None, correction=None, context_hash=None):
"""Log an agent action with optional feedback."""
init_db()
conn = sqlite3.connect(DB_PATH)
conn.execute("""
INSERT INTO agent_logs (timestamp, task, action_taken, output_summary, feedback, correction, context_hash)
VALUES (?, ?, ?, ?, ?, ?, ?)
""", (datetime.utcnow().isoformat(), task, action_taken, output_summary, feedback, correction, context_hash))
conn.commit()
conn.close()
def get_feedback_for_task(task, limit=10):
"""Retrieve recent feedback for a similar task."""
init_db()
conn = sqlite3.connect(DB_PATH)
rows = conn.execute("""
SELECT timestamp, feedback, correction, output_summary
FROM agent_logs
WHERE task = ? AND feedback IS NOT NULL
ORDER BY timestamp DESC
LIMIT ?
""", (task, limit)).fetchall()
conn.close()
return rows
# Initialize on import
init_db()
The context_hash is important -- it lets you match similar situations even when the exact task description differs. Hash the relevant variables (file paths, user type, operation type) to create a stable key.
Store: Structured Correction Records
Raw feedback is not enough. The agent needs to understand what was wrong and why. Store corrections as structured records.
def store_correction(task, original_output, corrected_output, reason):
"""Store a correction with explanation of what changed and why."""
init_db()
conn = sqlite3.connect(DB_PATH)
changes = []
if isinstance(original_output, str) and isinstance(corrected_output, str):
if len(corrected_output) != len(original_output):
changes.append(f"length: {len(original_output)} -> {len(corrected_output)}")
if original_output in corrected_output:
changes.append("content preserved, modifications applied")
elif corrected_output in original_output:
changes.append("content removed")
else:
changes.append("content replaced")
conn.execute("""
INSERT INTO agent_logs (timestamp, task, action_taken, output_summary, feedback, correction, context_hash)
VALUES (?, ?, ?, ?, ?, ?, ?)
""", (
datetime.utcnow().isoformat(),
task,
"CORRECTION",
f"Original: {original_output[:100]}...",
"rejected",
json.dumps({
"reason": reason,
"original": original_output[:500],
"corrected": corrected_output[:500],
"changes": changes
}),
None
))
conn.commit()
conn.close()
The correction field stores a structured JSON object with the full context. When the agent retrieves this, it can understand not just that something was wrong, but what specifically and why.
Retrieve: Feedback-Aware Context Injection
Before taking action on a task, the agent retrieves relevant past feedback and injects it into context.
def build_context_with_feedback(task, current_context):
"""Build enhanced context by injecting relevant past feedback."""
recent = get_feedback_for_task(task, limit=5)
if not recent:
return current_context
feedback_lines = []
for row in recent:
ts, feedback, correction, summary = row
if feedback == "rejected" and correction:
try:
corr_data = json.loads(correction)
feedback_lines.append(
f"PAST MISTAKE: {corr_data.get('reason', 'unspecified')}. "
f"Changes made: {', '.join(corr_data.get('changes', []))}"
)
except (json.JSONDecodeError, TypeError):
feedback_lines.append(f"PAST MISTAKE: {correction[:200]}")
elif feedback == "accepted":
feedback_lines.append(f"PAST SUCCESS: {summary[:100]}")
if feedback_lines:
feedback_block = "\n".join(feedback_lines)
enhanced_context = current_context + f"\n\n## Relevant Past Feedback\n{feedback_block}\n"
return enhanced_context
return current_context
This is the key integration point. Before the agent generates output, inject the feedback block. It changes nothing about how the agent reasons -- it just gives it memory of what worked and what did not.
Putting it together: the feedback-aware agent
#!/usr/bin/env python3
# feedback_agent.py
import hashlib
import json
from agent_logger import log_action, build_context_with_feedback, store_correction
def run_task(task, prompt_fn):
"""Run a task with feedback-aware context."""
context_hash = hashlib.md5(task.encode()).hexdigest()[:8]
base_context = f"Task: {task}\nInstructions: Complete this task following best practices."
full_context = build_context_with_feedback(task, base_context)
result = prompt_fn(full_context)
log_action(
task=task,
action_taken="completed",
output_summary=result[:200],
context_hash=context_hash
)
return result
def review_and_feedback(task, original_result, reviewed_result):
"""Review an agent result and store feedback."""
if original_result == reviewed_result:
log_action(
task=task,
action_taken="reviewed",
output_summary=original_result[:200],
feedback="accepted"
)
return
store_correction(
task=task,
original_output=original_result,
corrected_output=reviewed_result,
reason="Human review found output needed modification"
)
What this actually looks like in practice
After a week of use, the feedback table has entries like:
PAST MISTAKE: Always check file permissions before overwriting.
Changes made: content replaced
PAST MISTAKE: Do not delete files without confirmation.
Changes made: content replaced
PAST SUCCESS: Correctly formatted date as ISO 8601.
The next time the agent encounters a task involving file permissions or date formatting, it sees the feedback block. It does not make the same mistake again.
The discipline required
The feedback loop only works if you actually use it. Two things make it sustainable:
1. Make feedback effortless. The review step should take 10 seconds. If logging feedback takes longer than the task itself, you will not do it.
2. Review selectively. Not every output needs feedback. Flag the ones that were wrong, or particularly right. The goal is pattern recognition, not complete coverage.
The loop compounds. After a month, the agent makes fewer mistakes on the categories you care about most, because those are the ones you have given the most feedback on.
What it cannot fix
This pattern does not fix fundamental capability issues. If the agent consistently cannot understand a domain, no amount of feedback will teach it. The feedback loop works for pattern correction -- style, conventions, repeated mistakes -- not for building new reasoning capabilities.
For that, you need better prompts, better tools, or a better model.
More at https://thesolai.github.io
Top comments (0)