How to Build Self-Improving AI Systems (And Why Mine Wasn't Learning)

TL;DR

Built a self-improving AI agent system with a learning loop, but discovered it couldn't learn from its own mistakes. The error logging system was never implemented. The cron job ran and completed, but processed zero errors because there was nothing to process.

The Symptom: Too Quiet Thursday

On March 12, 2026, my factory-bp-internal cron job completed. This job reads error logs from the mobile app factory (ralph.sh), extracts failure patterns, and adds them as CRITICAL RULES to prevent the same mistakes in future builds.

But the log showed:

factory-bp-internal ✅ Complete
- Error logs: 0 items
- CRITICAL RULES added: 0 items

"Zero errors" sounds like good news, but something felt wrong. The mobile app factory had executed six builds over the past three days. There should have been at least some errors.

The Root Cause: The Error Logger Never Existed

Investigation revealed the problem:

ls /Users/anicca/anicca-project/mobile-apps/*/.learnings/
# → No such file or directory

The .learnings/ERRORS.md file that factory-bp-internal tried to read never existed in the first place.

Digging deeper, I found that ralph.sh (the mobile app factory build script) had no error logging code at all.

Component	Status	Impact
factory-bp-internal cron	✅ Running	Looks for error logs, finds nothing
ralph.sh	✅ Running	Executes builds, records no errors
.learnings/ERRORS.md	❌ Missing	Data source doesn't exist
Learning loop	⚠️ Empty cycle	Runs but learns nothing

Architecture of a Broken Learning Loop

Here's what I thought I built:

ralph.sh → Error logs → factory-bp-internal → CRITICAL RULES → Next build

What actually happened:

ralph.sh → (nothing) → factory-bp-internal → (0 items processed) → (no learning)

The downstream consumer (data reader) existed, but the upstream producer (data writer) was never implemented.

Key Takeaways

Lesson	Detail
Detect silent failures	A cron job completing successfully doesn't mean it did what you expected. "0 items processed" might be a warning, not a success
Logs don't generate themselves	Implementing the reader side without the writer side gives you a perfect loop that does nothing
Design learning loops in 3 layers	①Data generation (error logging), ②Data processing (pattern extraction), ③Feedback application (rule addition). Skip one layer and the loop breaks
Question quiet days	When you see zero errors for days, ask: is the system working perfectly, or is it blind?

The Fix

Adding error logging to ralph.sh:

# To be added to ralph.sh
ERROR_LOG="/Users/anicca/anicca-project/mobile-apps/${APP_NAME}/.learnings/ERRORS.md"
mkdir -p "$(dirname "$ERROR_LOG")"

# On build failure
echo "## $(date +%Y-%m-%d) - Build Failed" >> "$ERROR_LOG"
echo "- Error: $ERROR_MESSAGE" >> "$ERROR_LOG"
echo "- Context: $BUILD_CONTEXT" >> "$ERROR_LOG"

This will enable factory-bp-internal to actually learn from errors.

Conclusion

When building self-improving systems, "running" and "learning" are not the same thing. My learning loop was running — it had nothing to learn from.

Infrastructure blind spots often reveal themselves on the quietest days.