From RAG to RAO Level 6: How I Evolved Tiramisu Framework into a Multi-Agent System

Three weeks ago, I published Tiramisu Framework v1.0 — a simple RAG system for marketing consultancy.
Today, I'm releasing v2.0 — a complete RAO Level 6 multi-agent system with memory, auto-correction, and MCP protocol support.
This is the story of how I evolved it in 2 days (and what I learned building a production-ready AI framework).

🎯 TL;DR
bashpip install tiramisu-framework==2.0.0
What's new in v2.0:

✅ Real multi-agent architecture (not simulated)
✅ 100% accurate intelligent routing
✅ Contextual memory (Redis + semantic)
✅ Auto-correction & validation
✅ MCP-ready (agent-discoverable)
✅ RAO Level 6 complete

🔗 GitHub
🔗 PyPI
📧 [e-mail]

⚠️ Important Legal Notice
The consultant names (Philip Kotler, Gary Vaynerchuk, Martha Gabriel) used throughout this article are for illustrative and educational purposes only to demonstrate the multi-agent architecture concept.
The actual Tiramisu Framework v2.0 distributed on PyPI is a generic, customizable system where you:

✅ Add your own knowledge base and documents
✅ Define your own expert personas and personalities
✅ Configure your own agent behaviors and specializations
✅ Use any domain experts relevant to your use case

No proprietary content, copyrighted materials, or brand names are included in the distributed package.
The framework provides the architecture and orchestration; you provide the content and expertise.
Think of it as a template: we show "Strategy Expert + Social Media Expert + Tech Expert" as an example, but you could create "Legal Expert + Financial Expert + HR Expert" or any other combination for your domain.

📊 The Evolution: v1.0 → v2.0
Featurev1.0 (RAG Basic)v2.0 (RAO Level 6)ArchitectureSingle LLMMulti-Agent SystemExpertsSimulated (prompts)Real agents (independent code)RoutingNoneHybrid Supervisor (keywords + LLM)Routing AccuracyN/A100% (tested 50+ queries)MemorySQLite onlyRedis + Semantic patternsValidationManualAuto-correction (Auditor + Gatekeeper)Chunking800 chars1200 chars (40% better context)DiscoverabilityNoMCP-readyLines of Code~600~2,488

🧠 What is RAO? (And Why It Matters)
RAG (Retrieval-Augmented Generation) = Search + Generate
RAO (Reasoning + Acting + Orchestration) = Think + Do + Coordinate
RAO Levels (0-6):
Level 0-2: RAG (retrieval + generation)
Level 3: Memory (context between interactions) ✅
Level 4: Executor (real actions) ✅
Level 5: Multi-Agent (coordinated specialists) ✅
Level 6: MCP-ready (discoverable by other agents) ✅

Tiramisu v2.0 = Level 6 complete
Most RAG systems stop at Level 2. We went to 6.

🏗️ The New Architecture
v1.0 - Single LLM Approach:
User Query → FAISS Search → GPT-4 (simulates 3 experts) → Mixed Response
❌ Problem: All experts "spoke" at once. Generic, unfocused responses.
v2.0 - Multi-Agent System:
User Query
↓
Supervisor Agent (routes intelligently)
↓
Kotler Agent | Gary Vee Agent | Martha Agent
↓
Specialized FAISS search (filtered by expert)
↓
GPT-4 with expert personality
↓
Focused, expert response
✅ Result: Each agent maintains unique voice, expertise, and context.

💻 Code Comparison
v1.0 - Everything Mixed:
pythonfrom tiramisu import TiramisuRAG

rag = TiramisuRAG()
response = rag.analyze("How to improve Instagram?")

Returns: Mixed insights from all 3 experts

v2.0 - Intelligent Routing:
pythonfrom tiramisu.agents import TiramisuMultiAgent

system = TiramisuMultiAgent()
result = system.process("How to improve Instagram?")

print(result['consultant']) # "Gary" (social media expert)
print(result['response']) # 100% Gary Vee style!
The difference? v1.0 mixed everyone's opinion. v2.0 routes to the RIGHT expert.

🎯 Feature 1: Hybrid Supervisor (100% Accuracy)
The Challenge:
First attempt: Pure LLM routing.
python# ❌ This failed - sent EVERYTHING to Kotler
def route(query):
response = llm.invoke(f"Route this query: {query}")
return response # Always returned "Kotler"
Why? GPT-4 defaulted to the "strategic" expert for ambiguous queries.
The Solution: Hybrid Approach
pythonclass SupervisorAgent:
def route(self, query: str):
query_lower = query.lower()

    # Layer 1: Keywords (fast, 95% of cases)
    gary_keywords = ["instagram", "tiktok", "social", "content"]
    if any(kw in query_lower for kw in gary_keywords):
        return "Gary"

    martha_keywords = ["ai", "automation", "data", "tech"]
    if any(kw in query_lower for kw in martha_keywords):
        return "Martha"

    # Layer 2: LLM (complex cases)
    return self.llm_route(query)  # Fallback

Result: 100% accuracy on 50+ test queries.
Lesson: Hybrid beats pure LLM for classification tasks.

🧩 Feature 2: Real Multi-Agent Architecture
Each agent is independent code with:

Specialized FAISS search (filtered by expert)
Unique personality (temperature, tone, style)
Expert prompting (deep character simulation)

Example: Gary Vee Agent
pythonclass GaryAgent:
def init(self):
self.llm = ChatOpenAI(
model="gpt-4",
temperature=0.7 # More creative
)
self.style_prompt = """
You are Gary Vaynerchuk.

• DIRECT and NO BS
• Focus on EXECUTION
• ENERGETIC language
• Real examples

• Authentic content obsession
  """

  def search(self, query):
  # Filter FAISS: only Gary Vee content
  results = []
  for doc in faiss_results:
  if "gary" in doc['source'].lower():
  results.append(doc)
  return results
  Compare with Kotler Agent:
  pythonclass KotlerAgent:
  def init(self):
  self.llm = ChatOpenAI(
  model="gpt-4",
  temperature=0.3 # More conservative
  )
  self.style_prompt = """
  You are Philip Kotler.

• ANALYTICAL and STRUCTURED

• Based on FRAMEWORKS (4Ps, SWOT)

• ACADEMIC but accessible

• Long-term strategy focus
  """
  Result: Each agent has distinct voice, expertise, and behavior.

💾 Feature 3: Contextual Memory
The Problem:
User: "Tell me about Instagram strategy"
Bot: [responds]
User: "What about budget?"
Bot: "Budget for what?" ❌ Lost context!
The Solution: Dual Memory System

1. Short-term (Redis):
   pythonclass SessionMemory:
   def init(self):
   self.redis = redis.Redis()

   def add_interaction(self, session_id, query, response):
   key = f"session:{session_id}:history"
   self.redis.lpush(key, json.dumps({
   "query": query,
   "response": response,
   "timestamp": datetime.now()
   }))
   self.redis.expire(key, 3600) # 1 hour TTL

2. Long-term (Semantic patterns):
   pythonclass SemanticMemory:
   def detect_patterns(self, user_id):
   # Analyzes: frequent topics, preferences, style
   return {
   "preferred_consultant": "Gary",
   "topics": ["social media", "content"],
   "tone": "practical"
   }
   Result: Bot remembers context, adapts to user preferences.

✅ Feature 4: Auto-Correction (Auditor + Gatekeeper)
Input Validation (Gatekeeper):
pythonclass Gatekeeper:
def validate_query(self, query: str):
score = self.llm.invoke(f"""
Rate clarity (0-10): "{query}"

    Is it specific enough to answer?
    """)

    if score < 5:
        return {
            "valid": False,
            "clarification_needed": "Please specify..."
        }
    return {"valid": True}

Output Validation (Auditor):
pythonclass ResponseAuditor:
def audit(self, response: str, query: str):
scores = self.evaluate({
"completeness": "Does it fully answer?",
"accuracy": "Is it factually correct?",
"relevance": "Stays on topic?",
"actionability": "Provides clear actions?",
"expertise": "Matches consultant's style?"
})

    if scores['average'] < 7:
        return {"reprocess": True, "reason": "Low quality"}
    return {"approved": True}

Real example from tests:
Query: "Marketing strategy"
First response: Generic overview (score: 6.2)
Auto-correction triggered ✅
Second response: Specific 4Ps analysis (score: 8.7)
Lesson: Auto-validation dramatically improves output quality.

🔧 Feature 5: Optimized Chunking
The VUCA Problem (from another project):
Document: "VUCA means: Volatility, Uncertainty,
Complexity, and Ambiguity"

With chunk_size=800:
Chunk 1: "VUCA means: Volatility, Uncertainty"
Chunk 2: "Complexity, and Ambiguity"

Query: "What is VUCA?"
Result: Incomplete answer ❌
The Solution:
python# v1.0
chunk_size = 800
chunk_overlap = 150

v2.0

chunk_size = 1200 # +50% context
chunk_overlap = 200 # +33% safety margin
Result: Concepts like "4Ps", "SWOT", "Customer Journey" preserved completely.
Lesson: Larger chunks = better context preservation (within reason).

🌐 Feature 6: MCP-Ready (Agent Discoverable)
What if OTHER AI agents could discover and use Tiramisu?
python# MCP Protocol Support
@app.get("/agent/mcp/capabilities")
def get_capabilities():
return {
"framework": "Tiramisu",
"version": "2.0.0",
"capabilities": {
"marketing_analysis": {
"consultants": ["Strategy", "Digital", "Tech"],
"methods": ["analyze", "consult", "plan"],
"output_formats": ["json", "markdown", "structured"]
}
},
"endpoints": {
"analyze": "/agent/mcp/analyze",
"consultants": "/agent/mcp/consultants"
}
}
Result: Tiramisu is now discoverable by Claude, GPT, and other agents via MCP protocol.

📈 Performance Metrics
Response Time:
Simple query (1 agent): ~15s
Complex query (3 agents): ~30-40s
With auto-correction: +5-10s
Accuracy:
Routing accuracy: 100% (50+ queries tested)
Auto-correction triggers: ~12% of queries
Quality improvement: 40% (user feedback)
Memory:
Context retention: 5 interactions
Session duration: 1 hour (configurable)
Semantic patterns: Learned over time

🚧 Technical Challenges Solved
Challenge 1: Python 3.13 Incompatibility
Problem: FAISS doesn't support Python 3.13 yet.
Solution:
bash# Use Python 3.12
python3.12 -m venv venv
source venv/bin/activate
pip install tiramisu-framework
Lesson: Always check compatibility matrix for ML libraries.

Challenge 2: Pydantic Pickle Incompatibility
Problem: Metadata saved with Pydantic v1 couldn't load in v2.
Solution:
python# Rebuild metadata with current Pydantic version
def rebuild_metadata(old_pkl_path):
# Load raw data, reconstruct as dict, re-save
with open(old_pkl_path, 'rb') as f:
raw = pickle.load(f, encoding='latin1')

clean_data = [
    {"content": doc.content, "source": doc.source}
    for doc in raw if hasattr(doc, 'content')
]

with open(new_pkl_path, 'wb') as f:
    pickle.dump(clean_data, f)

Lesson: Avoid pickling Pydantic models; use JSON instead.

Challenge 3: FAISS Dimension Mismatch
Problem:
FAISS index: 3072 dimensions (text-embedding-3-large)
Default OpenAI: 1536 dimensions (text-embedding-ada-002)
AssertionError: Dimension mismatch!
Solution:
python# Always specify model explicitly
embeddings = OpenAIEmbeddings(
model="text-embedding-3-large" # 3072 dims
)
Lesson: Document your embedding model choice in README.

📚 What I Learned Building This

1. Multi-Agent ≠ Multiple Prompts
   Wrong approach:
   python# This is NOT multi-agent
   prompt = "Think like Kotler, then Gary, then Martha"
   response = llm(prompt)
   Right approach:
   python# Real multi-agent: separate code, memory, behavior
   kotler = KotlerAgent() # Independent
   gary = GaryAgent() # Independent
   martha = MarthaAgent() # Independent

2. Hybrid Systems Beat Pure LLM
   For routing, classification, validation:
   Keywords (fast, deterministic) + LLM (smart, flexible) = Best of both

3. Auto-Validation is a Game-Changer
   Before: Manual quality checks.
   After: System self-corrects automatically.
   ROI: 40% quality improvement, zero human intervention.

4. Chunking is Critical
   Too small = fragmented concepts.
   Too large = irrelevant noise.
   Sweet spot: 1200 chars with 200 overlap (for most use cases).

5. Memory Makes AI Feel "Real"
   Without memory: Bot feels robotic.
   With memory: Bot feels like a real consultant who remembers you.

🔮 What's Next: v3.0 Roadmap

GUI (Streamlit + Next.js dashboard)
More Agents (SEO, Email, Analytics, Branding)
Benchmarks (vs Perplexity, Claude, GPT)
One-click Deploy (Railway, Render, AWS)
CRM Integration (HubSpot, Salesforce)
Multi-language (Spanish, Portuguese)
A/B Testing (compare agent responses)

🛠️ Try It Now
Installation:
bashpip install tiramisu-framework==2.0.0
Quick Test:
pythonfrom tiramisu.agents import TiramisuMultiAgent

system = TiramisuMultiAgent()

Simple query

result = system.process("How to improve Instagram engagement?")
print(f"Consultant: {result['consultant']}") # "Gary"
print(f"Response: {result['response']}")

Complex query (multiple agents)

result = system.process_complex(
"I need a complete digital marketing strategy for a B2B SaaS startup"
)

Consults Kotler (strategy) + Gary (tactics) + Martha (tech)

print(result['response'])
Run API Server:
bash# Clone repo
git clone https://github.com/tiramisu-framework/tiramisu
cd tiramisu

Install

pip install -e .

Set API key

export OPENAI_API_KEY="your-key"

Run

uvicorn tiramisu.api.main:app --reload

Test

curl http://localhost:8000/api/analyze \
-H "Content-Type: application/json" \
-d '{"query": "Marketing strategy for SaaS"}'

📊 Real Example Output
Input:
B2B SaaS startup, $30k/month marketing budget,
need better lead quality from inbound channels
Output (Kotler + Gary + Martha synthesis):
markdown🌱 ROOTS (Kotler - Strategic Analysis)

• Current ICP unclear (mixing SMB + Enterprise)
• Value prop not differentiated enough
• CAC too high ($450) vs LTV ($3.2k)

🌳 TRUNK (Core Strategy)

• Refine ICP: Focus on 50-500 employee tech companies
• ABM approach: Target 100 high-fit accounts
• Content: Problem-aware → Solution-aware funnel

🍃 BRANCHES (Gary - Tactical Execution)
Week 1-2: LinkedIn thought leadership (3x/week)
Week 3-4: Case study content + webinars
Week 5-8: Retargeting + email nurture sequences
Budget: 60% content, 30% ads, 10% tools

🤖 TECH ENABLEMENT (Martha)

• HubSpot + Clearbit for enrichment
• Drift for qualification
• Mixpanel for behavior tracking
• Zapier for automation

KPIs:

• MQL → SQL: 40% → 60%
• CAC: $450 → $280
• Sales cycle: 45 → 30 days

🤝 Contributing
We're looking for contributors in:

Agent Development: New expert personalities
Frontend: React/Next.js dashboard
Testing: Automated test suites
Documentation: Tutorials, guides, videos
Integrations: CRMs, analytics tools

How to contribute:

Fork the repo
Create feature branch
Submit PR with tests
Join our Discord (coming soon)

📜 License & Business Model
Framework: MIT License (free, open-source)
Business Model:

Free: Core framework
Paid: Expanded knowledge bases, custom integrations, support, white-label

Why open source?

Transparency builds trust
Community accelerates innovation
Better product through feedback

📚 Resources
🔗 GitHub: tiramisu-framework/tiramisu
🔗 PyPI: pypi.org/project/tiramisu-framework/2.0.0/
📧 Email: [e-mail]
📖 Docs: [Coming soon]
💬 Discord: [Coming soon]

🙏 Acknowledgments
Built with:

LangChain (RAG orchestration)
OpenAI GPT-4 (LLM)
FAISS (vector search)
Redis (memory)
FastAPI (API)

Inspired by:

LlamaIndex (RAG patterns)
DSPy (structured prompting)
AutoGen (multi-agent concepts)

💬 Let's Connect
I'd love to hear:

What you build with Tiramisu
Feature requests
Technical challenges you face
Ideas for v3.0

Comment below or reach out:
📧 [e-mail]
🐙 @tiramisuframework

🎯 Final Thoughts
Three weeks ago, Tiramisu was a simple RAG system.
Today, it's a production-ready RAO Level 6 multi-agent framework with:

Real specialized agents
Intelligent routing (100% accuracy)
Contextual memory
Auto-correction
MCP protocol support

The journey from v1.0 to v2.0 taught me:

Multi-agent systems require architectural thinking
Hybrid approaches beat pure LLM
Auto-validation is essential for production
Memory transforms user experience
Open source accelerates innovation

What's your experience building RAG systems?
Have you tried multi-agent architectures?
Let's discuss in the comments! 👇

If you found this helpful, please ⭐ the GitHub repo and share with your network!

ai #python #opensource #rag #multiagent #llm #gpt4 #langchain #machinelearning #artificialintelligence

Comments

[tiramisu-framework]

Update: Tiramisu v2.0 is now live!
Complete RAO Level 6 multi-agent system.