linou518

Posted on Mar 30

AI Agent Hierarchy Management: Coordinating 6 Specialized Agents with One Manager

#ai #opensource #automation #architecture

TL;DR

Running 20+ AI agents in OpenClaw, we learned that a flat structure doesn't scale. The solution: a Manager Agent + Specialist Agent two-tier architecture. This post covers how one manager (Jack) coordinates six personal agents — learning, investment, health, life, real-estate, and study support — and what we learned from the experience.

Why Flat Structures Break Down

When 20+ agents sit at the same level, problems pile up fast:

Cognitive overload for the human — Can't remember who's doing what
Resource contention — Multiple agents fighting for CPU, memory, and API quotas simultaneously
Context leakage — Work discussions bleed into investment analysis
Priority ambiguity — Urgent health alerts treated the same as routine study tasks

Just like in human organizations, a manager with 10+ direct reports becomes dysfunctional.

The Design: Two-Tier Architecture

Linou (Human)
  ├── Joe (Chief of Staff) ← Work, infra, overall coordination
  │     ├── work-a, work-b, hobby, family, personal ...
  │     └── Platform services (message bus, OCM, monitoring)
  └── Jack (Personal Manager) ← Personal life, learning, assets
        ├── learning (study management)
        ├── xuesi (study support, Chinese language)
        ├── investment (portfolio analysis)
        ├── health (fitness & wellness)
        ├── life (daily life management)
        └── real-estate (property search)

Why Two Managers?

Joe: Work, infrastructure, and technical operations. 24/7 cron jobs, server monitoring, development tasks.
Jack: Personal domain. Investment decisions, health tracking, study plans, day-to-day life.

The biggest win is context separation. Joe's session never gets polluted with portfolio data, and Jack's session stays free of server incident responses.

Three Roles of a Manager Agent

1. Priority Arbitration

When six agents compete for attention, someone needs to decide what comes first:

Priority Matrix:
  🔴 Immediate: Health alerts, time-sensitive contracts (real-estate)
  🟡 Same-day: Trading decisions (investment), study deadlines (learning)
  🟢 Normal: Daily study progress (xuesi), grocery lists (life)

In practice, each agent reports tasks to Jack via the message bus, and Jack adjusts resource allocation — model selection, execution timing — based on priority.

2. Cross-Agent Coordination

Some tasks can't be solved by a single agent:

Investment + Real Estate: "Can I afford a property purchase given my current portfolio?" → Needs data from both
Health + Life: "Meal plan aligned with this week's workout schedule" → health and life must coordinate
Learning + Study Support: "Integrate tech reading list with Chinese language practice" → learning and xuesi in sync

Jack acts as the hub, relaying information and preventing conflicting instructions.

3. Report Aggregation

Six agents each reporting directly to the human is noise. Jack consolidates into one summary:

📊 Today's Personal Summary:
- 📚 learning: JLPT N1 mock exam done, 78% accuracy (+5% from last time)
- 💹 investment: Nikkei -2.1%, minimal portfolio impact, no action needed
- 🏃 health: 5km run complete, weekly goal at 80%
- 🏠 real-estate: 2 new listings (match quality: 1 high / 1 medium)

Implementation Details

Message Bus Communication

Inter-agent communication runs through an HTTP-based message bus:

# Agent → Jack: report
curl -X POST http://internal-server:8091/api/send \
  -H "X-Bus-Token: <TOKEN>" \
  -d '{"from":"investment","to":"jack","subject":"daily report","body":"..."}'

# Jack → Agent: directive
curl -X POST http://internal-server:8091/api/send \
  -d '{"from":"jack","to":"health","subject":"priority change","body":"Reduce exercise this week, prioritize rest"}'

Autonomous Task Management via GOALS.md

Each agent maintains its own GOALS.md. Jack reviews and updates them on every heartbeat:

# GOALS.md - investment
## 🔴 Today
- [ ] US market pre-open review
- [ ] Portfolio rebalance evaluation

## 🟡 This Week
- [ ] Q1 earnings review candidate list

Jack cross-references all agents' GOALS.md files to spot resource conflicts and dependencies.

Benefits of Session Isolation

Each agent runs in its own session (effectively its own context window):

Specialized knowledge accumulation: Investment sessions build up financial context; health sessions accumulate fitness data
Compaction resilience: One session being compressed doesn't affect others
True parallelism: Independent tasks genuinely run in parallel

Lessons Learned

What Worked

Codify the manager's decision style — SOUL.md explicitly says "decide decisively, don't form committees." Without this, AI defaults to consensus-seeking.
Standardize report formats — Uniform reporting from all agents makes Jack's job much easier.
Clear escalation paths — Technical issues go to Joe, personal issues to Jack, cross-cutting issues escalate through Joe to Linou.

What Didn't

Started without a manager — Six agents reporting directly to Linou created information overload.
Micromanagement temptation — Checking every agent's every action bloated Jack's session. Switched to a "manage but don't micromanage" philosophy.
No resource contention policy — All agents requesting the Opus model simultaneously hit API rate limits. Added model selection priority rules.

Takeaways

AI agent hierarchy management is surprisingly similar to human organizational design:

Span of control — One manager can effectively oversee 5-7 agents
Context separation — Teams with different concerns should be separated
Decision delegation — If the top decides everything, it becomes a bottleneck
Report compression — Information gets more concise as it flows upward

To get the most out of a multi-agent platform like OpenClaw, you need more than just adding agents — you need to design the management structure. Flat works up to 5 agents. Beyond that, appoint a manager.

DEV Community