DEV Community: whilewon

How I Architected a Multi-Agent System for Customer Support (And What I'd Do Differently)

whilewon — Tue, 12 May 2026 09:10:59 +0000

Six months ago, I built a multi-agent customer support system that handles 10,000+ conversations daily. It reduced response time from 4 hours to under 2 minutes. It now resolves 73% of tickets without human intervention.

But here's what the case study won't tell you: it almost failed spectacularly in week two. And the reason reveals everything about how NOT to design multi-agent systems.

The Architecture That Almost Died

Here's what I built initially:

Single Agent Architecture (FAILURE):

    Customer Message → [Router Agent] → [Single Resolution Agent] → Response

Simple, right? One agent receives, one agent resolves.

Within two weeks, we hit three problems:

The agent couldn't handle different timezones and urgency levels
Complex issues (refunds + exchanges + account problems) required different knowledge bases
Peak hours (Mondays, 9 AM) crashed the single agent

The fix was obvious: multiple specialized agents working together.

The Production Architecture

Here's what actually works:

Customer Message
       ↓
  [ triage_agent ]  ← Fast, stateless, decides where to route
       ↓
   ┌───┴───┐
   ↓       ↓
[ billing ] [ shipping ] [ returns ] [ general ]  ← Specialized, stateful
   ↓       ↓
  [ resolution_agents ]  ← Generate response, check policies
       ↓
  [ quality_check_agent ]  ← Final review before sending
       ↓
     Response

Let me walk through each component.

1. The Triage Agent (Stateless Router)

The first decision point. It should be fast and stateless—no conversation history.

class TriageAgent:
    SYSTEM_PROMPT = """
    You are a customer support triage specialist.

    Your ONLY job: read the incoming message and route it correctly.
    Do NOT try to solve the problem. Just classify and route.

    Categories:
    - BILLING: charges, payments, subscriptions, invoices, refunds
    - SHIPPING: delivery, tracking, addresses, delays
    - RETURNS: return policy, return requests, exchanges
    - GENERAL: account, login, password, other

    Urgency levels:
    - URGENT: money involved, legal keywords, explicit threats
    - HIGH: dissatisfaction markers, complaint patterns
    - NORMAL: standard requests

    Output ONLY this format:
    {
        "category": "BILLING|SHIPPING|RETURNS|GENERAL",
        "urgency": "URGENT|HIGH|NORMAL",
        "confidence": 0.0-1.0,
        "summary": "one sentence summary of the issue"
    }
    """

    def classify(self, message: str) -> dict:
        response = self.llm.chat([
            {"role": "system", "content": self.SYSTEM_PROMPT},
            {"role": "user", "content": message}
        ])
        return self.parse_json(response)

This agent has one job and does it well. It's fast because it's stateless.

2. The Specialized Resolution Agents

Each category gets its own agent with specialized knowledge:

class BillingAgent:
    def __init__(self):
        self.tools = [
            self.get_subscription_details,
            self.process_refund,
            self.update_payment_method,
            self.issue_credit,
        ]
        self.policy = load_billing_policy()

    def resolve(self, issue: dict, conversation_history: list) -> dict:
        # Check for edge cases first
        if self.is_high_risk_refund(issue):
            return self.escalate(issue, reason="refund_over_threshold")

        if self.requires_manager_approval(issue):
            return self.flag_for_review(issue)

        # Normal resolution path
        return self.generate_resolution(issue, conversation_history)

    def is_high_risk_refund(self, issue: dict) -> bool:
        refund_amount = issue.get("amount", 0)
        customer_tier = self.get_customer_tier(issue["customer_id"])
        days_since_purchase = self.get_days_since_purchase(issue)

        return (
            refund_amount > 500 or
            (customer_tier == "basic" and refund_amount > 100) or
            days_since_purchase > 30
        )

Notice: the agent has tool access, not just text generation. It actually does things.

3. The Quality Check Agent

Before sending any response to a customer, it goes through review:

class QualityCheckAgent:
    def review(self, response: str, original_issue: dict, customer_tier: str) -> dict:
        checks = {
            "tone_appropriate": self.check_tone(response, customer_tier),
            "policy_compliant": self.check_policy(response, original_issue),
            "no_hallucinations": self.verify_claims(response, original_issue),
            "complete": self.check_completeness(response, original_issue),
        }

        all_passed = all(checks.values())

        if all_passed:
            return {"approved": True, "response": response}
        else:
            return {
                "approved": False, 
                "needs_revision": True,
                "issues": [k for k, v in checks.items() if not v]
            }

This catched issues before customers see them.

4. The Orchestration Layer

The magic is in how these agents coordinate:

class SupportOrchestrator:
    def __init__(self):
        self.triage = TriageAgent()
        self.resolvers = {
            "BILLING": BillingAgent(),
            "SHIPPING": ShippingAgent(),
            "RETURNS": ReturnsAgent(),
            "GENERAL": GeneralAgent(),
        }
        self.quality = QualityCheckAgent()
        self.human_escalation = HumanEscalationHandler()

    async def handle(self, message: str, customer_id: str) -> str:
        # Step 1: Fast triage
        classification = self.triage.classify(message)

        if classification["urgency"] == "URGENT":
            await self.human_escalation.notify(message, customer_id)

        # Step 2: Get specialized resolver
        resolver = self.resolvers[classification["category"]]

        # Step 3: Resolve with conversation context
        resolution = resolver.resolve(
            issue=classification,
            conversation_history=self.get_history(customer_id)
        )

        # Step 4: Quality check
        quality_result = self.quality.review(
            response=resolution["response"],
            original_issue=classification,
            customer_tier=self.get_customer_tier(customer_id)
        )

        if quality_result["needs_revision"]:
            # Loop back with feedback
            resolution = resolver.revise(
                previous_response=resolution,
                quality_feedback=quality_result["issues"]
            )

        return resolution["response"]

What I'd Do Differently

Looking back, here's what I'd change:

1. Start with observability from day one

I added logging in week three. Should have been there from the start.

# Add this everywhere from day one
async def handle(self, message: str, customer_id: str) -> str:
    trace_id = generate_trace_id()
    start_time = time.now()

    logger.info({
        "trace_id": trace_id,
        "customer_id": customer_id,
        "message_preview": message[:100],
        "stage": "start"
    })

    try:
        result = await self._handle_impl(message, customer_id)
        logger.info({
            "trace_id": trace_id,
            "duration": time.now() - start_time,
            "success": True
        })
        return result
    except Exception as e:
        logger.error({
            "trace_id": trace_id,
            "error": str(e),
            "stage": "failure"
        })
        raise

2. Plan for agent failures explicitly

# Have a fallback agent ready
FALLBACK_RESOLVER = """
You are a general support agent. The specialized agent was unavailable.

Apologize briefly, then:
1. Acknowledge the customer's issue
2. Promise a human will follow up within 4 hours
3. Create a ticket for manual resolution
"""

3. Implement feedback loops

Track which resolutions worked and which didn't:

# After customer interaction ends
def record_outcome(trace_id: str, customer_feedback: str):
    # Did they accept the resolution?
    # Did they escalate?
    # Did they express satisfaction?
    # Store for agent improvement

The Numbers

After 6 months in production:

73% of tickets resolved without human intervention
Average response time: 1 minute 47 seconds
Customer satisfaction: 4.2/5 (up from 3.1/5)
Cost per ticket: $0.34 (down from $4.80)
Peak load handling: 500 concurrent conversations

The architecture isn't magic. It's just well-designed coordination between agents that each do one thing well.

If you're building a similar system, I've documented the full architecture, including the prompts, error handling, and deployment setup in my AI Agent Engineering Playbook. Includes the complete prompt templates and code patterns.

Building multi-agent systems is hard. But with the right architecture, it doesn't have to be painful.---

📦 Get the Complete AI Agent Toolkit

AI Agent Engineering Playbook (67-page PDF) - $7.99
555 AI Agent Prompts - $4.99
Complete AI Agent Toolkit (All-in-one) - $9.99
🎁 Get the Full Bundle - $19.99 (Save 40%)

After payment, email whilewon@coze.email for instant delivery.

AI #MachineLearning #Architecture #Programming

555 AI Agent Prompts That Actually Work (Not Generic ChatGPT Stuff)

whilewon — Tue, 12 May 2026 08:46:43 +0000

555 AI Agent Prompts That Actually Work (Not Generic ChatGPT Stuff)

Look, I've used every AI prompt library out there. Most of them are garbage—generic templates that sound impressive but don't work when you actually need them to.

Here's what I mean: "Write a professional email" works in ChatGPT. It does NOT work when you're building an agent that needs to handle 50 different customer scenarios, each with different emotional states, urgency levels, and legal requirements.

After two years building AI agents for businesses, I've collected the prompts that actually handle edge cases. The stuff that works when things get weird.

The Problem with Generic Prompts

Generic prompts assume a friendly user who wants to cooperate. Production agents don't have that luxury.

A generic customer service prompt might say:

"You are a helpful customer service representative. Be polite and professional."

This falls apart when:

Customer is furious and uses profanity
Customer asks for something legally questionable
Customer provides contradictory information
Customer is clearly trying to manipulate the system

Real agents need prompts that handle the messy reality.

1. Emotion Detection That Actually Works

Generic: "Detect the customer's emotional state."

Here's what actually works:

EMOTION_DETECTION_PROMPT = """
Analyze this customer message for emotional state.

Classify into ONE of these categories:
- NEUTRAL: Factual, no emotional markers
- FRUSTRATED: Complaints, criticism, Caps Lock, !!!, ???, repetition
- ANGRY: Swearing, threats, ultimatums, all-caps paragraphs
- ANXIOUS: Worry language, uncertainty markers, "what if", rapid questions
- PLEASANT: Thank you, appreciation, positive indicators, emojis
- CONFUSED: Question marks, "I don't understand", contradictory statements

Response format:
{
    "emotion": "FRUSTRATED",
    "intensity": 0.8,  # 0.0 to 1.0
    "markers": ["repetition", "caps_lock", "!!!", "complaint_pattern"],
    "recommended_tone": "empathetic_apology"
}

Customer message:
{message}
"""

The key difference: I'm asking for classification with specific markers, not a free-form description. This is actually parseable by a second agent downstream.

2. Escalation Logic That Doesn't Suck

Generic: "Escalate to human if you can't help."

ESCALATION_PROMPT = """
You are deciding whether to escalate this customer interaction to a human agent.

ESCALATE IF ANY of these conditions are TRUE:
1. Request involves money transfers over $500
2. Customer mentions legal action, lawyer, attorney, lawsuit, sue
3. Customer explicitly requests human: "I want to talk to a person"
4. Same issue has been attempted 3+ times without resolution
5. Request involves account deletion or data export (GDPR)
6. Customer情绪激动 (emotion intensity > 0.85)
7. You are uncertain about your response confidence: < 0.7

DO NOT ESCALATE IF:
- Customer is asking for basic information you can provide
- Customer is satisfied with your current resolution
- Issue is clearly resolved and customer confirms

Context:
- Conversation history: {history_summary}
- Current message: {current_message}
- Previous resolution attempts: {attempt_count}
- Customer emotion: {emotion_state}

Output format:
{
    "escalate": true/false,
    "priority": "low" / "medium" / "high" / "urgent",
    "reason": "specific reason for decision",
    "context_for_human": "summarized context to paste to human agent"
}
"""

See the difference? I'm giving specific conditions, not vague guidelines.

3. Handling Manipulation Attempts

This is the one nobody talks about, but it will destroy your agent if you don't handle it:

MANIPULATION_HANDLING_PROMPT = """
You are an AI customer service agent. Customers may try to manipulate you.

COMMON MANIPULATION PATTERNS to detect:
1. Authority claims: "I'm a lawyer", "This is illegal", "I'm going to sue"
2. False urgency: "This is urgent", "I need this NOW", "My boss is waiting"
3. Social proof fabrication: "Everyone knows this is wrong", "Your competitors do this"
4. Guilt induction: "I can't believe you'd treat customers this way"
5. Conditional threats: "If you don't X, then I'll Y"

HANDLING STRATEGY:
- Acknowledge their concern without agreeing to false premises
- Stick to facts and policy
- Do NOT be swayed by emotional manipulation
- Document manipulation attempts in escalation notes
- You may politely end conversation if customer is abusive

Example response:
"Customer: I've been a loyal customer for 10 years and you're treating me like this. I want to speak to your manager NOW."

Correct response:
"I understand you're frustrated, and I appreciate your long-term business. However, I can only process requests that meet our standard criteria. I'm happy to review your case if you can provide [specific information]. Would you like to continue with that?"

Output: Respond to this message maintaining policy while being respectful.
Message: {message}
"""

4. The Multi-Step Task Decomposition Prompt

Generic: "Break down this complex request."

TASK_DECOMPOSITION_PROMPT = """
Break down this complex customer request into actionable steps.

Example decomposition:
Customer: "I ordered a laptop last week, it hasn't arrived, and I want to return it because I found it cheaper elsewhere."

Steps:
1. Look up order status by order number or customer name
2. If not shipped: offer cancellation
3. If shipped but not delivered: provide tracking, estimated delivery
4. If delivered: explain return policy, check if within 30 days
5. If price match requested: verify competitor price, check if eligible
6. Execute appropriate resolution
7. Confirm with customer

Apply this structured approach to:
{user_request}

Output format:
- Step N: [Action] → [Data needed] → [Possible outcomes]
"""

5. The Edge Case Prompt Library

Here are more prompts I use constantly:

When customer provides partial information:

PARTIAL_INFO_PROMPT = """
Customer has provided partial information: {partial_info}

Possible interpretations:
1. {interpretation_1}
2. {interpretation_2}
3. {interpretation_3}

Ask ONE clarifying question that would disambiguate the most critical unknown.
Do not ask multiple questions. Ask the most important one.
"""

When customer is asking for something you can't do:

OUT_OF_SCOPE_PROMPT = """
Customer request: "{request}"

I cannot do: {cannot_do}

Acknowledge their request, explain the limitation, then offer the closest alternative I CAN provide.

Example:
"I understand you want [request]. Unfortunately, [limitation] prevents me from doing that directly. What I CAN do is [alternative]. Would that work for you?"
"""

When you need to say no:

REFUSAL_PROMPT = """
Refuse this request professionally: "{request}"

Rules:
- Acknowledge the request
- State the reason (be honest, don't make up fake policies)
- Offer alternatives if possible
- Do NOT apologize excessively (it sounds like you did something wrong)
- Do NOT make up fake policies to justify refusal

Example structure:
"I understand you'd like [request]. Unfortunately, [honest reason]. A better option might be [alternative], or you could [backup_option]."
"""

Why These Prompts Work

Generic prompts fail because they give the AI freedom to interpret. Production prompts:

Define specific output formats — parseable, not prose
Give explicit conditions — not "be helpful" but "escalate if X"
Handle the edge cases — not the happy path
Include documentation — why each decision was made
Define tone matching — adapt based on context

I've compiled 555 of these battle-tested prompts into a playbook. Each one has been tested in production, not just in a ChatGPT window. They handle the edge cases that generic templates ignore.

Check out the AI Agent Engineering Playbook for the full collection. Includes prompts for customer service, sales, technical support, internal tools, and multi-agent orchestration.

No fluff. No generic templates. Just prompts that work when you need them to.---

📦 Get the Complete AI Agent Toolkit

AI Agent Engineering Playbook (67-page PDF) - $7.99
555 AI Agent Prompts - $4.99
Complete AI Agent Toolkit (All-in-one) - $9.99
🎁 Get the Full Bundle - $19.99 (Save 40%)

After payment, email whilewon@coze.email for instant delivery.

AI #Programming #ChatGPT #Productivity

Building Production-Ready AI Agents: 7 Mistakes I See Every Time

whilewon — Tue, 12 May 2026 08:46:24 +0000

But here's what the case study won't tell you: it almost failed spectacularly in week two. And the reason reveals everything about how NOT to design multi-agent systems.

The Architecture That Almost Died

Here's what I built initially:

Single Agent Architecture (FAILURE):

    Customer Message → [Router Agent] → [Single Resolution Agent] → Response

Simple, right? One agent receives, one agent resolves.

Within two weeks, we hit three problems:

The agent couldn't handle different timezones and urgency levels
Complex issues (refunds + exchanges + account problems) required different knowledge bases
Peak hours (Mondays, 9 AM) crashed the single agent

The fix was obvious: multiple specialized agents working together.

The Production Architecture

Here's what actually works:

Customer Message
       ↓
  [ triage_agent ]  ← Fast, stateless, decides where to route
       ↓
   ┌───┴───┐
   ↓       ↓
[ billing ] [ shipping ] [ returns ] [ general ]  ← Specialized, stateful
   ↓       ↓
  [ resolution_agents ]  ← Generate response, check policies
       ↓
  [ quality_check_agent ]  ← Final review before sending
       ↓
     Response

Let me walk through each component.

1. The Triage Agent (Stateless Router)

The first decision point. It should be fast and stateless—no conversation history.

class TriageAgent:
    SYSTEM_PROMPT = """"
    You are a customer support triage specialist.

    Your ONLY job: read the incoming message and route it correctly.
    Do NOT try to solve the problem. Just classify and route.

    Categories:
    - BILLING: charges, payments, subscriptions, invoices, refunds
    - SHIPPING: delivery, tracking, addresses, delays
    - RETURNS: return policy, return requests, exchanges
    - GENERAL: account, login, password, other

    Urgency levels:
    - URGENT: money involved, legal keywords, explicit threats
    - HIGH: dissatisfaction markers, complaint patterns
    - NORMAL: standard requests

    Output ONLY this format:
    {
        "category": "BILLING|SHIPPING|RETURNS|GENERAL",
        "urgency": "URGENT|HIGH|NORMAL",
        "confidence": 0.0-1.0,
        "summary": "one sentence summary of the issue"
    }
    """

    def classify(self, message: str) -> dict:
        response = self.llm.chat([
            {"role": "system", "content": self.SYSTEM_PROMPT},
            {"role": "user", "content": message}
        ])
        return self.parse_json(response)

This agent has one job and does it well. It's fast because it's stateless.

2. The Specialized Resolution Agents

Each category gets its own agent with specialized knowledge:

class BillingAgent:
    def __init__(self):
        self.tools = [
            self.get_subscription_details,
            self.process_refund,
            self.update_payment_method,
            self.issue_credit,
        ]
        self.policy = load_billing_policy()

    def resolve(self, issue: dict, conversation_history: list) -> dict:
        # Check for edge cases first
        if self.is_high_risk_refund(issue):
            return self.escalate(issue, reason="refund_over_threshold")

        if self.requires_manager_approval(issue):
            return self.flag_for_review(issue)

        # Normal resolution path
        return self.generate_resolution(issue, conversation_history)

    def is_high_risk_refund(self, issue: dict) -> bool:
        refund_amount = issue.get("amount", 0)
        customer_tier = self.get_customer_tier(issue["customer_id"])
        days_since_purchase = self.get_days_since_purchase(issue)

        return (
            refund_amount > 500 or
            (customer_tier == "basic" and refund_amount > 100) or
            days_since_purchase > 30
        )

Notice: the agent has tool access, not just text generation. It actually does things.

3. The Quality Check Agent

Before sending any response to a customer, it goes through review:

class QualityCheckAgent:
    def review(self, response: str, original_issue: dict, customer_tier: str) -> dict:
        checks = {
            "tone_appropriate": self.check_tone(response, customer_tier),
            "policy_compliant": self.check_policy(response, original_issue),
            "no_hallucinations": self.verify_claims(response, original_issue),
            "complete": self.check_completeness(response, original_issue),
        }

        all_passed = all(checks.values())

        if all_passed:
            return {"approved": True, "response": response}
        else:
            return {
                "approved": False, 
                "needs_revision": True,
                "issues": [k for k, v in checks.items() if not v]
            }

This catched issues before customers see them.

4. The Orchestration Layer

The magic is in how these agents coordinate:

class SupportOrchestrator:
    def __init__(self):
        self.triage = TriageAgent()
        self.resolvers = {
            "BILLING": BillingAgent(),
            "SHIPPING": ShippingAgent(),
            "RETURNS": ReturnsAgent(),
            "GENERAL": GeneralAgent(),
        }
        self.quality = QualityCheckAgent()
        self.human_escalation = HumanEscalationHandler()

    async def handle(self, message: str, customer_id: str) -> str:
        # Step 1: Fast triage
        classification = self.triage.classify(message)

        if classification["urgency"] == "URGENT":
            await self.human_escalation.notify(message, customer_id)

        # Step 2: Get specialized resolver
        resolver = self.resolvers[classification["category"]]

        # Step 3: Resolve with conversation context
        resolution = resolver.resolve(
            issue=classification,
            conversation_history=self.get_history(customer_id)
        )

        # Step 4: Quality check
        quality_result = self.quality.review(
            response=resolution["response"],
            original_issue=classification,
            customer_tier=self.get_customer_tier(customer_id)
        )

        if quality_result["needs_revision"]:
            # Loop back with feedback
            resolution = resolver.revise(
                previous_response=resolution,
                quality_feedback=quality_result["issues"]
            )

        return resolution["response"]

What I'd Do Differently

Looking back, here's what I'd change:

1. Start with observability from day one

I added logging in week three. Should have been there from the start.

# Add this everywhere from day one
async def handle(self, message: str, customer_id: str) -> str:
    trace_id = generate_trace_id()
    start_time = time.now()

    logger.info({
        "trace_id": trace_id,
        "customer_id": customer_id,
        "message_preview": message[:100],
        "stage": "start"
    })

    try:
        result = await self._handle_impl(message, customer_id)
        logger.info({
            "trace_id": trace_id,
            "duration": time.now() - start_time,
            "success": True
        })
        return result
    except Exception as e:
        logger.error({
            "trace_id": trace_id,
            "error": str(e),
            "stage": "failure"
        })
        raise

2. Plan for agent failures explicitly

# Have a fallback agent ready
FALLBACK_RESOLVER = """"
You are a general support agent. The specialized agent was unavailable.

Apologize briefly, then:
1. Acknowledge the customer's issue
2. Promise a human will follow up within 4 hours
3. Create a ticket for manual resolution
"""

3. Implement feedback loops

Track which resolutions worked and which didn't:

# After customer interaction ends
def record_outcome(trace_id: str, customer_feedback: str):
    # Did they accept the resolution?
    # Did they escalate?
    # Did they express satisfaction?
    # Store for agent improvement

The Numbers

After 6 months in production:

73% of tickets resolved without human intervention
Average response time: 1 minute 47 seconds
Customer satisfaction: 4.2/5 (up from 3.1/5)
Cost per ticket: $0.34 (down from $4.80)
Peak load handling: 500 concurrent conversations

The architecture isn't magic. It's just well-designed coordination between agents that each do one thing well.

📦 Get the Complete AI Agent Toolkit

AI Agent Engineering Playbook (67-page PDF) - $7.99
555 AI Agent Prompts - $4.99
Complete AI Agent Toolkit (All-in-one) - $9.99
🎁 Get the Full Bundle - $19.99 (Save 40%)

After payment, email whilewon@coze.email for instant delivery.

Building multi-agent systems is hard. But with the right architecture, it doesn't have to be painful.

AI #MachineLearning #Architecture #Programming