How to Set Up Synthetic Evals for Non-Deterministic Support Agent Workflows

Testing AI support agents is tricky because they rarely produce the exact same output twice. Traditional unit tests break immediately when applied to non-deterministic workflows, such as customer refund requests, order updates, or troubleshooting steps. By generating synthetic data to simulate edge-case conversations and evaluating the agent against predefined assertions, you can build reliable pass-rate tests and production guardrails without waiting for real users to break your system.

Python 3.10 or higher installed
An OpenAI API key or access to an alternative LLM provider
A dataset framework or library like Promptfoo, Ragas, or custom Python scripts
Basic understanding of JSON schemas and function calling

Step 1: Generate Synthetic Personas and Test Cases

Real user logs are useful, but they rarely cover every edge case. You need a separate script to generate synthetic user personas with varying tones, misspellings, and incomplete information.

Create a script named generate_synthetic_data.py to produce a structured JSON file of test cases:

import json
from openai import OpenAI

client = OpenAI()

system_prompt = """
You are an expert QA engineer creating test cases for a customer support AI.
Generate 5 diverse customer queries for a return policy workflow.
Include varying emotional states, typos, missing order numbers, and edge cases.
Return a valid JSON array of objects with keys: "id", "user_message", "expected_intent", "should_escalate".
"""

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": system_prompt}],
    response_format={"type": "json_object"}
)

with open("synthetic_tests.json", "w") as f:
    f.write(response.choices[0].message.content)

print("Synthetic test cases generated successfully.")

Run this script to create your initial eval dataset.

Step 2: Define Assertions and Evaluator Metrics

Exact string matching will not work here. Instead, write an LLM-as-a-judge evaluation function that reviews the support agent's output against your criteria.

If you are working on custom AI agent development, focus on three main metrics: policy adherence, tone, and tool choice accuracy.

Create a judge function in eval_judge.py:

def evaluate_response(user_query, agent_response, expected_intent):
    judge_prompt = f"""
    Evaluate the following support agent response based on the customer query.

    Customer Query: {user_query}
    Expected Intent: {expected_intent}
    Agent Response: {agent_response}

    Criteria:
    1. Did the agent address the query accurately according to standard return rules?
    2. Was the tone professional and helpful?
    3. Did it avoid making up fake policies?

    Reply with 'PASS' if all criteria are met, or 'FAIL' followed by a short reason.
    """

    res = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": judge_prompt}]
    )

    output = res.choices[0].message.content.strip()
    return "PASS" in output, output

Step 3: Run Pass Rate Testing Across Iterations

Run your support agent through all synthetic test cases and calculate the baseline pass rate.

import json

with open("synthetic_tests.json", "r") as f:
    data = json.load(f)
    test_cases = data.get("test_cases", data)

passed = 0
total = len(test_cases)

for test in test_cases:
    # Replace this with your actual support agent function call
    agent_output = "I can help with that return! Please provide your order ID." 

    is_pass, reason = evaluate_response(test["user_message"], agent_output, test["expected_intent"])
    if is_pass:
        passed += 1
    else:
        print(f"Failed Test {test['id']}: {reason}")

pass_rate = (passed / total) * 100
print(f"\nOverall Pass Rate: {pass_rate:.1f}%")

Whenever you update your system prompts or retrieval context, re-run this test suite. If the pass rate drops below your benchmark (for example, 90%), block the deployment.

Step 4: Hook Evals into Production Guardrails

Synthetic evals should not live only in your test environment. Run light, fast assertions directly inside your live workflow automation setup.

If a live response fails critical assertions (such as detecting toxic language or hallucinatory refund promises), interrupt the execution chain immediately and route the user to a human representative.

Expected Outcomes

After completing this setup, you can expect:

A repeatable methodology to score non-deterministic responses mathematically.
Clear pass rate metrics (e.g., target 92% accuracy across 100+ synthetic scenarios).
Prevention of prompt regressions before code hits production.
Real-time guardrails that prevent rogue LLM outputs from reaching actual customers.

If you need help building reliable AI pipelines or scaling custom agents for your business, the developers at Gaper can help you architect and deploy production-grade systems tailored to your technical stack.

DEV Community