<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: WTFearB</title>
    <description>The latest articles on DEV Community by WTFearB (@wtfearb).</description>
    <link>https://dev.to/wtfearb</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4253928%2F3c26dded-5ad8-45b7-ac69-99adb9bc45ad.png</url>
      <title>DEV Community: WTFearB</title>
      <link>https://dev.to/wtfearb</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/wtfearb"/>
    <language>en</language>
    <item>
      <title>How to Architect a Production Multi-Agent System Without Hallucinations (And When Not to Use an LLM)</title>
      <dc:creator>WTFearB</dc:creator>
      <pubDate>Thu, 19 Nov 2026 13:36:11 +0000</pubDate>
      <link>https://dev.to/wtfearb/how-to-architect-a-production-multi-agent-system-without-hallucinations-and-when-not-to-use-an-llm-2l1h</link>
      <guid>https://dev.to/wtfearb/how-to-architect-a-production-multi-agent-system-without-hallucinations-and-when-not-to-use-an-llm-2l1h</guid>
      <description>&lt;p&gt;Most AI agent demos fail the moment they touch real business data. Not because the underlying models aren't capable, but because engineers treat probabilistic LLMs like deterministic microservices.&lt;/p&gt;

&lt;p&gt;At &lt;a href="https://wtfearb.tech" rel="noopener noreferrer"&gt;WTFearB&lt;/a&gt;, our core engineering philosophy is simple: &lt;strong&gt;Make What If Work&lt;/strong&gt;. When founders and operations teams ask, &lt;em&gt;"What if the agent hallucinates a refund?"&lt;/em&gt; or &lt;em&gt;"What if latency spikes under load?"&lt;/em&gt;, the answer isn't a longer system prompt—it's better software architecture.&lt;/p&gt;

&lt;p&gt;Here is the exact blueprint we use when engineering &lt;a href="https://wtfearb.tech/services/ai-automation" rel="noopener noreferrer"&gt;custom AI workflow automation&lt;/a&gt; and multi-agent systems for production environments.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Do Multi-Agent Systems Fail in Production?
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Multi-agent systems fail in production primarily due to unbounded state loops, unvalidated tool outputs, and routing simple deterministic logic through expensive LLMs.&lt;/strong&gt; Without strict state machines and schema enforcement, small reasoning errors compound across every agent handoff.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Failure Mode&lt;/th&gt;
&lt;th&gt;Demo Approach (Fragile)&lt;/th&gt;
&lt;th&gt;Production Architecture (Resilient)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Intent Routing&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Asking an LLM to decide every next step&lt;/td&gt;
&lt;td&gt;Deterministic classifier / semantic router first; LLM only for ambiguity&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tool Execution&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Raw JSON parsing from free-form text&lt;/td&gt;
&lt;td&gt;Strict Pydantic / JSON Schema validation with automatic retry budgets&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Memory &amp;amp; Context&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Stuffing entire chat histories into the prompt&lt;/td&gt;
&lt;td&gt;Scoped state graphs + hybrid vector/graph retrieval (Qdrant + Neo4j)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;High-Stakes Actions&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Full autonomy on writes/payments&lt;/td&gt;
&lt;td&gt;Confidence thresholds + async human-in-the-loop approval queues&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  1. Stop Paying LLM Latency for Deterministic Decisions
&lt;/h2&gt;

&lt;p&gt;Before spinning up an orchestrator, separate your workflow into &lt;strong&gt;deterministic steps&lt;/strong&gt; and &lt;strong&gt;reasoning steps&lt;/strong&gt;. If a routing decision can be solved with a regex, a SQL lookup, or a lightweight embedding similarity check, keep the LLM out of the critical path.&lt;/p&gt;

&lt;p&gt;Here is a lightweight pattern using Python and Pydantic to enforce deterministic routing before invoking an agent:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;enum&lt;/span&gt; &lt;span class="nf"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;Enum&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;RouteTarget&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Enum&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="no"&gt;BILLING_SQL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"billing_sql"&lt;/span&gt;
    &lt;span class="no"&gt;TECH_RAG&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"tech_rag"&lt;/span&gt;
    &lt;span class="no"&gt;HUMAN_ESCALATION&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"human_escalation"&lt;/span&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;RoutingDecision&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;target&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;RouteTarget&lt;/span&gt;
    &lt;span class="n"&gt;confidence&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ge&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;le&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;1.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;requires_approval&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;bool&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;fast_gatekeeper&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;RoutingDecision&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="s2"&gt;"""Handle obvious deterministic routes in &amp;lt;5ms before calling an LLM."""&lt;/span&gt;
    &lt;span class="n"&gt;q&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nb"&gt;any&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;q&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"invoice"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"receipt"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"gst"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"tax id"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;RoutingDecision&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;target&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;RouteTarget&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;BILLING_SQL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;confidence&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.98&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;requires_approval&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nb"&gt;any&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;q&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"cancel contract"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"legal"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"breach"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;RoutingDecision&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;target&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;RouteTarget&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;HUMAN_ESCALATION&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;confidence&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;1.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;requires_approval&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;  &lt;span class="c1"&gt;# Fall back to LLM router only for ambiguous queries&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This single gatekeeper layer typically cuts token costs by &lt;strong&gt;35%–50%&lt;/strong&gt; and drops median response times below 200ms for routine operational queries.&lt;/p&gt;




&lt;h2&gt;
  
  
  2. Enforce State Machines Over Free-Form Agent Loops
&lt;/h2&gt;

&lt;p&gt;Giving five agents access to a shared group chat and hoping they converge is a recipe for infinite token burn. Instead, model your workflow as a &lt;strong&gt;Directed Acyclic Graph (DAG)&lt;/strong&gt; or a bounded state graph (using frameworks like LangGraph or Temporal):&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Explicit State Schema:&lt;/strong&gt; Define every variable an agent can read or mutate in a typed state object.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hard Iteration Caps:&lt;/strong&gt; Set &lt;code&gt;max_steps=5&lt;/code&gt; on every subgraph. If an agent hasn't resolved the task within five hops, fail gracefully and route to a fallback handler.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Idempotent Tools:&lt;/strong&gt; Every external API call (CRM updates, email triggers, ERP entries) must accept an idempotency key so retries never create duplicate records.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  3. Ground Retrieval with Hybrid Vector + Graph Context
&lt;/h2&gt;

&lt;p&gt;Pure vector search struggles when queries require multi-hop relationships—for example, &lt;em&gt;"Which enterprise clients on Plan B had an unresolved support ticket in the last 14 days?"&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;In our &lt;a href="https://wtfearb.tech/services/agentic-ai" rel="noopener noreferrer"&gt;agentic AI development&lt;/a&gt; builds, we pair:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Vector Search (Qdrant / pgvector):&lt;/strong&gt; For unstructured documentation, SOPs, and call transcripts.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Graph / Relational Lookups (Neo4j / PostgreSQL):&lt;/strong&gt; For exact entity relationships, permissions, and account hierarchies.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Always attach &lt;strong&gt;source chunk IDs&lt;/strong&gt; to every retrieved fact and require the synthesis agent to cite them in its structured output. If no chunk supports a claim, the model must return &lt;code&gt;"insufficient_context"&lt;/code&gt; rather than guessing.&lt;/p&gt;




&lt;h2&gt;
  
  
  4. Production Readiness Checklist Before Launch
&lt;/h2&gt;

&lt;p&gt;Before shipping any AI agent to live users or internal operations teams, verify these five controls:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[ ] &lt;strong&gt;Structured Outputs Only:&lt;/strong&gt; Zero regex scraping on markdown responses.&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Scoped Permissions:&lt;/strong&gt; Read-only database credentials for retrieval agents; separate write-scoped service accounts behind approval gates.&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Prompt Injection Filtering:&lt;/strong&gt; Input sanitization before user-supplied text enters tool-calling contexts.&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Traceability:&lt;/strong&gt; Full token, latency, and tool-call logging per trace ID.&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Graceful Degradation:&lt;/strong&gt; Automatic fallback to deterministic workflows if upstream model APIs rate-limit or time out.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  About WTFearB
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href="https://wtfearb.tech" rel="noopener noreferrer"&gt;WTFearB&lt;/a&gt;&lt;/strong&gt; (&lt;em&gt;What The Fear, Boss / Business&lt;/em&gt;) is a custom AI software development and business automation agency. Guided by our core philosophy—&lt;strong&gt;Make What If Work&lt;/strong&gt;—we engineer production-grade multi-agent AI systems, custom LLM workflows, and full-stack enterprise SaaS platforms for ambitious businesses.&lt;/p&gt;

&lt;p&gt;Have a complex operational workflow you want to automate without fragile prototypes? &lt;strong&gt;&lt;a href="https://wtfearb.tech" rel="noopener noreferrer"&gt;Explore how we build at wtfearb.tech&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>agents</category>
      <category>python</category>
      <category>architecture</category>
    </item>
  </channel>
</rss>
