Agentic Architectures — Article 2: Advanced Coordination and Reasoning Patterns

Solving the “Stochastic Parrot” Problem with Structured Logic

There’s a criticism of large language models that has stuck around since 2021, and it still stings a little: the “stochastic parrot” argument. The idea is that LLMs are sophisticated pattern-matchers that produce statistically plausible text without any genuine understanding behind it. They’re parroting, not reasoning.

I’m not here to settle that philosophical debate. What I am here to tell you is this: if your agentic system behaves like a stochastic parrot — confidently producing plausible-sounding but wrong answers, failing to backtrack when it hits a dead end, unable to break a hard problem into manageable pieces — the fix is almost never the model. It’s the architecture.

The difference between an agent that looks intelligent in a demo and one that stays intelligent in production comes down to coordination and reasoning patterns. How does your agent plan? How does it check its own work? How do multiple agents share what they know without drowning each other in JSON?

That’s what this article is about.

Dynamic Planning: From Static Chains to Hierarchical Thinking

The first generation of “agentic” products were really just dressed-up chains. You’d define a fixed sequence of LLM calls — summarize, then classify, then respond — and call it a pipeline. It worked for simple, predictable tasks. It fell apart the moment the real world showed up.

Real tasks are rarely linear. A user asking “research our top three competitors and draft a positioning document” doesn’t map cleanly to a fixed sequence of steps. The number of competitors might be two or five. Each competitor might require a different depth of research. The positioning document might need a complete rewrite after the research reveals something unexpected.

What you need is Hierarchical Planning  — a “Manager” agent that treats the task as a problem to be decomposed, not a script to be executed.

The pattern works like this:

User Task
    └── Manager Agent (Planner)
            ├── Sub-task A → Worker Agent 1
            ├── Sub-task B → Worker Agent 2
            └── Sub-task C → Worker Agent 3
                    └── Sub-sub-task C1 → Worker Agent 3a

The Manager receives the top-level goal and produces a structured plan — a list of sub-tasks with dependencies, assigned roles, and success criteria. Worker agents execute their assigned sub-tasks and report results back. The Manager synthesizes the results, evaluates whether the goal has been met, and either delivers the final output or replans if something went wrong.

The critical implementation detail that most tutorials skip: the plan must be a living document, not a frozen spec. If Worker Agent 2 comes back with an unexpected result — say, a competitor has already pivoted out of your market — the Manager needs to update the plan in response. A Manager that rigidly executes the original plan in the face of new information isn’t planning; it’s just executing a slightly fancier chain.

In practice, this means storing the plan in a mutable shared state that the Manager can read and rewrite between steps. LangGraph handles this elegantly with its state graph model. CrewAI has a more opinionated take with its hierarchical process mode. Both work — the choice depends on how much control you want over the graph structure (more on that shortly).

Fractal Chain-of-Thought: Reasoning That Zooms In

Standard Chain-of-Thought prompting — “think step by step before answering” — is one of the most reliable techniques for improving LLM reasoning quality. But it has a ceiling. For deeply complex problems, a flat sequence of reasoning steps runs out of resolution. The model is reasoning about the right things at the wrong granularity.

Fractal Chain-of-Thought (FCoT) addresses this by making reasoning recursive. When an agent encounters a sub-problem that is itself complex enough to warrant multi-step reasoning, it spawns a nested reasoning process rather than trying to resolve it in a single step.

Think of it like a zoom function. The top-level reasoning operates at the problem level:

Problem: Optimize our database query performance
  Step 1: Identify the slow queries
  Step 2: Analyze the execution plans
  Step 3: Propose index changes
  Step 4: Estimate performance impact

But Step 2 — “analyze the execution plans” — is itself a multi-step reasoning problem that deserves its own chain:

Sub-problem: Analyze execution plan for Query #7
  Step 2.1: Identify full table scans
  Step 2.2: Check join order efficiency
  Step 2.3: Evaluate predicate pushdown opportunities
  Step 2.4: Flag missing statistics

And Step 2.3 might zoom in further still.

The implementation is cleaner than it sounds. You give the agent a tool called something like deep_reason(sub_problem: str) -> str that recursively invokes the same reasoning architecture on the sub-problem. The result gets folded back into the parent reasoning chain. You set a maximum recursion depth (3-4 levels is usually plenty) to prevent infinite descent.

The payoff is significant for domains with nested complexity — legal analysis, systems debugging, financial modeling. The cost is proportionally higher token usage. FCoT is a targeted tool, not a default setting.

The Reflection Pattern: Building a Critic That Actually Criticizes

Here’s a failure mode that bites almost every team eventually: you implement a self-review step where the same model that generated an output also reviews it. The model gives itself a pass. Every time. The “reflection” becomes a rubber stamp.

This happens because LLMs are, to put it charitably, optimistic about their own work. The same statistical patterns that produced the original output will evaluate it favorably. You’ve built a conflict of interest into your architecture.

The fix is the Critic Pattern , and the key design principle is model diversity.

Generator (Model A) → Output → Critic (Model B) → Feedback → Generator → Revised Output

Using a different model for the critic role — Claude reviewing GPT-4o output, or Gemini reviewing Claude output — introduces genuine perspective diversity. Each model has different training data emphases, different failure modes, and different stylistic biases. A cross-model critic is far more likely to catch errors that the generator is systematically blind to.

The Critic agent should be given a structured evaluation rubric, not a vague “review this” prompt. A good rubric for a code-generating agent might look like:

• Correctness : Does the code do what the spec requires?
• Edge cases : Are null inputs, empty collections, and boundary values handled?
• Security : Are there injection vectors, exposed secrets, or unsafe deserialization?
• Readability : Would a mid-level engineer understand this without comments?
• Test coverage : Are the happy path and at least two failure paths tested?

The Critic returns a structured response — pass/fail per criterion, plus specific feedback for each failure. The Generator receives this structured feedback and revises. This continues until all criteria pass or the maximum iteration count is reached.

One underrated implementation detail: give the Critic explicit permission to fail things. If your Critic prompt says “review this and suggest improvements,” you’ll get suggestions. If it says “your job is to find reasons this should not ship — be adversarial,” you’ll get a real review.

State Machines vs. DAGs: Choosing Your Control Flow Model

This is the question that causes more architecture debates than almost any other in the multi-agent space, and the answer is genuinely context-dependent.

Directed Acyclic Graphs (DAGs) model workflows that flow in one direction without cycles. Task A feeds into Task B and Task C; B and C feed into Task D; done. This is the natural model for pipelines where each step produces input for the next and you never need to revisit a completed step.

CrewAI’s sequential and hierarchical processes are essentially DAG-based. Temporal workflows are explicitly DAG-structured. They’re excellent for deterministic, well-understood workflows where the shape of the computation is known in advance.

Cyclic graphs (State Machines) allow loops — the ability to return to a previous state based on new information. This is what LangGraph was purpose-built for, and it’s the right model for any agent that needs to:

• Retry a failed tool call with modified parameters
• Return to a planning step after discovering the current plan won’t work
• Run a reflection loop until quality criteria are met
• Wait for human approval before proceeding

The decision rule I’ve converged on after shipping several production systems:

Does your agent ever need to go backwards?
                           / \
                         YES NO
                          | |
                     LangGraph CrewAI / Temporal
                  (cyclic graph) (DAG model)

“Going backwards” means any scenario where the correct next step depends on the outcome of a previous step in a way that might require revisiting earlier work. Reflection loops go backwards. Replanning goes backwards. Waiting for a human to approve and then resuming goes backwards.

If your workflow is genuinely linear — always the same steps, always in the same order, with no branching based on intermediate results — a DAG model is simpler and easier to reason about. But be honest with yourself about whether your workflow is actually linear or whether you’re just assuming it will be.

The infrastructure implications differ significantly:

                  DAG Model Cyclic / State Machine
-----------------------------------------------------------------------------------------------
Execution model Step functions / pipelines Long-running stateful process
State management Passed between steps Persisted in graph state store
Debugging Linear trace, easy to follow Requires full state inspection per node
Scalability Each step independently Entire graph runs in one execution context
Failure recovery Retry from last step Checkpoint and resume from last stable state
Cost predictability High (bounded steps) Lower (loop count is variable)

Shared Epistemic Memory: The Blackboard Architecture

Here’s a scaling problem that hits every team that gets beyond three or four agents: how do agents share what they know?

The naive approach is to pass everything as function arguments — the output of Agent A becomes the input to Agent B as a large JSON blob. This works until it doesn’t. The blob grows. Context windows fill up. You start seeing agents with 80% of their context window consumed by state they don’t actually need for their specific sub-task.

The sophisticated approach is the Blackboard Architecture  — a pattern borrowed from classical AI and distributed systems that is experiencing a quiet renaissance in the agentic era.

The concept is simple: instead of passing state between agents directly, all agents read from and write to a shared “blackboard” — a structured, queryable state store that sits outside any individual agent.

                        ┌─────────────────┐
                        │ BLACKBOARD │
                        │ (Shared State) │
                        └────────┬────────┘
                    ┌────────────┼────────────┐
                    ↓ ↓ ↓
              Agent A Agent B Agent C
            (reads/writes) (reads/writes) (reads/writes)

Each agent reads only the sections of the blackboard relevant to its current task. Each agent writes its outputs back to designated sections. No agent needs to know what other agents are doing — it just needs to know the schema of the blackboard.

In practice, the blackboard is typically implemented as:

• A structured document in a database (DynamoDB, Redis, or Postgres with JSONB) for fast key-based access to specific state sections
• A vector store for semantic retrieval when agents need to find relevant context without knowing the exact key
• A message log for ordered history that agents can replay or summarize

The schema design of your blackboard is one of the most important architectural decisions you’ll make. Too flat and agents can’t find what they need without reading everything. Too nested and updates become complex. A layered approach works well: top-level sections for task metadata, agent outputs, shared knowledge, and execution history.

One design principle worth emphasizing: write provenance into every blackboard entry. Every piece of information written to the shared state should include which agent wrote it, when, and with what confidence level. When a downstream agent reads a fact and makes a decision based on it, you want to be able to trace that decision back to its source when something goes wrong.

Production Reality Check

The patterns in this article are genuinely powerful. They’re also genuinely expensive, and the cost compounds in ways that aren’t obvious until you’re staring at a cloud bill.

Let’s put some real numbers on it.

A Reflection loop that runs an average of 2 iterations doubles your model call count. If your base cost is $0.05 per task, reflection takes it to $0.10. That sounds manageable — until you’re handling 50,000 tasks per day, at which point reflection alone adds $2,500/day in model costs.

Fractal Chain-of-Thought with 3 levels of recursion and an average of 4 steps per level generates roughly 64 reasoning steps (⁴³) for a single complex query. At even modest token counts per step, this can push a single query cost into the $0.50–$2.00 range. Reserve it for problems that actually need that depth.

Cross-model Critic patterns (e.g., Claude reviewing GPT-4o) introduce a second API dependency with its own latency, rate limits, and cost curve. Budget for both. More importantly, test what happens when the Critic’s API goes down — your system should degrade gracefully, not grind to a halt.

The honest question to ask before adding any coordination pattern is: what’s the quality delta, and what’s it worth?

Reflection improving accuracy from 78% to 91% on a customer-facing recommendation engine that drives revenue? Worth the cost. Reflection improving accuracy from 94% to 96% on an internal summarization tool that saves analysts 10 minutes a day? Probably not.

Measure first. Add complexity second.

What Comes Next

In Article 3, we shift from how agents think to how agents fail — and more importantly, how you detect and recover from those failures before your users do.

AgentOps is the unglamorous but absolutely essential discipline of treating your AI system like the distributed system it actually is: with observability, guardrails, eval pipelines, and human checkpoints baked in from the start. Not bolted on after the first production incident.

Because there will be a production incident. There always is.

                  L1: Stateless L2: Tool-Augmented L3: Autonomous L4: Multi-Agent L5: Self-Correcting
------------------------------------------------------------------------------------------------------------------------------------------------------
Execution Serverless / Edge Serverless + integr. Long-running container Distributed orchestrator Distributed + feedback loops
State None None Short + long-term memory Shared state across agents State + mutation history
Latency profile Predictable Slightly variable Variable (loop-dependent) High, parallelizable Highest, bounded by budget
Cost model Linear (tokens) Linear + tool costs Nonlinear (calls per task) Nonlinear × agent count Nonlinear × iteration count
Primary failure Bad retrieval Tool hallucination Context overflow Cascade failures Runaway loops
Observability Basic logging Tool call tracing Full trace per loop Cross-agent tracing Cost + quality dashboards

Included for reference from Article 1 — the coordination patterns in this article map primarily to L3, L4, and L5.

This is Article 2 of a 4-part series on Agentic AI Architectures. Ready for Article 3 — AgentOps — whenever you are.