Understanding AI Agent Architecture - Part 2

I'm currently reading AI Agents: The Definitive Guide by Nicole Koenigstein for a book club. In an effort to solidify my learnings and get the most out of the book, I'm noting things that stand out to me as I read through each chapter. Maybe this can also provide some quick knowledge to someone else who's interested in learning about AI agents :)

Chapter 2 of this book was titled 'Architecture and Patterns: Planning, Reactivity, and Multi-Agent Systems'. This chapter focused on agent workflows, including using multi-agent systems to improve the outputs and capabilities of these systems as well as the design patterns these systems can follow.

1. Chain of thought (CoT) prompting is a way to have the model provide reasoning steps as it plans to return an output. This can give "transparency over the reasoning process" as well as improving the accuracy of the output from a model due to the attention to smaller parts of a complex task.

2. Tree-of-thoughts (ToT) prompting is another prompting technique used to prompt the model to provide multiple possible plans of action that can be evaluated by either the same model or a different model to determine the best path forward. The book discusses this as an "engineering shortcut" that can eliminate the need for a full multi-agent system, because the design is simplified and it avoids "repeated calls to large reasoning models when a smaller setup already provides reliable results".

3. Handoffs define the passing of control between agents: "who speaks next, who acts next, and under what conditions". In LangGraph, nodes would be the agent logic, and the edges would be the handoffs between the nodes. For example, a node would handoff the path it took and why it took that path (reasoning) so that the next nodes can understand the decision making and use that to make more informed decisions as the control flow moves through the system. This can be represented in shared state values (defined & added by the developer) that are updated each time the system hands off control between agents.

4. ReAct loop - this chapter introduced the ReAct loop and how it is used as the basis for agent workflows. ReAct stands for reasoning and action. "ReAct treats each step as a choice between thinking in language and acting in the environment". The 3 pieces of the ReAct flow are reasoning, acting, and observing. This loop allows for the agent to have more specific reasoning, since it can adjust its tool use or decision making in each iteration of the loop.

5. Human-in-the-loop workflows are an important part of the agents we use today. These allow for safe execution and a human to be present in the workflow process. A human-in-the-loop step allows a human to step in to approve or reject tool calls or other action executions, review or edit the current state, and validate human input to the agent. This can be implemented in LangGraph using the interrupt() function with an input requirement before execution can continue. If an agent requires multiple human validations, parallel input can be collected by interrupting multiple flows with specific IDs assigned to the interrupts. LangGraph provides the Command(resume=...) that can take in a dictionary of ids mapping interrupts to responses.

6. Supervisor and hierarchical agents are two ways to handle a complex agent workflow. Supervisor agents have multiple agents that report to them. They decide who should act next. Hierarchical agents provide a more complex hierarchy of agents where a supervisor can supervise other supervisors. A hierarchical multi-agent system can use "routing to decide which specialized agent or subteam should act next", and choose the best agent from multiple levels of the hierarchy. In LangGraph, this can be implemented by creating many nodes for each agent, building the graph with edges between the nodes to map the hierarchy, adding a 'super graph' that treats each agent team as a node in that graph, and adding helper functions that decide which node executes next.

7. Swarm of agents is the final concept discussed in this chapter. These are multiple agents that also work together but might not be organized in a hierarchy. Instead, they might run parallel tasks and participate in more peer-to-peer collaboration. It's important to design handoffs between these agents as well, which can be created as handoff tools using the create_handoff_tool method. When each agent, or node, is defined, that handoff tool can be passed into the agent as part of its tools array. Internally, LangGraph uses a Command that switches the execution to the new agent. The create_swarm method actually instantiates the swarm, passing in the agents and a default_active_agent to begin the execution. Swarms allow for more flexibility in the MAS execution flow.

My biggest takeaway from this chapter was the thought process of creating a multi-agent system and the paradigms of how multiple agents can work together to complete a task. These concepts explain the architecture of many AI products we use today for both business and personal use cases (e.g., Muse, OpenAI Dots, Claude Code, Codex, etc.). This chapter provided a way to conceptualize the structure and patterns that these systems can operate with as well as the patterns they can use to strengthen accuracy and output from the language model as well as efficiency from the agent system. Excited to learn more in the next chapters. Thanks for reading!