Day 20/30: Agents in Disagreement

I recently worked on a project where two independent agents, AgentA and AgentB, were tasked with generating and reviewing product descriptions. The goal was for AgentA to produce an initial description, and then AgentB would review and critique it, providing feedback that AgentA could use to improve its output. However, I quickly realized that the agents often disagreed on what constituted a "good" description, leading to an infinite loop of revisions with no clear resolution.

The issue arose because each agent had its own understanding of the product and its target audience, shaped by their respective training data and objectives. AgentA focused on highlighting the product's technical features, while AgentB emphasized its benefits and user experience. As a result, AgentB would consistently point out that AgentA's descriptions were too dry and technical, while AgentA would argue that AgentB's suggestions were too vague and marketing-oriented.

To break this cycle, I introduced a review/critique loop between the two agents, using LangGraph's StateGraph to model their interactions and MCP's tools module to facilitate their communication. Here's an example of how I implemented this loop in Python:

import langgraph as lg
from mcp import tools

# Define the agents and their objectives
agent_a = lg.Agent("AgentA", objective="generate_product_description")
agent_b = lg.Agent("AgentB", objective="review_product_description")

# Create a StateGraph to model the agents' interactions
graph = lg.StateGraph()

# Add nodes for the agents' states
graph.add_node("agent_a_initial", agent_a.initial_state)
graph.add_node("agent_b_review", agent_b.initial_state)

# Add conditional edges to model the review/critique loop
graph.add_conditional_edges(
    ("agent_a_initial", "agent_b_review", lambda x: x["review"] == "pending"),
    ("agent_b_review", "agent_a_initial", lambda x: x["revision"] == "requested")
)

# Define the critique function for AgentB
def critique(description):
    # Simulate AgentB's review process
    review = tools.generate_text("Review of product description", context=description)
    return review

# Define the revision function for AgentA
def revise(description, review):
    # Simulate AgentA's revision process
    revised_description = tools.generate_text("Revised product description", context=description, feedback=review)
    return revised_description

# Run the review/critique loop
while True:
    # Get the current state of the agents
    agent_a_state = graph.get_node("agent_a_initial")
    agent_b_state = graph.get_node("agent_b_review")

    # Generate an initial description with AgentA
    description = agent_a.generate_text("Product description")

    # Review the description with AgentB
    review = critique(description)

    # Check if the review is satisfactory
    if review["satisfactory"] == True:
        break

    # Revise the description with AgentA
    revised_description = revise(description, review)

    # Update the agents' states
    graph.update_node("agent_a_initial", agent_a_state, revised_description)
    graph.update_node("agent_b_review", agent_b_state, review)

    # Checkpoint the current state
    tools.checkpoint("review_critique_loop", graph.get_state())

This implementation allows the agents to engage in a iterative review and revision process, with AgentB providing feedback on AgentA's output and AgentA incorporating that feedback into its next iteration. The StateGraph models the agents' interactions, and the tools module facilitates their communication and provides utilities for generating text and managing the review/critique loop.

One practical gotcha I encountered while implementing this loop was the need to carefully manage the agents' states and ensure that they were properly synchronized. If the agents' states become desynchronized, the review/critique loop can become stuck or produce inconsistent results. To avoid this, I used LangGraph's checkpointers module to regularly save the current state of the agents and the StateGraph, allowing me to easily recover in case of errors or inconsistencies.

As we continue to explore the possibilities of agentic AI, we'll delve deeper into the challenges of coordinating multiple agents and ensuring that they work together effectively towards a common goal. Tomorrow, we'll examine how to scale up our agent-based systems to tackle more complex tasks and domains.