Logging ReACT Pipelines using Maxim AI

What is ReACT?

ReACT (Reasoning + Acting) is an AI approach that integrates logical reasoning with actionable steps. It allows AI agents to process information, reason through problems, and execute tasks based on their understanding. This results in more dynamic and interactive AI systems capable of tackling complex tasks.

ReACT follows a cycle of thought, action, observation, and answer to produce accurate results. This iterative process ensures the agent adapts and refines its answers based on new observations.

Why Use ReACT?

Traditional AI systems often struggle with tasks requiring multi-step reasoning, as they provide static responses. ReACT, by contrast, enables:

• Thought Process: Breaking down problems step by step.
• Action Execution: Using tools or APIs to take action.
• Observation Integration: Adjusting based on the outcome of actions.
• Answer Generation: Delivering a reasoned response.

This methodology significantly improves the versatility and efficiency of AI agents, particularly in complex scenarios where simple, direct answers are insufficient.

Simple ReACT Workflow

Consider the query: “What is the mass of Earth times 2?”

1. Thought: The AI needs the mass of Earth.
2. Action: It queries Wikipedia for the mass of Earth.
3. Observation: It receives the mass value from Wikipedia.
4. Thought: It realizes a calculation is needed.
5. Action: It performs the multiplication.
6. Observation: It confirms the result.
7. Answer: The AI provides the final calculated value.

The AI arrives at an accurate and reasoned response by iterating through these steps.

Building a ReACT Pipeline with Maxim AI

In this tutorial, we'll build a ReACT pipeline using Maxim AI. This pipeline will:

1. Log interactions and actions with Maxim Logger.
2. Use tools like Wikipedia for data retrieval and Python for calculations.
3. Run iteratively to answer queries, logging every step.

Setting Up the Environment

To get started, create an account on Maxim AI and install the Maxim library:

pip install maxim-py tiktoken openai httpx python-dotenv

Then, set up the necessary API keys and configure the Maxim logger, you can learn more about the logger through the following documentation.

from maxim.maxim import Maxim, Config, LoggerConfig, Logger
from maxim.logger.components.session import SessionConfig
from maxim.logger.components.trace import TraceConfig
from maxim.logger.components.span import Span, SpanConfig
from maxim.logger.components.generation import GenerationConfig
import tiktoken
from time import time
import openai
import os

# Set up Maxim API and Logger
maxim = Maxim(Config(apiKey=os.environ["MAXIM_API_KEY"]))
logger_config = LoggerConfig(id=os.environ["LOG_API_KEY"])
logger = Logger(config=logger_config, api_key=os.environ["MAXIM_API_KEY"], base_url="https://app.getmaxim.ai")
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

API Keys and Log IDs can be acquired from the Maxim Platform. For more details refer to the video:

Creating the Agent Class

The Agent class manages interactions, generates responses, and logs each step. It communicates with OpenAI's GPT models and tracks token usage.

class Agent:
    def __init__(self, client: OpenAI, system: str = "") -> None:
        self.client = client
        self.system = system
        self.messages: list = []
        if self.system:
            self.messages.append({"role": "system", "content": system})

    def __call__(self, span: Span, message=""):
        if message:
            self.messages.append({"role": "user", "content": message})
        self.generationConfig = GenerationConfig(id=str(uuid4()), name="generation", provider="OpenAI", model="gpt-4o", model_parameters={"temperature": 0}, messages=self.messages)
        self.generation = span.generation(self.generationConfig)
        result = self.execute()
        self.messages.append({"role": "assistant", "content": result})
        return result

    def execute(self):
        completion = client.chat.completions.create(
            model="gpt-4o", messages=self.messages
        )
        return completion.choices[0].message.content

System Prompt for the Assistant

The system prompt defines how the agent should reason through the query. The agent runs in an iterative loop of Thought, Action, PAUSE, and Observation, ensuring each step is logically connected.

system_prompt = """
You run in a loop of Thought, Action, PAUSE, Observation.
At the end of the loop you output an Answer
Use Thought to describe your thoughts about the question you have been asked.
Use Action to run one of the actions available to you - then return PAUSE.
Observation will be the result of running those actions.

Your available actions are:

calculate:
e.g. calculate: 4 * 7 / 3
Runs a calculation and returns the number - uses Python so be sure to use floating point syntax if necessary

wikipedia:
e.g. wikipedia: Django
Returns a summary from searching Wikipedia

Always look things up on Wikipedia if you have the opportunity to do so.

Example session:

Question: What is the capital of France?
Thought: I should look up France on Wikipedia
Action: wikipedia: France
PAUSE 

You will be called again with this:

Observation: France is a country. The capital is Paris.
Thought: I think I have found the answer
Action: Paris.
You should then call the appropriate action and determine the answer from the result

You then output:

Answer: The capital of France is Paris

Now it's your turn:
"""

Read the entire System Prompt and Code in Colab.

Defining Tools

For actions like querying Wikipedia or performing calculations, we define two tools:

import httpx
def wikipedia(q):
    return httpx.get("https://en.wikipedia.org/w/api.php", params={
        "action": "query",
        "list": "search",
        "srsearch": q,
        "format": "json"
    }).json()["query"]["search"][0]["snippet"]

def calculate(operation: str) -> float:
    return eval(operation)

Running the ReACT Pipeline

Once everything is set up, we can start the ReACT pipeline. The system will iterate through the Thought-Action-Observation cycle until a final answer is produced.

def run(max_iterations=10, query: str = ""):
    agent = Agent(client=client, system=system_prompt)
    tools = {"calculate": calculate, "wikipedia": wikipedia}
    next_prompt = query
    i = 0

    while i < max_iterations:
        i += 1
        span = trace.span(SpanConfig(id=str(uuid4()), name=f"Span : {i}"))
        result = agent(span, next_prompt)

        if "PAUSE" in result and "Action" in result:
            action = re.findall(r"Action: ([a-z_]+): (.+)", result, re.IGNORECASE)
            if action:
                action_name, action_arg = action[0]
                tool_func = tools.get(action_name)
                if tool_func:
                    result_tool = tool_func(action_arg)
                    next_prompt = f"Observation: {result_tool}"
                else:
                    next_prompt = f"Observation: Tool {action_name} not found."

        elif "Answer" in result:
            print(f"Final Answer: {result}")
            break
        else:
            break

Example Query: How many planets in the solar system have rings?

You can now run the pipeline for a given query:

run(query="How many planets in the solar system have rings? Now, calculate the total circumference of the rings put together.")

This will engage the agent to fetch the necessary information and perform the required calculations iteratively.

This code, however, locally should output something like this:

--- Iteration 1 ---
466 46
Agent Result: Thought: I need to find out which planets in the solar system have rings and gather information about each of their ring circumferences. I'll start by finding out which planets have rings.
Action: wikipedia: Planetary rings
PAUSE
Action: wikipedia, Argument: Planetary rings
Tool Result: meteoroids, planetoids or moonlets and stellar objects. <span class="searchmatch">Ring</span> systems are best known as <span class="searchmatch">planetary</span> <span class="searchmatch">rings</span>, common components of satellite systems around giant

--- Iteration 2 ---
582 44
Agent Result: Thought: The information provided is not complete about which specific planets have rings. I need to search directly for which planets in the solar system have rings.
Action: wikipedia: Rings of planets in the solar system
PAUSE
Action: wikipedia, Argument: Rings of planets in the solar system
Tool Result: Kirkwood gaps Outer <span class="searchmatch">Solar</span> <span class="searchmatch">System</span> Outer <span class="searchmatch">planets</span> Jupiter ♃ Moons <span class="searchmatch">of</span> Jupiter Io Europa Ganymede Callisto <span class="searchmatch">Rings</span> <span class="searchmatch">of</span> Jupiter Magnetosphere <span class="searchmatch">of</span> Jupiter Jupiter trojans

...
...
Truncated
...
...

--- Iteration 9 ---
1769 162
Agent Result: Thought: Direct searches for the specific circumference or dimensions of the rings are not returning useful information. I'll need to rely on general knowledge that Saturn has the most prominent rings and estimate the average radii from common data sources for calculations.

Average radii for rings:
- Jupiter's rings: ~120,000 km
- Saturn's rings: ~140,000 km
- Uranus' rings: ~51,000 km
- Neptune's rings: ~63,000 km
Using the formula for circumference \(C = 2 \pi \times \text{radius}\), I will calculate each one separately, then sum them for the total.

Action: calculate: 2 * 3.14159 * (120000 + 140000 + 51000 + 63000)
PAUSE
Action: calculate, Argument: 2 * 3.14159 * (120000 + 140000 + 51000 + 63000)
Tool Result: 2349909.32

--- Iteration 10 ---
1833 46
Agent Result: Answer: There are four planets in the solar system with rings: Jupiter, Saturn, Uranus, and Neptune. The total estimated circumference of the rings of these planets combined is approximately 2,349,909.32 kilometers.
Final Answer: Answer: There are four planets in the solar system with rings: Jupiter, Saturn, Uranus, and Neptune. The total estimated circumference of the rings of these planets combined is approximately 2,349,909.32 kilometers.

Conclusion

Using Maxim AI’s Evaluation Framework, we've created an interactive ReACT Pipeline in which every step is logged and stored for further workflow monitoring and refinement. We can handle complex queries intelligently and systematically by integrating tools and logging interactions. This approach enhances AI systems' flexibility, transparency, interpretability, and accuracy, making them well-suited for tasks requiring multi-step reasoning.