DEV Community

Cover image for What is an AI Agent Harness?
ElementalSilk
ElementalSilk

Posted on

What is an AI Agent Harness?

An AI agent harness is the software environment that surrounds an LLM to give it the tools and context needed to complete multi-step tasks.
It turns your basic LLM calls to an operational system executing real world task.

The model reasons through a prompt and decides the actions.
The harness connects the agent it to the tools, systems, memory and execution environments needed to carry out those actions.

In a AI harness Agents typically run in a loop: an LLM decides what to do, a tool executes, a model evaluates the results and then continues in that loop until the task is complete.

Model(Reason)+ Harness(Action)= Agent (Output)
Or
Model (Brain)
Agent (as the Body)
and Harness as the hands/tools the body uses to finish a given task.
You get the picture.

The reason - act - observe loop
This is at the core of many AI agents. Understanding this loop is critical to understanding how a harness works.

Reason- The model reads everything in its context, including the task, relevant memory and previous results, then decides what action to take next.
Act - The harness carries out that action by running a tool, executing code in a sandbox, calling an API or writing to storage.
Observe- The harness captures the result and feeds it back to the model as new context.
Repeat - The model uses that result to decide what to do next. The loop continues until the task is complete.
The ReAct: Synergizing Reasoning and Acting in Language Models- is published in this paper and is an excellent read.

Ok enough of "simile", let's jump to a concrete example

Let's say we have a file in our local system "sales and vendor commission.txt" and we want to perform complex and kind of weird calculation on this.
This is your final prompt -

"I have a text file called 'sales and commission.txt' in the current directory -
if it exists, extract the column with header 'Sales Person Name' , calculate it's word count.
Also, if the word count is > 20 calculate the vendor commission which is 15% of column header Sales or if the word count is < 20 calculate the vendor commission which is 10% of column header Sales
and create a new column call it vendor commission, and also get the current UTC Time and add it to report against each row"

*for the sake of an example let's say you create a Python code to do it (there are multiple way you could do it like upload the sheet into a Agent etc).

*Step 1 - Define the Tools your AGENT will have access to *

TOOLS: list[dict[str, Any]] = [
    {
        "name": "calculator",
        "description": (
            "Evaluate a basic arithmetic expression (+, -, *, /, **, %, parentheses). "
            "Use this for any math instead of computing it yourself."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "expression": {
                    "type": "string",
                    "description": "The arithmetic expression to evaluate, e.g. '(12.5/100) * 842'",
                }
            },
            "required": ["expression"],
        },
    },
    {
        "name": "get_current_time",
        "description": "Get the current date and time, optionally offset by a UTC offset in hours.",
        "input_schema": {
            "type": "object",
            "properties": {
                "utc_offset_hours": {
                    "type": "number",
                    "description": "Hours offset from UTC, e.g. -8 for US Pacific. Defaults to 0 (UTC).",
                }
            },
            "required": [],
        },
    },
    {
        "name": "word_count",
        "description": "Count words and characters in a piece of text.",
        "input_schema": {
            "type": "object",
            "properties": {
                "text": {"type": "string", "description": "The text to analyze."}
            },
            "required": ["text"],
        },
    },
    {
        "name": "read_local_file",
        "description": (
            "Read the contents of a text file from the local working directory. "
            "Paths are restricted to the sandbox directory for safety."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "filename": {"type": "string", "description": "Name of the file to read."}
            },
            "required": ["filename"],
        },
    },
]

Enter fullscreen mode Exit fullscreen mode

Step 2 - Define the Tools methods

def tool_calculator(expression: str) -> dict[str, Any]:
     try:
         tree = ast.parse(expression, mode="eval")
         result = _safe_eval(tree.body)
         return {"result": result}
     except Exception as e:
         return {"error": f"Could not evaluate expression: {e}"}


 def tool_get_current_time(utc_offset_hours: float = 0) -> dict[str, Any]:
     now = datetime.now(timezone.utc)
     if utc_offset_hours:
         from datetime import timedelta
         now = now + timedelta(hours=utc_offset_hours)
     return {"iso_timestamp": now.isoformat(), "utc_offset_hours": utc_offset_hours}


 def tool_word_count(text: str) -> dict[str, Any]:
     words = text.split()
    return {"word_count": len(words), "character_count": len(text)}
Enter fullscreen mode Exit fullscreen mode

Step 3 - System Prompt

 self.system_prompt = system_prompt or (
            "You are a helpful assistant with access to tools. "
            "Use tools whenever they would give a more accurate or reliable answer "
            "than reasoning alone (e.g. always use the calculator for arithmetic). "
            "Once you have everything you need, give a clear, direct final answer."
        )
Enter fullscreen mode Exit fullscreen mode

Step 4 - This is the heart of the whole logic, Use the stop_reason to check if the Agent is done is the reason is tool_use then keep going in a loop

if response.stop_reason != "tool_use":
                # Final answer reached.
                final_text = "".join(
                    block.text for block in response.content if block.type == "text"
                )
                return final_text

            # Otherwise, execute every requested tool call and collect results.
            tool_results = []
            for block in response.content:
                if block.type != "tool_use":
                    continue
                self._log(f"[tool call] {block.name}({json.dumps(block.input)})")
                result_json = self._execute_tool(block.name, block.input)
                self._log(f"[tool result] {result_json}")
                tool_results.append(
                    {
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": result_json,
                    }
                )        
Enter fullscreen mode Exit fullscreen mode

Core loop:

Send the conversation + tool schemas to Claude.
If stop_reason == "tool_use", execute each requested tool locally and append the results as tool_result blocks.
Loop back to step 1.
Stop when Claude returns a final text answer (stop_reason == "end_turn") or max_iterations is hit (safety limit).

Top comments (0)