DEV Community

Cover image for Build an AI Agent From Scratch in Plain Python (4 Small Classes, No Framework)
Ritesh Totlani
Ritesh Totlani

Posted on AI-assisted

Build an AI Agent From Scratch in Plain Python (4 Small Classes, No Framework)

Most people use the word "agent" as if it were magic. It is not. An agent is a small loop wrapped around a language model, and you can build one in four short classes.

In this post we build one from scratch, in plain Python, with no agent framework. It runs on a local Qwen model through LM Studio, so it costs nothing to try. You will see the real run logs, the real results on 40 test invoices, and one thing the model did that I did not expect.

The post explains the ideas first. The complete code of the agent is in the appendix at the end: seven small files, about 330 lines in total, plus a tiny sample dataset and a 14-line script to run it. The larger test harness I used for the 40-invoice results is not needed to understand the agent, so I left it out (more on that below).

What is Agentic AI?

Let's build the idea one step at a time.

Step 1: a plain language model answers once.
You send a prompt, you get text back. That is one question and one answer. The model knows what it learned in training and whatever you paste into the prompt. Nothing else.

Step 2: that is not enough for real work.
A model does not know your vendor list, your purchase orders or what you paid last month. You could paste all of it into the prompt, but real data is large, changes often, and is mostly irrelevant to any single question. And a model alone cannot do anything: it cannot look something up or call your system.

Step 3: a tool is how the model reaches your data.
A tool is an ordinary function in your code, for example lookup_vendor(name). The model cannot run code itself. What it can do is ask for a tool: "please call lookup_vendor with this name". Your program runs the function and hands the result back. The model only ever sees a short description of each tool and the results.

Step 4: an agent is a model, tools and a loop.
Put those three together and you have an agent:

ask the model -> model wants a tool? -> run it -> give the result back -> ask again
                 model has an answer? -> done
Enter fullscreen mode Exit fullscreen mode

The loop repeats until the model decides it has enough to answer. That is the whole mechanism. There is no hidden intelligence in the loop: it is a for loop, and we will read all of it below.

Step 5: "agentic" means the model decides the next step.
This is the real difference, so it is worth stating clearly. Compare three ways to build the same invoice check:

Approach Who decides the steps? Example
Single call Nobody. One prompt, one answer. Paste the invoice and all reference data into a prompt and ask for a verdict.
Fixed workflow Your code decides. The steps and their order are written in advance. if vendor unknown: reject; else check duplicate; then check PO; ... The model may help inside a step, but the path is fixed.
Agent The model decides. It picks the next tool based on what it has seen so far. The model looks up the vendor, sees it is unknown, and stops. Another invoice goes down a longer path.

"Agentic AI" is the third row: software where the model chooses the next action, uses tools to get what it is missing, and keeps going until the job is done.

Three ways to build the same invoice check: single call, fixed workflow, agent
Who decides the next step: nobody, your code, or the model.

Why this matters, and what it costs.
An agent can handle tasks where the right steps depend on what it finds along the way, and it can work with live data instead of a pasted copy. The price is real, though: several model calls instead of one, so it is slower and costs more, and the path is less predictable because the model chooses it. That is why the second half of this post is about logging every step and validating the answer, and why the third post of this series measures an agent against a single call with real numbers. An agent is a tool for some jobs, not an upgrade for every job.

The task: checking an invoice

To make this concrete, the agent plays an accounts-payable clerk. An invoice arrives (already extracted into fields), and the agent decides:

  • APPROVE: pay it
  • REVIEW: a human should look at it
  • REJECT: do not pay it

It also reports which problems it found, using six issue codes:

Code Meaning
UNKNOWN_VENDOR vendor not found, or not approved
IBAN_CHANGED bank details differ from the vendor record
DUPLICATE this invoice number was already paid for this vendor
PO_NOT_FOUND the purchase order does not exist
TOTAL_MISMATCH line items plus tax do not add up to the stated total
OVER_PO the invoice is higher than the purchase order amount

The verdict rule: no issues means APPROVE. Only TOTAL_MISMATCH and/or OVER_PO means REVIEW. Anything else means REJECT.

A clerk cannot answer this from memory. They have to look things up: the vendor list, the payment history, the purchase orders. That is why this task needs tools, and why it suits an agent.

The test data

The data is synthetic and created by one seeded script. It holds 30 vendors (German, British and French style names with valid IBANs), 300 already-paid invoices, about 85 purchase orders, and 40 test invoices.

Of the 40 invoices, 14 are clean, 20 have one problem, and 6 have two. Because we planted the problems ourselves, we know the right answer for every invoice. A small rule-based checker (plain Python, no AI) computes it. We compare the agent against that.

A note on what is in this post. The data generator, that checker and the batch runner together are about 450 lines. They are test scaffolding, not part of the agent, so they are not printed here. The appendix instead gives you a tiny sample dataset (2 vendors, 1 purchase order, 4 invoices) so you can run the real agent right away. The results and logs below come from the full 40-invoice run.

Setup

You need Python 3.11+, LM Studio with a model loaded and its local server running, and one package:

pip install openai
Enter fullscreen mode Exit fullscreen mode

The openai package is only the client that talks to LM Studio's OpenAI-compatible server. The agent itself is our own code. In agent_loop.py, two lines point at your server and model (I used qwen/qwen3.5-9b). Change them if yours differ:

BASE_URL = "http://localhost:1234/v1"
MODEL = "qwen/qwen3.5-9b"
Enter fullscreen mode Exit fullscreen mode

Put the files from the appendix into the folder layout shown there, then run the agent on the four sample invoices:

python -m post1_scratch.run_demo
Enter fullscreen mode Exit fullscreen mode

With the sample data you should see a clean invoice approved, a wrong total sent to review, a duplicate rejected and an unapproved vendor rejected. A small model can vary a little between runs, so wording may differ.

The four classes

There are four classes. Each one has one job.

How the four classes Tool, Agent, AgentLoop and Verdict fit together with the model and the plain Python functions
The four classes around one model. The tools are plain Python functions behind a readable card.

1. Tool: a function the model can read about

A tool is an ordinary Python function plus a small "card" the model can read: a name, a description, and the arguments it takes. The model never sees your code, only the card.

@dataclass
class Tool:
    name: str
    description: str
    parameters: dict
    function: callable

    def schema(self) -> dict:      # the card the model reads
        ...
    def run(self, arguments: dict):  # call the real function
        return self.function(**arguments)
Enter fullscreen mode Exit fullscreen mode

Here is one real card, for the vendor lookup:

Tool(
    name="lookup_vendor",
    description=(
        "Look up a vendor by name in the vendor master. Returns found, id, approved flag, "
        "IBAN and payment terms, or found=false if no such vendor exists."
    ),
    parameters=arguments({"name": {"type": "string", "description": "Vendor name exactly as on the invoice"}}),
    function=lookup_vendor,
)
Enter fullscreen mode Exit fullscreen mode

The description is the most important part. It is the only thing that tells the model when to use the tool. Look at the descriptions of the other three tools in invoice_tools.py in the appendix: they say things like "Needs the vendor id from lookup_vendor". Those sentences are how the model learns which tool has to come first.

We have four tools:

Tool The question it answers
lookup_vendor Is this vendor known and approved? What is their IBAN?
check_duplicate Was this invoice number already paid?
get_purchase_order Does this PO exist, and what is its amount?
check_amounts Do the line items add up, and is the total within the PO?

The tools depend on each other. check_duplicate and get_purchase_order need the vendor id from lookup_vendor. check_amounts needs the PO amount from get_purchase_order. So the order matters.

Notice that no tool compares IBANs. The agent has to do that comparison itself, using the IBAN in the vendor record and the one on the invoice.

2. Agent: instructions plus tools

The agent is a holder. It keeps the instructions and the list of tools, and it can hand the cards to the model and find a tool by name.

@dataclass
class Agent:
    instructions: str
    tools: list[Tool]

    def schemas(self) -> list[dict]: ...   # all tool cards
    def find(self, name: str) -> Tool | None: ...  # None if the model made up a name
Enter fullscreen mode Exit fullscreen mode

The instructions are the system prompt. They explain the issue codes, the verdict rule, and the exact JSON shape we want back:

INSTRUCTIONS = """You are an accounts-payable assistant. You check one invoice and decide if it can be paid.

Use the tools to check the invoice. Do not guess and do not calculate yourself.
Issues:
- UNKNOWN_VENDOR: the vendor is not found, or is not approved. Stop there and answer REJECT with only this issue.
- IBAN_CHANGED: the invoice IBAN differs from the vendor's IBAN.
- DUPLICATE: the invoice number was already paid for this vendor.
- PO_NOT_FOUND: the purchase order does not exist. Without it the amounts cannot be checked.
- TOTAL_MISMATCH: the computed total differs from the invoice total.
- OVER_PO: the invoice total is higher than the PO amount.

Verdict rule:
- no issues: APPROVE
- only TOTAL_MISMATCH and/or OVER_PO: REVIEW
- anything else: REJECT

When you are done, answer with JSON only, no other text:
{"verdict": "APPROVE or REVIEW or REJECT", "issues": ["ISSUE_CODE", ...], "reason": "one short sentence"}
"""
Enter fullscreen mode Exit fullscreen mode

3. AgentLoop: the heart of it

The agent loop: ask the model, run the tool, add the result, repeat until the final answer
The whole agent in one picture. The model chooses; our code runs the tool.

This class is the agent. Here is its main method, trimmed to the part that matters (the full file is in the appendix):

for step in range(1, self.max_steps + 1):
    reply = self.ask(messages)              # 1. ask the model
    if not reply.tool_calls:                # 2. no tool wanted: this is the answer
        return reply.content
    messages.append(reply)
    for call in reply.tool_calls:           # 3. the model wants tools: run them
        result = self.run_tool(call)
        messages.append({                   # 4. give each result back
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })
raise RuntimeError(f"no final answer after {self.max_steps} steps")
Enter fullscreen mode Exit fullscreen mode

Read it slowly, because this is the whole idea:

  1. Ask the model what to do.
  2. If it asks for no tool, it has an answer, and we return it.
  3. If it asks for tools, we run them.
  4. We add each result to the conversation and go round again.

Two small details carry a lot of weight:

  • Mistakes go back to the model. If the model invents a tool name or sends broken arguments, run_tool returns an error as the tool result. The model reads it and can correct itself, instead of the program crashing.
  • There is a step limit. max_steps stops a confused model from looping forever.

Every step is also printed. That printout is the run log you will see below, and it is how you find out what an agent really does.

4. Verdict: a fixed shape for the answer

The model answers in text. We want that text in a fixed shape, and we want to refuse anything that does not fit.

@classmethod
def from_json(cls, text: str) -> "Verdict":
    data = json.loads(text)
    ...
    if verdict not in VERDICTS:
        raise ValueError(f"bad verdict: {verdict!r}")
    if not isinstance(issues, list) or any(i not in ISSUE_CODES for i in issues):
        raise ValueError(f"bad issues: {issues!r}")
    ...
    return cls(verdict, issues, reason)
Enter fullscreen mode Exit fullscreen mode

If the model says anything outside the allowed verdicts or issue codes, we get an error and not a silent wrong answer. That is all four classes: Tool, Agent, AgentLoop, Verdict.

What the agent did

I ran all 40 invoices once, at temperature 0, on a local Qwen 3.5 9B model. This is what the script printed at the end:

verdict correct: 40/40
issues correct:  40/40
failures:        0
average steps:   4.6
Enter fullscreen mode Exit fullscreen mode

All 40 answers matched the checker, on both the verdict and the exact set of issue codes. A run took about 10 seconds per invoice on average, and the whole set took roughly 6 to 7 minutes.

The average of 4.6 steps is interesting. It is not 5 for every invoice, because the agent does not always need all four tools. It took three different paths:

Path Invoices Steps
Stopped right after the vendor lookup (vendor unknown or not approved) 5 2
Stopped after the PO lookup (no check_amounts) 2 4
Used all four tools 33 5

The agent chose these paths. Nothing in our code says "stop if the vendor is unknown". The instructions say it, and the model followed them. Let's look at real logs for each path.

Path 1: unknown vendor, 2 steps

=== T01 Bradshaw Systems LLP ===
correct answer: REJECT ['UNKNOWN_VENDOR']
step 1: tool call lookup_vendor({"name":"Bradshaw Systems LLP"}) -> {'found': False}
step 2: final answer {"verdict": "REJECT", "issues": ["UNKNOWN_VENDOR"], "reason": "Vendor Bradshaw Systems LLP not found in vendor master"}
agent answer:   {'verdict': 'REJECT', 'issues': ['UNKNOWN_VENDOR']}
Enter fullscreen mode Exit fullscreen mode

The vendor is not in the master, so there is nothing else to check. One tool call, then the answer. A fixed script could do this too, but note that here the model decided that nothing else was worth checking.

Path 3: a clean invoice, all four tools

One real run for invoice T03: five steps, with the vendor id and PO amount passed forward by the model
The same run as the log below, step by step.

=== T03 Hayward Logistics LLP ===
correct answer: APPROVE []
step 1: tool call lookup_vendor({"name":"Hayward Logistics LLP"}) -> {'found': True, 'id': 'V020', 'name': 'Hayward Logistics LLP', 'country': 'GB', 'approved': True, 'iban': 'GB09FYEE54189423280598', 'payment_terms': 'Net 30'}
step 2: tool call check_duplicate({"vendor_id":"V020","invoice_number":"2025-06-6134"}) -> False
step 3: tool call get_purchase_order({"vendor_id":"V020","po_number":"PO-2025-9883"}) -> {'found': True, 'po_number': 'PO-2025-9883', 'vendor_id': 'V020', 'amount': 1135.12}
step 4: tool call check_amounts({"line_items":[{"description":"Transport charges","quantity":8,"unit_price":23.68},{"description":"Technical survey","quantity":34,"unit_price":18.38}],"tax_rate":0.05,"invoice_total":855.08,"po_amount":1135.12}) -> {'computed_total': 855.08, 'total_matches_invoice': True, 'within_po_amount': True}
step 5: final answer {"verdict": "APPROVE", "issues": [], "reason": "Vendor approved, IBAN matches, not duplicate, PO exists, amounts match and within PO limit"}
agent answer:   {'verdict': 'APPROVE', 'issues': []}
Enter fullscreen mode Exit fullscreen mode

Read this log step by step, because it shows the agent's reasoning without any magic:

  • Step 1: it starts with the vendor, because every other tool needs the vendor id.
  • Step 2: it passes V020 from the first result into check_duplicate. The model copied a value from one result into the next call.
  • Step 3: it does the same for the purchase order.
  • Step 4: it takes the PO amount (1135.12) from step 3 and the line items from the invoice and passes both to check_amounts.
  • Step 5: no more tools, so this is the answer. Its reason mentions the IBAN, which it compared itself: there was no tool for that.

This chaining, where one result feeds the next call, is the whole point of tools that depend on each other. We never wrote that order in code. The tool descriptions ("Needs the vendor id from lookup_vendor") told the model it had to follow it.

A double issue, and the unexpected part

=== T21 Chevalier Services SARL ===
correct answer: REJECT ['IBAN_CHANGED', 'PO_NOT_FOUND']
step 1: tool call lookup_vendor({"name":"Chevalier Services SARL"}) -> {'found': True, 'id': 'V022', 'name': 'Chevalier Services SARL', 'country': 'FR', 'approved': True, 'iban': 'FR7630185283444852791198997', 'payment_terms': 'Net 60'}
step 2: tool call check_duplicate({"vendor_id":"V022","invoice_number":"R-2025-2549"}) -> False
step 3: tool call get_purchase_order({"vendor_id":"V022","po_number":"PO-2025-3246"}) -> {'found': False, 'message': 'No purchase order with this number exists for this vendor.'}
step 4: tool call check_amounts({"line_items":[{"description":"Formation","quantity":27,"unit_price":122.24},{"description":"Fourniture de matériel","quantity":23,"unit_price":89.23},{"description":"Expertise technique","quantity":31,"unit_price":135.14},{"description":"Documentation","quantity":24,"unit_price":44.93},{"description":"Travaux de montage","quantity":11,"unit_price":86.02}],"tax_rate":0.055,"invoice_total":12202.82,"po_amount":0}) -> {'computed_total': 12202.82, 'total_matches_invoice': True, 'within_po_amount': False}
step 5: final answer {"verdict": "REJECT", "issues": ["IBAN_CHANGED", "PO_NOT_FOUND"], "reason": "Vendor IBAN differs from invoice and purchase order not found"}
agent answer:   {'verdict': 'REJECT', 'issues': ['IBAN_CHANGED', 'PO_NOT_FOUND']}
Enter fullscreen mode Exit fullscreen mode

The final answer is correct, but look at step 4. The purchase order was not found in step 3, so there is no PO amount. Our instructions even say that without the PO the amounts cannot be checked. The model called check_amounts anyway and filled in "po_amount": 0, a number it made up.

It happened on 2 of the 4 invoices with a missing PO (T21 and T26). The other two (T17 and T23) stopped after step 3, as the instructions intended. The tool answered within_po_amount: False, and the model did not turn that into a wrong OVER_PO, so the final answers were still right.

This is a good example of why you keep step logs. The final accuracy is 40/40, and you would never know about this extra, pointless call from the accuracy number alone. A stricter tool description or a guard inside check_amounts would stop it. I left it as it was, so you see what a model really does.

What if the answer has the wrong shape?

A model can return something that is valid JSON but still not valid for us, like a verdict of "maybe". To test this on purpose, AgentLoop has a switch, inject_bad_output, that swaps the final answer for a bad one. To reproduce it, create AgentLoop(agent, inject_bad_output=True) in run_demo.py and run one invoice. This is the real output from the full run:

=== T02 Thompson Services Limited (bad output injected) ===
step 1: tool call lookup_vendor({"name":"Thompson Services Limited"}) -> {'found': True, 'id': 'V018', 'name': 'Thompson Services Limited', 'country': 'GB', 'approved': True, 'iban': 'GB57JBKF59435189991330', 'payment_terms': 'Net 30'}
step 2: tool call check_duplicate({"vendor_id":"V018","invoice_number":"2025-11-1179"}) -> False
step 3: tool call get_purchase_order({"vendor_id":"V018","po_number":"PO-2025-6333"}) -> {'found': True, 'po_number': 'PO-2025-6333', 'vendor_id': 'V018', 'amount': 12431.04}
step 4: tool call check_amounts({"line_items":[{"description":"Installation","quantity":4,"unit_price":144.76},{"description":"Training workshop","quantity":31,"unit_price":19.1},{"description":"Documentation","quantity":20,"unit_price":153.34},{"description":"Support retainer","quantity":21,"unit_price":201.88}],"tax_rate":0.05,"invoice_total":9374.84,"po_amount":12431.04}) -> {'computed_total': 8901.29, 'total_matches_invoice': False, 'within_po_amount': True}
step 5: model answered {"verdict": "REVIEW", "issues": ["TOTAL_MISMATCH"], "reason": "Computed total (8901.29) differs from invoice total (9374.84)"}
step 5: (test) answer replaced by a bad one
step 5: final answer {"verdict": "maybe", "issues": [], "reason": "Not sure."}
FAILED: ValueError: bad verdict: 'maybe'
The scratch agent has no retry: the run for this invoice ends here.
Enter fullscreen mode Exit fullscreen mode

The model's real answer in step 5 was correct (REVIEW, TOTAL_MISMATCH). The test replaced it with an invalid one. Verdict.from_json caught it and raised bad verdict: 'maybe', so a wrong answer never slips through silently.

But look at the last line. Our agent has no way to recover: it does not ask the model to try again, so the run for that invoice simply ends. The model had already done all the work, and we lose it. In a real system you would feed the error back to the model and let it retry, in the same way we already return bad tool calls. That retry plumbing is the part a framework can take over, and it is where the next post starts.

What this shows, and what it does not

What you now have:

  • An agent is a loop. Ask the model, run the tool it chooses, give the result back, repeat until it answers.
  • Tools are plain functions with a card. The description on the card is how the model decides when and in what order to use them.
  • The model passes results forward. The vendor id from one call went into the next calls without us wiring it.
  • Validate the answer. Verdict turns free text into a fixed shape or fails loudly.
  • Keep a log of every step. It is how we spotted the made-up po_amount: 0, which the 40/40 score hides.

What it does not prove:

  • This is one run of 40 synthetic invoices, at temperature 0, on one small local model. 40/40 here does not mean 100% on real invoices, and the planted problems are cleaner than real ones.
  • The checker uses the same tool functions as the agent, so it would not catch a bug inside a tool.
  • It does not tell us whether we needed an agent at all. A single prompt that contains all the reference data might do as well, or not. That is the question for the third post.

What's next

In the next post I rebuild the exact same agent with PydanticAI, on the same tools and the same 40 invoices, and show which parts of today's code the framework takes over, including the retry we just missed. After that, I measure this agent against a single-call version with real numbers.


Appendix: the complete agent code

Every file of the agent, exactly as it ran. Open a block to see the code. Folder layout:

agentic-invoice/
  data/
    vendors.json  purchase_orders.json  invoice_history.json  demo_invoices.json
  common/
    tools.py            # the plain Python functions
  post1_scratch/
    tool.py             # class 1
    agent.py            # class 2
    agent_loop.py       # class 3
    verdict.py          # class 4
    instructions.py     # the system prompt
    invoice_tools.py    # the tool cards
    run_demo.py         # runs the agent on the sample invoices
Enter fullscreen mode Exit fullscreen mode

Also create an empty __init__.py in common/ and in post1_scratch/.

common/tools.py: the plain Python functions

import json
from decimal import ROUND_HALF_UP, Decimal
from functools import lru_cache
from pathlib import Path

DATA_DIR = Path(__file__).resolve().parent.parent / "data"


def money(value: Decimal) -> Decimal:
    """Round to 2 decimal places, the way money actually works."""
    return value.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)


def verify_totals(line_items: list[dict], tax_rate: float) -> float:
    """The total the invoice should have: sum of quantity x unit_price, plus tax."""
    subtotal = sum(
        money(Decimal(str(i["quantity"])) * Decimal(str(i["unit_price"])))
        for i in line_items
    )
    return float(subtotal + money(subtotal * Decimal(str(tax_rate))))


@lru_cache(maxsize=None)  # read each JSON file once, then reuse it
def _load(name: str) -> list[dict]:
    return json.loads((DATA_DIR / f"{name}.json").read_text(encoding="utf-8"))


def lookup_vendor(name: str) -> dict:
    """Find a vendor by name in the vendor master. Returns the record or found=False."""
    for vendor in _load("vendors"):
        if vendor["name"].casefold() == name.strip().casefold():
            return {"found": True, **vendor}
    return {"found": False}


def check_duplicate(vendor_id: str, invoice_number: str) -> bool:
    """True if this vendor's invoice number was already paid."""
    return any(
        h["vendor_id"] == vendor_id and h["invoice_number"] == invoice_number
        for h in _load("invoice_history")
    )


def get_purchase_order(vendor_id: str, po_number: str) -> dict:
    """Find a purchase order by number. Returns the record or found=False."""
    for po in _load("purchase_orders"):
        if po["po_number"] == po_number and po["vendor_id"] == vendor_id:
            return {"found": True, **po}
    return {
        "found": False,
        "message": "No purchase order with this number exists for this vendor.",
    }


def check_amounts(
    line_items: list[dict], tax_rate: float, invoice_total: float, po_amount: float
) -> dict:
    """Compare the invoice with its own line items and with the PO limit.
    Needs po_amount, which only exists after get_purchase_order has run."""
    computed = verify_totals(line_items, tax_rate)
    return {
        "computed_total": computed,
        "total_matches_invoice": abs(computed - invoice_total) <= 0.005,
        "within_po_amount": invoice_total <= po_amount,
    }
Enter fullscreen mode Exit fullscreen mode

post1_scratch/tool.py: class 1

from dataclasses import dataclass


@dataclass
class Tool:
    name: str
    description: str
    parameters: dict
    function: callable

    def schema(self) -> dict:
        """The card the model reads: name, description, arguments."""
        return {
            "type": "function",
            "function": {
                "name": self.name,
                "description": self.description,
                "parameters": self.parameters,
            },
        }

    def run(self, arguments: dict):
        """Call the real Python function with the arguments the model chose."""
        return self.function(**arguments)
Enter fullscreen mode Exit fullscreen mode

post1_scratch/invoice_tools.py: the four tool cards

from common.tools import (
    check_duplicate,
    get_purchase_order,
    lookup_vendor,
    verify_totals,
    check_amounts,
)
from post1_scratch.tool import Tool


def arguments(properties: dict) -> dict:
    """The JSON shape the model must follow when it calls a tool. All arguments are required."""
    return {"type": "object", "properties": properties, "required": list(properties)}


TOOLS = [
    Tool(
        name="lookup_vendor",
        description=(
            "Look up a vendor by name in the vendor master. Returns found, id, approved flag, "
            "IBAN and payment terms, or found=false if no such vendor exists."
        ),
        parameters=arguments(
            {
                "name": {
                    "type": "string",
                    "description": "Vendor name exactly as on the invoice",
                }
            }
        ),
        function=lookup_vendor,
    ),
    Tool(
        name="check_duplicate",
        description=(
            "Check whether this vendor's invoice number was already paid. Returns true or false. "
            "Needs the vendor id from lookup_vendor."
        ),
        parameters=arguments(
            {"vendor_id": {"type": "string"}, "invoice_number": {"type": "string"}}
        ),
        function=check_duplicate,
    ),
    Tool(
        name="get_purchase_order",
        description=(
            "Find this vendor's purchase order by number. Returns found, the approved amount, "
            "or found=false if no such purchase order exists. Needs the vendor id from lookup_vendor."
        ),
        parameters=arguments(
            {"vendor_id": {"type": "string"}, "po_number": {"type": "string"}}
        ),
        function=get_purchase_order,
    ),
    Tool(
        name="check_amounts",
        description=(
            "Compare the invoice with its own line items and with the purchase order limit. "
            "Returns the computed total, whether it matches the invoice total, and whether the "
            "invoice total is within the PO amount. Needs the PO amount from get_purchase_order."
        ),
        parameters=arguments(
            {
                "line_items": {
                    "type": "array",
                    "items": arguments(
                        {
                            "description": {"type": "string"},
                            "quantity": {"type": "number"},
                            "unit_price": {"type": "number"},
                        }
                    ),
                },
                "tax_rate": {
                    "type": "number",
                    "description": "For example 0.19 for 19%",
                },
                "invoice_total": {"type": "number"},
                "po_amount": {
                    "type": "number",
                    "description": "The amount from get_purchase_order",
                },
            }
        ),
        function=check_amounts,
    ),
]
Enter fullscreen mode Exit fullscreen mode

post1_scratch/agent.py: class 2

from dataclasses import dataclass

from post1_scratch.tool import Tool


@dataclass
class Agent:
    instructions: str
    tools: list[Tool]

    def schemas(self) -> list[dict]:
        """All tool cards, ready to send to the model."""
        return [tool.schema() for tool in self.tools]

    def find(self, name: str) -> Tool | None:
        """The tool the model asked for, or None if it made up a name."""
        for tool in self.tools:
            if tool.name == name:
                return tool
        return None
Enter fullscreen mode Exit fullscreen mode

post1_scratch/instructions.py: the system prompt

INSTRUCTIONS = """You are an accounts-payable assistant. You check one invoice and decide if it can be paid.

Use the tools to check the invoice. Do not guess and do not calculate yourself.
Issues:
- UNKNOWN_VENDOR: the vendor is not found, or is not approved. Stop there and answer REJECT with only this issue.
- IBAN_CHANGED: the invoice IBAN differs from the vendor's IBAN.
- DUPLICATE: the invoice number was already paid for this vendor.
- PO_NOT_FOUND: the purchase order does not exist. Without it the amounts cannot be checked.
- TOTAL_MISMATCH: the computed total differs from the invoice total.
- OVER_PO: the invoice total is higher than the PO amount.

Verdict rule:
- no issues: APPROVE
- only TOTAL_MISMATCH and/or OVER_PO: REVIEW
- anything else: REJECT

When you are done, answer with JSON only, no other text:
{"verdict": "APPROVE or REVIEW or REJECT", "issues": ["ISSUE_CODE", ...], "reason": "one short sentence"}
"""
Enter fullscreen mode Exit fullscreen mode

post1_scratch/agent_loop.py: class 3

import json

from openai import OpenAI

from post1_scratch.agent import Agent


class AgentLoop:
    BASE_URL = "http://localhost:1234/v1"
    MODEL = "qwen/qwen3.5-9b"

    def __init__(self, agent: Agent, max_steps: int = 10, inject_bad_output: bool = False):
        self.agent = agent
        self.max_steps = max_steps
        self.inject_bad_output = inject_bad_output  # test switch: swap the final answer for a bad one
        self.client = OpenAI(base_url=self.BASE_URL, api_key="not-needed")
        self.model = self.MODEL
        self.steps = 0  # trips to the model for the last invoice
        self.tool_calls = 0  # tools run for the last invoice

    def ask(self, messages: list[dict]):
        """One trip to the model. Returns its reply message."""
        response = self.client.chat.completions.create(
            model=self.model,
            messages=messages,
            tools=self.agent.schemas(),
            temperature=0,
        )
        return response.choices[0].message

    def run_tool(self, call) -> object:
        """Run the tool the model asked for. Mistakes come back as an error the model can read."""
        tool = self.agent.find(call.function.name)
        if tool is None:
            return {"error": f"no such tool: {call.function.name}"}
        try:
            return tool.run(json.loads(call.function.arguments))
        except (json.JSONDecodeError, TypeError) as error:
            return {"error": str(error)}

    def run(self, invoice: dict) -> str:
        """Check one invoice. Returns the model's final answer as text."""
        self.steps = 0
        self.tool_calls = 0
        messages = [
            {"role": "system", "content": self.agent.instructions},
            {"role": "user", "content": json.dumps(invoice, ensure_ascii=False)},
        ]
        for step in range(1, self.max_steps + 1):
            self.steps = step
            reply = self.ask(messages)
            if not reply.tool_calls:  # no tool requested: this is the final answer
                answer = reply.content
                if self.inject_bad_output:  # test: swap in an invalid answer, and say so in the log
                    print(f"step {step}: model answered {answer}")
                    answer = '{"verdict": "maybe", "issues": [], "reason": "Not sure."}'
                    print(f"step {step}: (test) answer replaced by a bad one")
                print(f"step {step}: final answer {answer}")
                return answer
            messages.append(reply)
            for call in reply.tool_calls:
                result = self.run_tool(call)
                self.tool_calls += 1
                print(
                    f"step {step}: tool call {call.function.name}({call.function.arguments}) -> {result}"
                )
                messages.append(
                    {
                        "role": "tool",
                        "tool_call_id": call.id,
                        "content": json.dumps(result),
                    }
                )
        raise RuntimeError(f"no final answer after {self.max_steps} steps")
Enter fullscreen mode Exit fullscreen mode

post1_scratch/verdict.py: class 4

import json
from dataclasses import dataclass

VERDICTS = ["APPROVE", "REVIEW", "REJECT"]
ISSUE_CODES = [
    "TOTAL_MISMATCH",
    "DUPLICATE",
    "UNKNOWN_VENDOR",
    "IBAN_CHANGED",
    "OVER_PO",
    "PO_NOT_FOUND",
]


@dataclass
class Verdict:
    verdict: str
    issues: list[str]
    reason: str

    @classmethod
    def from_json(cls, text: str) -> "Verdict":
        """Turn the model's answer into a Verdict. Raises ValueError if anything is wrong."""
        data = json.loads(text)
        if not isinstance(data, dict):
            raise ValueError(f"expected a JSON object, got {type(data).__name__}")
        verdict, issues, reason = (
            data.get("verdict"),
            data.get("issues"),
            data.get("reason"),
        )
        if verdict not in VERDICTS:
            raise ValueError(f"bad verdict: {verdict!r}")
        if not isinstance(issues, list) or any(i not in ISSUE_CODES for i in issues):
            raise ValueError(f"bad issues: {issues!r}")
        if not isinstance(reason, str):
            raise ValueError(f"bad reason: {reason!r}")
        return cls(verdict, issues, reason)
Enter fullscreen mode Exit fullscreen mode

Sample data and the run script

Four invoices to try: a clean one, one with a wrong total, a duplicate, and one from an unapproved vendor. Create these four files in data/.

data/vendors.json

[
  {"id": "V001", "name": "Weber Consulting GmbH", "country": "DE", "approved": true, "iban": "DE89370400440532013000", "payment_terms": "Net 30"},
  {"id": "V002", "name": "Koch Handel AG", "country": "DE", "approved": false, "iban": "DE02120300000000202051", "payment_terms": "Net 14"}
]
Enter fullscreen mode Exit fullscreen mode

data/purchase_orders.json

[
  {"po_number": "PO-2025-1001", "vendor_id": "V001", "amount": 1500.0}
]
Enter fullscreen mode Exit fullscreen mode

data/invoice_history.json

[
  {"vendor_id": "V001", "invoice_number": "RE-2025-0100"}
]
Enter fullscreen mode Exit fullscreen mode

data/demo_invoices.json

[
  {"invoice_number": "RE-2025-0200", "vendor_name": "Weber Consulting GmbH", "iban": "DE89370400440532013000", "po_number": "PO-2025-1001", "date": "2025-03-14", "line_items": [{"description": "Beratungsleistung", "quantity": 8, "unit_price": 100.0}, {"description": "Schulung", "quantity": 2, "unit_price": 150.0}], "tax_rate": 0.19, "total": 1309.0},
  {"invoice_number": "RE-2025-0201", "vendor_name": "Weber Consulting GmbH", "iban": "DE89370400440532013000", "po_number": "PO-2025-1001", "date": "2025-03-14", "line_items": [{"description": "Beratungsleistung", "quantity": 8, "unit_price": 100.0}, {"description": "Schulung", "quantity": 2, "unit_price": 150.0}], "tax_rate": 0.19, "total": 1400.0},
  {"invoice_number": "RE-2025-0100", "vendor_name": "Weber Consulting GmbH", "iban": "DE89370400440532013000", "po_number": "PO-2025-1001", "date": "2025-03-14", "line_items": [{"description": "Beratungsleistung", "quantity": 8, "unit_price": 100.0}, {"description": "Schulung", "quantity": 2, "unit_price": 150.0}], "tax_rate": 0.19, "total": 1309.0},
  {"invoice_number": "RE-2025-0300", "vendor_name": "Koch Handel AG", "iban": "DE02120300000000202051", "po_number": "PO-2025-1001", "date": "2025-03-14", "line_items": [{"description": "Beratungsleistung", "quantity": 8, "unit_price": 100.0}, {"description": "Schulung", "quantity": 2, "unit_price": 150.0}], "tax_rate": 0.19, "total": 1309.0}
]
Enter fullscreen mode Exit fullscreen mode

post1_scratch/run_demo.py

# Run from the agentic-invoice folder:  python -m post1_scratch.run_demo
import json

from common.tools import DATA_DIR
from post1_scratch.agent import Agent
from post1_scratch.agent_loop import AgentLoop
from post1_scratch.instructions import INSTRUCTIONS
from post1_scratch.invoice_tools import TOOLS
from post1_scratch.verdict import Verdict

loop = AgentLoop(Agent(INSTRUCTIONS, TOOLS))
for invoice in json.loads((DATA_DIR / "demo_invoices.json").read_text(encoding="utf-8")):
    print(f"--- {invoice['invoice_number']} ---")
    print(Verdict.from_json(loop.run(invoice)))
Enter fullscreen mode Exit fullscreen mode

What to expect with the sample data: RE-2025-0200 approve, RE-2025-0201 review with TOTAL_MISMATCH, RE-2025-0100 reject as DUPLICATE, RE-2025-0300 reject as UNKNOWN_VENDOR. The wording of the reasons will differ from run to run.

Top comments (0)