3 problems I hit connecting an AI agent to financial data (and how I fixed them)

I've been wiring AI assistants into things that have to be correct, and financial data is where the wheels come off fastest. "Roughly right" is fine for a chatbot; it is not fine for a number someone files a tax return on.

Here are three problems I actually hit, in order, and what fixed each. None of the fixes are exotic — but the order matters, because each one exposes the next.

Problem 1: the model makes up numbers, confidently

The first version just asked the model. "What's the GST on $4,180?" It answered instantly and looked right. It was subtly wrong about the threshold, and — worse — it had no idea it was wrong.

This is the base failure of using a raw LLM for facts: it answers from training data with a cut-off date, and it cannot tell you how current that data is. For anything regulated, that disqualifies it on its own.

Fix: stop asking the model to remember, make it look up. This is what MCP (Model Context Protocol) is for — an open standard for giving an assistant a tool it calls at question time. I pointed it at a public tax server instead of trusting recall:

{ "mcpServers": { "tax": { "url": "https://taxmcp.ai2fin.com" } } }

Now the assistant fetches the figure from a defined source rather than inventing it. The number stopped drifting between runs, which was the first tell that it was real.

Problem 2: I could not tell a real answer from a lucky guess

Grounding fixed the accuracy, but it created a subtler problem: the answers looked the same whether they came from the tool or from the model filling a gap. If a call silently failed and the model improvised, I had no way to see it.

Fix: demand provenance on every field, and refuse anything without it. The server I used returns the source authority and a verification date on every response:

> income_tax_estimate { country: "AU", income: 95000 }
< { incomeTax: ..., takeHome: ...,
    source: "ATO — ato.gov.au", dataVerifiedOn: "2026-06-01" }

So my rule became: if a number arrives without a source and a dataVerifiedOn, it does not get shown to the user. That one check turns "the model said so" into "the ATO said so, as of this date, and here is the link" — which is the difference between a demo and a tool you would put in front of a client.

Problem 3: two copies of the truth drifted apart

By now I had the assistant answering from the tool, and a web page showing the same calculators. Predictably, they disagreed. Someone updated one rate table and not the other, and now the chatbot and the website quoted different take-home numbers for the same salary. This is the oldest bug in the book and it is brutal in finance, because both answers look authoritative.

Fix: one engine, many surfaces — never two copies of the rate tables. The setup that worked has the MCP server and the web calculators reading the same underlying engine, so there is nothing to keep in sync by hand. When I saw the same income_tax_estimate value in the chat and on the page, I knew there was only one source of truth behind both.

If you have ever maintained two copies of anything that changes yearly, you know why this is the fix that actually lets you sleep.

The pattern, stripped down

Three rules, and they generalise well beyond tax:

1. Don't let the model remember facts — give it a tool that fetches them.
2. Require provenance (source + date) on every fetched value, and reject the ones without it.
3. Keep one engine behind every surface, so your answers cannot disagree with each other.

The tax domain just makes the stakes legible — a wrong GST figure is obvious and embarrassing. But the same three problems show up any time an assistant touches data it has no business memorising: pricing, inventory, medical dosages, anything that changes.

The server I used for all of this is free and needs no account, if you want something real to test the pattern against: taxmcp.ai2fin.com. Point your client at it and try to catch it giving you a number without a source. You won't — and that constraint is the whole point.