3 problems I hit connecting an AI agent to financial data (and how I fixed them)

edu — Wed, 19 Aug 2026 04:31:06 +0000

I've been wiring AI assistants into things that have to be correct, and financial data is where the wheels come off fastest. "Roughly right" is fine for a chatbot; it is not fine for a number someone files a tax return on.

Here are three problems I actually hit, in order, and what fixed each. None of the fixes are exotic — but the order matters, because each one exposes the next.

Problem 1: the model makes up numbers, confidently

The first version just asked the model. "What's the GST on $4,180?" It answered instantly and looked right. It was subtly wrong about the threshold, and — worse — it had no idea it was wrong.

This is the base failure of using a raw LLM for facts: it answers from training data with a cut-off date, and it cannot tell you how current that data is. For anything regulated, that disqualifies it on its own.

Fix: stop asking the model to remember, make it look up. This is what MCP (Model Context Protocol) is for — an open standard for giving an assistant a tool it calls at question time. I pointed it at a public tax server instead of trusting recall:

{ "mcpServers": { "tax": { "url": "https://taxmcp.ai2fin.com" } } }

Now the assistant fetches the figure from a defined source rather than inventing it. The number stopped drifting between runs, which was the first tell that it was real.

Problem 2: I could not tell a real answer from a lucky guess

Grounding fixed the accuracy, but it created a subtler problem: the answers looked the same whether they came from the tool or from the model filling a gap. If a call silently failed and the model improvised, I had no way to see it.

Fix: demand provenance on every field, and refuse anything without it. The server I used returns the source authority and a verification date on every response:

> income_tax_estimate { country: "AU", income: 95000 }
< { incomeTax: ..., takeHome: ...,
    source: "ATO — ato.gov.au", dataVerifiedOn: "2026-06-01" }

So my rule became: if a number arrives without a source and a dataVerifiedOn, it does not get shown to the user. That one check turns "the model said so" into "the ATO said so, as of this date, and here is the link" — which is the difference between a demo and a tool you would put in front of a client.

Problem 3: two copies of the truth drifted apart

By now I had the assistant answering from the tool, and a web page showing the same calculators. Predictably, they disagreed. Someone updated one rate table and not the other, and now the chatbot and the website quoted different take-home numbers for the same salary. This is the oldest bug in the book and it is brutal in finance, because both answers look authoritative.

Fix: one engine, many surfaces — never two copies of the rate tables. The setup that worked has the MCP server and the web calculators reading the same underlying engine, so there is nothing to keep in sync by hand. When I saw the same income_tax_estimate value in the chat and on the page, I knew there was only one source of truth behind both.

If you have ever maintained two copies of anything that changes yearly, you know why this is the fix that actually lets you sleep.

The pattern, stripped down

Three rules, and they generalise well beyond tax:

Don't let the model remember facts — give it a tool that fetches them.
Require provenance (source + date) on every fetched value, and reject the ones without it.
Keep one engine behind every surface, so your answers cannot disagree with each other.

The tax domain just makes the stakes legible — a wrong GST figure is obvious and embarrassing. But the same three problems show up any time an assistant touches data it has no business memorising: pricing, inventory, medical dosages, anything that changes.

The server I used for all of this is free and needs no account, if you want something real to test the pattern against: taxmcp.ai2fin.com. Point your client at it and try to catch it giving you a number without a source. You won't — and that constraint is the whole point.

Grounding an LLM in real tax data with a public MCP server

edu — Wed, 19 Aug 2026 04:26:39 +0000

We all know the failure mode: ask an LLM for a tax rate and it'll hand you a confident number with zero provenance. Fine for a demo, useless for anything you'd ship. The interesting question is how you ground the model in a source of truth — and MCP has quietly made that a one-liner.

Quick recap for anyone who missed it: Model Context Protocol is an open standard (Anthropic, late 2024) for connecting an assistant to external tools and data. Client support is now broad — Claude Desktop, Claude Code, ChatGPT, Cursor, VS Code, Windsurf.

A free public server to try it against

2Fin runs a free public MCP server for tax at taxmcp.ai2fin.com. No key, no auth. Point any MCP client at it:

{
  "mcpServers": {
    "tax": { "url": "https://taxmcp.ai2fin.com" }
  }
}

You get eight tools: tax_rate_lookup, compute_gst_vat, income_tax_estimate, company_tax_estimate, cgt_estimate, superannuation_estimate, student_loan_repayment, compare_countries — across 88 countries.

The design choice worth stealing

Every response includes its source authority and a dataVerifiedOn date. That's the part most "AI + data" integrations skip, and it's exactly what makes the output auditable instead of vibes. A wrong or stale number is visible rather than hidden inside a confident sentence.

> income_tax_estimate { country: "AU", income: 95000 }
< { incomeTax: ..., medicareLevy: ..., takeHome: ...,
    source: "ATO — ato.gov.au", dataVerifiedOn: "2026-06-01" }

The same engine backs their web calculators, so the MCP answers and the site can't drift apart — which, if you've ever maintained two copies of the same rate table, you'll appreciate.

The takeaway

Good worked example of the pattern: don't make the model remember facts, give it a tool that fetches them with a citation attached. The tax domain just makes the stakes obvious — but the same design applies to any data your assistant has no business memorising.

If you want to poke at it, it's free and needs no account: taxmcp.ai2fin.com.

DEV Community: edu