I let the LLM read the loan question, never do the maths

People don't think about loans in form fields. They think in sentences:

"What's my EMI on a 20 lakh home loan at 9% over 15 years? And what if the rate drops to 8.5?"

A normal loan calculator makes them break that sentence into boxes. I wanted an assistant that takes the sentence as it is and answers with a side-by-side comparison. That became QuantumCopilot, an AI loan assistant now live in production. I built its backend and LLM pipeline on my own.

The obvious way to build it is to hand the question to an LLM and let it answer. I didn't do that, because the answer is money. An EMI that's off by a few hundred rupees isn't a small mistake, it's a wrong product. So the whole design rests on one rule:

The LLM interprets. Code decides.

What the LLM is allowed to do

The model gets one job: turn language into typed parameters. Nothing else.

From the question above it should produce something like this:

{
  "loan_type": "home",
  "principal": 2000000,
  "interest_rates": [9.0, 8.5],
  "tenures": [15],
  "tenure_unit": "years"
}

That's where its work ends. It never calculates an EMI, never adds up interest, never rounds anything.

Extraction runs at temperature 0 with JSON output, and the result goes through validation before anything else touches it. A simplified version of the idea (not the production code):

from pydantic import BaseModel, Field

class LoanParams(BaseModel):
    loan_type: str
    principal: float = Field(gt=0)
    interest_rates: list[float]
    tenures: list[float]

def parse(raw: str) -> LoanParams | None:
    try:
        return LoanParams.model_validate_json(raw)
    except ValueError:
        return None  # not a loan question, so route it to general chat

If no usable loan details come back, the request doesn't go on to the calculation step with a guess. It goes to a general finance chat path instead. A wrong number is worse than no number.

The hard part is the language, not the maths

Most of my time went into the rules that turn messy phrasing into clean numbers:

• Indian number words. "20 lakh" is 2,000,000. "2.5 crore" is 25,000,000. People mix these with "₹3,00,000" and "$250K" in the same conversation.
• Units that depend on the loan. Home and personal loans are discussed in years. Car, education and gold loans are usually discussed in months. "36 months" on a home loan has to become 3 years, and "2 years" on a car loan has to become 24 months.
• Six loan products, each with its own fields. Business loans have a repayment frequency (daily, weekly, bi-weekly, monthly). Gold loans start from weight and carat rather than an amount. Education loans have a course duration and a moratorium.

None of this is maths. It's interpretation, which is exactly what an LLM is good at, as long as the rules are written down clearly and the output is checked.

The engine does every number

Once the parameters are valid, up to three scenarios are built from the rates and tenures in the question. Each scenario goes to a deterministic calculation engine.

The three calls run at the same time with asyncio.gather:

results = await asyncio.gather(*(engine.calculate(s) for s in scenarios))

Calling them one after another would make the user wait for the sum of three calls. Running them together makes the wait roughly the slowest single call. My estimate is that this makes the scenario stage about 3x faster. It's an estimate, not a benchmark.

Follow-up questions without re-asking everything

Real conversations go like this:

"OK, and what about 20 years instead?"

That sentence has no amount, no rate and no loan type. A stateless assistant would have to ask for all of it again.

So each turn is saved as a short summary. On the next turn, the model sees those summaries and changes only what the user changed. "Make it 20 years" updates the tenure and carries everything else over.

I chose summaries over full chat history on purpose. They're short, cheap to send with every request, and they keep the state that matters without the filler.

The boring parts that make it production

None of this is exciting, and all of it matters more than the prompt:

• API keys checked in constant time, with key rotation.
• Parameterised SQL everywhere.
• Retries with backoff when the LLM provider rate-limits or returns a server error.
• One consistent error format, so the frontend never has to guess what went wrong.

It runs on FastAPI with 8 Uvicorn workers.

Where the rule still leaks

There's one place where the rule doesn't fully hold yet, and it's worth being honest about.

After the engine returns its results, the LLM writes the final reply, including the comparison table. The numbers in that table come from the engine. But the model is still copying them into text, and a model copying numbers can, in principle, round one, reorder rows or drop a digit.

The engine computes every figure. The LLM still formats them. Those aren't the same guarantee.

The fix has two parts:

1. Build the table in code from the engine's response, and let the LLM write only the sentences around it.
2. Test it. Parse every number out of the reply and compare it with what the engine returned. The target is exact: any mismatch is a failure, not a 99%.

That second part is the bigger lesson. "The LLM never does the maths" is a design claim. Until there's a test that checks it on every answer, it's a hope.

What I'd tell someone building something similar

• Decide what the model is allowed to do, and write it down. In my case: read the question, write the words. Never compute.
• Validate everything the model produces before it reaches anything that matters.
• Put most of your effort into the messy input rules: units, number words, follow-ups. That's where users actually break things.
• If your design depends on the model not doing something, write a test for it.

You can see the full case study, with the architecture and the key decisions, on my site.

I'm Harsh Nanavati, an Applied AI engineer. I build production LLM systems: RAG, AI agents and evaluation. More of my work is at harshnanavati.com.