Function calling vs free-form generation

LLMs Should Decide What to Ask For, Not Secretly Execute Your Business Logic

Last month I was knee‑deep in a fintech app that let users snap a receipt and instantly see a spending summary. The LLM was supposed to read the image, pull out numbers, and dump them into our ledger. Instead it started “doing its own business logic”: rounding oddly, applying a grandma‑style discount, even flagging a transaction as fraud on a hunch. It felt like handing a chef a pan and watching them secretly add mystery spices.

The issue was clear: I was using free‑form generation as a Swiss‑army knife and letting the model decide how to do the work. The result? Non‑deterministic output, audit nightmares, and a pile of “why did the AI do that?” tickets.

Enter function calling. Think of it as a polite waiter: you tell the model, “If you need a number, call the extract_amount(image) function and I’ll give you a float.” The model can’t magically compute inside its brain; it must ask your API for the exact piece you typed. Your service handles the heavy lifting, has auth checks, logs every request, retries on failure, and returns a deterministic result. This is exactly what OpenAI’s Function Calling and the “Tools” feature do – expose narrow, typed endpoints and keep the business logic on the server side.

A quick analogy: free‑form generation is like giving a kid a 3‑D printer and saying “make me a toy.” They might print a dinosaur, a spaceship, or a banana. Function calling is like handing them a LEGO set with a clear instruction sheet – they can only build what the instructions allow, and you can count each piece as it’s added.

In our fintech case we defined three tools:

• extract_amount(image_bytes) → float
• record_transaction(user_id, amount, category) → bool
• check_fraud(user_id, amount) → bool

Now the LLM says, “I need the amount from this receipt – call extract_amount,” and we get a clean number. If the API hiccups we retry, everything is logged, and auditors can see exactly what was asked and what was returned. No hidden discounts.

In edutech we built a tutoring bot that could answer math questions, but we didn’t want it to hallucinate formulas. We exposed a solve_linear(a, b) → float function. The model now asks, “What’s x in 2x+3=7? Call solve_linear with a=2, b=-4.” The answer is always correct, traceable, and teachers can audit the logs.

Key take‑aways

• Separate interpretation (LLM decides what to ask) from execution (your code does the how).
• Keep tools narrow and parameters typed – no free‑form JSON blobs.
• Wire up auth, audit trails, retries, and error handling on the tool side.
• Keep business logic deterministic in your code, not hidden in the model’s imagination.

So the next time you’re tempted to let the LLM write the whole recipe, remember: let it be the clever waiter, not the secret chef. What narrow tool would you love to add to your stack? Drop a comment and let’s swap ideas!

If you are someone who loves to know the technical work and architecture design I have shared more details based on my experience on this here: https://github.com/SalmonJoy/My_guide_for_building_AI_systems/blob/main/Function_calling_vs_free-form_generation.md