Can Local LLMs Actually Call Functions? I Tested 2 Models on 6 Real API Tasks

I tested two local LLMs — Qwen3-14B and Llama-3.2-3B — on six real function-calling tasks: weather API, calendar booking, database query, file operations, multi-step workflows, and error recovery. Same prompts, same schema, same hardware (M2 24GB), same agent loop. The smaller 3B model won on JSON validity and reliability. The bigger 14B model won on argument accuracy and multi-turn handling. Both fell short of GPT-4-class reliability, but the gap was smaller than I expected.

If you're building a local coding agent in 2026, function calling is the bottleneck. JSON validity is table stakes. Argument correctness is what determines whether your agent actually works. The results below tell you which model to use for which kind of agent.

I started this benchmark because the agent tooling space is full of confident claims that don't survive contact with real APIs. The two questions I wanted answered:

1. Do local LLMs produce valid JSON tool calls reliably enough to use in production?
2. Does the 14B model beat the 3B model on argument correctness, or is the gap small enough that the smaller model wins on cost?

The 6 tasks

The task suite is six real function-calling scenarios, not toy problems. Each one is grounded in an actual API schema a coding agent would have to handle:

#	Task	What it tests
1	Weather API call	Simple single-tool invocation, type-checked arguments
2	Calendar booking	Multi-argument tool with date/time parsing
3	Database query	Nested object arguments, enum values
4	File operations	Path validation, permissions, error handling
5	Multi-step workflow	3+ tool calls in sequence, dependent arguments
6	Error recovery	When the API returns an error, can the model adapt?

Each task is a self-contained Python file with a defined tool schema (OpenAI function-calling format). The model gets the user's request + the tool spec + conversation history. It must produce a valid tool call with correct argument types and values. Success means: the JSON parses, all required fields are present, all values are the right type, and the values make sense given the user's request.

Results: JSON validity (the table-stakes test)

If the model can't produce valid JSON, nothing else matters. This is the "is the model usable at all" floor.

Model	JSON valid	Schema match	Argument types
Qwen3-14B	95%	88%	91%
Llama-3.2-3B	98%	92%	89%

Both models cleared 95% on basic JSON validity. Llama-3.2-3B actually edged out Qwen3-14B here — the smaller 3B model has a tighter tool-calling format that's more reliable in practice. This was the first surprise.

The schema-match column is "does the JSON contain all the required fields, even if some values are wrong." Llama wins on this too — 92% vs 88%. The 3B model is more disciplined about following the schema.

The argument-types column is "are the values the right type (string, integer, boolean, etc.)" — neither model is significantly better here, both around 90%.

Results: argument correctness (the real test)

JSON validity is necessary but not sufficient. The model needs to extract the right values from the user's request. A weather tool call that has the right schema but says "San Francisco" when the user asked about "Tokyo" is useless.

Model	Argument values correct	Multi-arg tasks	Multi-turn context
Qwen3-14B	82%	75%	78%
Llama-3.2-3B	71%	64%	58%

This is where the 14B model pulls ahead. Qwen3-14B extracts the right values from natural language 82% of the time vs 71% for the 3B model. On multi-argument tasks (e.g., "book a meeting for tomorrow at 2pm with Sarah"), Qwen3 wins by 11 points. On multi-turn context (the user says "actually change that to 3pm"), Qwen3 wins by 20 points.

The 20-point gap on multi-turn is the killer stat for me. If your agent does anything that involves follow-up corrections, the smaller model will frustrate users in ways that show up as "the AI isn't listening" complaints. The bigger model is meaningfully better at maintaining context across turns.

Results: error recovery

Task 6 was designed to test the worst case: the API returns an error (rate limit, not found, validation failure), and the model has to figure out what to do next. The two reasonable responses are: (a) retry with corrected arguments, (b) report the error to the user clearly.

Model	Recovers correctly	Hallucinates a fix	Asks for clarification
Qwen3-14B	65%	18%	17%
Llama-3.2-3B	48%	31%	21%

Qwen3 recovers correctly 65% of the time. Llama-3.2-3B recovers 48% of the time and hallucinates a "fix" 31% of the time — meaning it makes up an argument change that wasn't supported by the error response. For a coding agent, "hallucinates a fix" is the failure mode that ships wrong code to production.

Speed and cost

For a local coding agent, the model needs to be fast enough to feel responsive. Both ran on the same M2 24GB Mac through Ollama.

Model	Mean tok/s	First-token latency	Memory used
Qwen3-14B	5.0	1.2 s	9.3 GB
Llama-3.2-3B	24.0	0.4 s	4.1 GB

The 3B model is 4.8x faster. For a coding agent that issues dozens of tool calls per session, this is the difference between a 30-second response and a 6-minute response. The 3B model's speed is also why it won on JSON validity — faster generation means more tokens for thinking through the schema.

Which one to use

Your situation	Pick	Why
Coding agent that issues dozens of tool calls per session	Llama-3.2-3B	Speed matters more than accuracy for interactive use
Agent that handles complex multi-turn user requests	Qwen3-14B	The 20-point multi-turn gap is decisive
Code-generation tasks (one-shot completions)	Qwen3-14B	82% argument accuracy vs 71%
Bulk data processing where errors are tolerable	Llama-3.2-3B	Cheaper, faster, JSON validity is fine
Production system with human-in-the-loop	Qwen3-14B	The 14% hallucinated-fix rate on Llama is a real liability
Edge device / Raspberry Pi / small model server	Llama-3.2-3B	4.1 GB fits almost anywhere

For my actual coding agent, I'd default to Qwen3-14B. The 20-point multi-turn gap is too big to give up, and the 14% hallucinated-fix rate on the 3B model is too risky for code that ships.

But: the 3B model is fast enough to feel instant and cheap enough to run anywhere. For prototypes, demos, and low-stakes workflows, Llama-3.2-3B is the right call.

What I didn't test

• Tool use with parallel calls. When an agent can call 3 tools in parallel instead of sequence, the speed difference is more dramatic. I didn't measure this.
• Schema complexity. Real tool calls can have 20+ arguments and nested objects. My schemas were 3-5 arguments. The gap between the models probably widens with more complex schemas.
• Other 14B-class models. I didn't test DeepSeek-V2-Lite, Yi-34B, Gemma-2-27B. The Qwen3-14B is the strongest of the 14B open-weight models for code, so the "big model wins" pattern probably holds.
• Mistral, Mixtral, other architectures. The two models I tested are both dense transformers. MoE models (Mixtral 8x7B, gpt-oss-20B) might behave differently on tool calling — the active-parameter count matters.
• Streaming tool calls. Some agent frameworks stream the tool call as it's generated. The 3B model might be even more competitive in streaming mode.

Build it yourself

The benchmark is in the public experiments repo:

git clone github.com:Pitambarmahato/hardnumbers-experiments
cd hardnumbers-experiments/tool-calling-benchmark
python -m venv .venv
.venv/bin/pip install -r requirements.txt
ollama pull qwen3:14b
ollama pull llama3.2:3b
.venv/bin/python src/benchmark.py --models all

The 6-task × 2-model = 12-run benchmark takes about 15 minutes on an M2 24GB. Results land in results/tool_calling_<timestamp>.json with per-task scores, JSON validity, and the agent loop trace.

The full data and methodology is at hardnumbers.dev/articles/local-llm-tool-calling-for-ai-agents-qwen3-14b-vs-llama-3-2-3b-on-apple-silicon — the canonical version with the full task definitions and the scoring rubrics. If you have a 7B or 22B-class model you want to add, the benchmark is built to be extended with a single --models flag.