I built an agentic assistant for battery engineering. A physics-based digital twin runs the actual electrochemical simulation, an LLM orchestrates and explains. Partway through, I ran into a failure that had nothing to do with the physics or the prompt wording. It was about how many tool calls I was asking a 7B model to plan in one go, and it changed how I designed every tool after that.
What the system does
The principle behind the whole project: physics does the maths, the model does the language.
A PyBaMM digital twin simulates an LG M50 21700 cell using the open Chen2020 parameter set — a real electrochemical model, not a language model's best guess at a discharge curve. That twin is wrapped as six tools and exposed through FastMCP over stdio, so it's not hard-wired into one agent; any MCP client can drive it. On the other side, a LangGraph ReAct agent, backed by a local Ollama model, reads a plain-English question, decides which tool to call, and turns the physics result into an explanation.
The six tools:
| Tool | What it simulates |
|---|---|
cell_info\ |
Static description of the modelled cell |
simulate_discharge\ |
Constant-current discharge (runtime, delivered capacity/energy) |
simulate_cccv_charge\ |
CC-CV charge (charge time, energy in) |
compare_charging_strategies\ |
Runs and ranks multiple charge rates in one call |
compare_discharge_rates\ |
Runs and compares multiple discharge rates in one call |
simulate_degradation\ |
Capacity fade over N cycles via SEI growth |
The model never touches a number. It picks a tool, reads the structured result, and narrates it. That split only holds up, though, if the agent reliably calls the right tool — and that's where a smaller local model started to show its limits.
Where it broke
Single-tool questions worked from early on. Ask "how long does it run at 1C?" and the agent calls simulate_discharge(c_rate=1.0)\, reads the result, answers. Reliable every time.
The break came with comparisons. Ask "2C versus 0.5C" and the natural ReAct pattern is to call simulate_discharge\ twice, once per rate, then reason over both results. That's standard multi-step tool use, and it's the kind of thing larger models handle without issue. Running on qwen2.5:7b\, though, multi-rate discharge comparisons dropped a rate: the model would not reliably chain two simulate_discharge\ calls, so the answer came back based on only one of the two rates, stated as if it covered the comparison the user actually asked for.
That's a quiet failure. It doesn't throw an error. It produces a confident, well-formatted answer that's simply incomplete, the exact failure mode the physics-grounding was supposed to prevent, just relocated one layer up, into tool orchestration instead of number generation.
It wasn't the only rough edge. Two others, for context: llama3.1:8b\ at one point printed tool calls as literal JSON text instead of actually invoking them, so no simulation ran at all, which is what pushed the model choice to qwen2.5:7b\. And separately, the agent would sometimes speculate about why a number looked a certain way, labelling a delivered capacity above the 5.0 Ah nominal rating as "inefficiency" or "over-discharge," when a capacity above nominal at gentle rates is just normal cell behaviour, not something the simulation had reported as a problem.
Why the multi-call problem happens
It's tempting to read "the model dropped a tool call" as "get a bigger model" and stop there. That's true as far as it goes, but it skips the more useful question: what am I actually asking a 7B model to plan?
A two-call comparison isn't one decision, it's several in sequence — call tool A, hold its result in context, decide to call tool B with different arguments, hold that result too, then reconcile both before answering. Every one of those is a place a small model can drop a step, and on a 7B model run locally, without the depth of training data that gives larger models a stronger prior on "plan, then execute, then synthesize," that margin is thin.
The fix: collapse the plan into the tool, not the model's head
Instead of asking the agent to orchestrate two simulate_discharge\ calls and reconcile them itself, I added compare_discharge_rates\, mirroring the compare_charging_strategies\ tool that already existed for the charging side. It takes a list of rates, runs every simulation internally in Python, and returns runtime and capacity for all of them in one structured result.
The agent's job shrinks to one decision: recognize this is a comparison question, call the one tool built for it, narrate the result. No multi-step plan to hold in working memory, no partial result to silently drop. The system prompt makes the preference explicit too — it tells the model to prefer compare_discharge_rates\ and compare_charging_strategies\ over calling the single-simulation tools repeatedly, so the model isn't left to rediscover the right pattern on its own each time.
This is a small code change with a bigger implication for how I think about tool design for agents. The instinct, especially coming from examples built around large frontier models, is to expose small, composable primitives and let the agent chain them however a question requires. That's the right call when the model doing the chaining is strong. When it isn't, composability becomes a liability, because every extra hop is another place a plan can quietly fall apart. The tool boundary isn't just an API decision — it's a decision about how much sequential reasoning you're willing to hand to the model versus how much you do deterministically in code before the model ever sees it.
Where the line still sits
This didn't make the problem disappear everywhere, and I'd rather say that directly than imply otherwise. Chaining different tools in sequence — a charge comparison followed by a degradation run, with the agent reasoning across both — can still drop a step on a 7B model. The system prompt and the single-call comparison tools solve the specific case that came up most (comparing the same kind of simulation across several rates), not general multi-tool planning. For that, the honest options are a bigger model like qwen2.5:14b\, or an explicit planning step ahead of execution, and I've left that as a known limitation rather than pretending the fix generalizes further than it does.
The broader takeaway
None of this is specific to batteries or to qwen2.5:7b\. Any project running a smaller model locally — for cost, latency, privacy, or air-gap reasons — is going to hit some version of this. The generalizable move is: test the multi-step cases you expect people to actually ask for, watch for the quiet failures rather than just the crashes, and design tool boundaries around what the model actually does reliably, not around what looks cleanest in an architecture diagram. It's a less glamorous kind of engineering than swapping in a bigger model and moving on. It's also the kind that matters when the actual constraint is a system that runs entirely on hardware someone already owns, with no API bill and no dependency on a provider staying online.
If you're running a small model locally, where's the line you've found between letting the agent chain tool calls itself versus just collapsing the sequence into one call in code? Curious whether others have hit this with 7B-class models specifically, or whether it's more about the stdio/MCP setup than the model size.
The full project — the PyBaMM twin, the MCP server, the LangGraph agent, and the rest of the issues hit along the way — is open source: github.com/BinuShefieldShifani/Batterytwin-mcp. Portfolio and other projects at binushefieldshifani.github.io.``
Top comments (9)
The "call simulate_discharge twice and reason over both" failure is such a clean example of something people blame on the model when it's really a tool-shape problem. A 7B model planning two independent calls and then holding both results in working memory to compare is asking it to be an orchestrator; giving it
compare_discharge_ratesas a single tool moves that orchestration into deterministic code where it belongs. We ran into the same thing and the rule that stuck was: if two tool calls are almost always followed by a comparison, that comparison is a tool, not a reasoning step.The quiet part you named — a confident, well-formatted, incomplete answer — is the one that actually scares me, because it survives every smoke test. Did you end up adding anything on the read side to catch it, like having the tool return the set of rates it actually simulated so the agent (or a cheap validator) can notice one is missing before it narrates? Curious whether you found the fix was mostly tool granularity or also constraining the ReAct loop itself.
No read-side check, honestly the fix was tool granularity plus a system-prompt nudge to prefer compare_discharge_rates over calling simulate_discharge twice. Didn't touch the ReAct loop itself.
Your question did surface something though: that tool already returns a results list with every c_rate it actually ran. Nothing currently checks that against what was asked, but the data's right there. Might be worth adding.
Great example of moving the brittle part of the plan into deterministic code. One extra guard I would add is to make completeness machine-checkable in the batch tool's result. Return the requested item count/digest, one result keyed by each requested rate, an explicit missing/failed list, and a
partialflag. Then the agent cannot quietly narrate one successful simulation as a complete comparison.\n\nThe collapsed tool also needs bounded batch size, per-item timeout/error status, stable ordering or IDs, and cancellation semantics. I would put 1, 2, max-N, duplicate, invalid, and one-failure-among-many cases in the eval set. That preserves the reliability benefit of one call without hiding partial execution behind a convenient abstraction."A confident, well-formatted answer that's simply incomplete" is the nastiest failure class in tool use — nothing throws, no retry triggers, a whole reasoning step just vanishes behind a plausible artifact. In my experience it even sails through schema validation, since every field that is there is present and well-typed.
Your fix — adding compare_discharge_rates so the agent stops orchestrating two simulate_discharge calls itself — is the right instinct, and the underlying principle deserves a name: tool design as capability budgeting. A 7B model has a fixed chaining budget, and every orchestration step the tool absorbs is budget handed back for the reasoning only the model can do. Since the tool already returns per-rate runtime and capacity in one structured result, +1 to the echo-back-the-simulated-rates idea in the thread — that turns a silently missing condition into something a cheap validator can catch.
A useful read-side contract is to make the comparison tool return both requested_rates and completed_rates, plus a missing_rates array and a run_id for each simulation. Then a tiny deterministic validator can reject the result if completed_rates does not cover the request before the language model sees it.
I would also keep that check at the harness boundary, not only in the prompt: treat a missing rate as an incomplete tool result, record it as a bounded failure, and never let the model narrate a partial comparison as complete. That gives small local models a safer failure mode without pretending the general multi-tool planning problem is solved.
The harness-boundary point is right, and it's the correction I needed, I was still thinking "give the model better information" when the actual fix is "don't let it see an incomplete result at all." requested_rates / completed_rates / missing_rates is going in.
run_id per simulation I'm less sold on for this specific case, though. PyBaMM runs here are fast, deterministic, and synchronous, there's no retry, caching, or async fan-out where I'd need to trace a result back to a specific run. Feels like it'd earn its keep in a setup with concurrent or long-running simulations, but here it might just be complexity without a failure mode it's solving for. Are you thinking of a case where it'd matter even in a synchronous setup like this one, or is that more future-proofing for when it stops being synchronous
The dropped second call got worse for me exactly when both calls hit the same tool with different args — once the first result is in context, the model seems to treat the second call as redundant and just narrates over the gap. Two things helped: making the partial path loud, i.e. the orchestrator counts how many simulations actually ran against how many the question asked for and refuses to answer if they don't match, and removing the single-rate tools from the exposed list once the compare tools exist, so the wrong choice isn't available at all. A system prompt preference works until it doesn't; not exposing the option is the only version I trust. Did you consider dropping simulate_discharge from the tool list for comparison questions, or do you still need it reachable for the single-rate case?
The number-of-tool-calls-per-plan constraint is underrated for small local models. I've seen the same thing where a 7B model is fine picking one tool but falls apart sequencing three, so collapsing multi-step operations into a single higher-level tool like your compare_charging_strategies does more than any prompt tweak. Keeping the model on language and letting the twin own the numbers is the split that makes a local setup trustworthy.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.