TL;DR. If your LLM agent uses tool calling, the model can return several tool calls in a single turn, and most agent loops execute those calls at the same time. For about a year I treated the bugs that fell out of this as prompt problems: tighter schemas, better tool descriptions, more few-shot examples. The bugs were not coming from the prompts at all; they were concurrency problems: several handlers doing read-modify-write against shared state with no coordination, which is the same read-modify-write race databases have handled for decades, just triggered by a new caller. Below are four production incidents, the mechanism behind each, and the specific guard that fixed it. The through-line is straightforward: parallel tool calling amounts to concurrent writes against shared state, and nothing in the framework will tell you that.
Here is the setup, because the wording in the docs hides it.
When you send tools to the OpenAI or Anthropic API, the model can reply with more than one tool call in one assistant turn. OpenAI calls this parallel function calling and returns them as a tool_calls array (function-calling guide: platform.openai.com/docs/guides/function-calling). Anthropic documents the same behavior for tool use (docs.anthropic.com/en/docs/build-with-claude/tool-use). The part that matters happens next, and it happens in your code, not theirs. The model does not run anything. Your executor takes that array and runs the handlers, and in every agent loop I have read or written, the default is to run them concurrently (an asyncio.gather, a thread pool, a task group). The model emitted them together because, from where it sits, the calls look independent. It has no idea that call A and call B both land on account 4021.
That gap (independent to the model, contended at the data layer) is the whole article. What follows is four times it cost us something real, in the order we hit them.
Notice how the framing pushes you the wrong way. Every doc I have read sells parallel tool calling as a latency win: the model can ask for the weather in three cities at once, so why make it wait. True, and for read-only calls that is the entire story. But the same feature, pointed at tools that write, is unsynchronized concurrent access to shared state, and none of the guides say that part out loud. You opt into a distributed-systems problem by flipping what reads like a performance setting. That is the part worth being suspicious about before you ship: a setting that reads as performance tuning is actually opting you into concurrent writes.
1. The double write (lost update)
What we saw. A support agent could grant account credits. One afternoon, 14 accounts ended the day exactly one credit short of what our own logs said we had granted. Not random amounts. Each was missing precisely one apply_credit out of the two the model had issued in the same turn.
The mechanism. The model, trying to be thorough, split a goodwill gesture into two apply_credit calls of 2500 cents each (it does this more than you would think, especially when it reasons about two separate reasons to compensate someone). Both calls hit the same handler at the same time. The handler read the balance, added to it, and wrote it back:
def apply_credit(account_id: str, amount_cents: int) -> dict:
balance = db.get_balance(account_id) # both siblings read 5000
new_balance = balance + amount_cents # both compute 7500 from 5000
db.set_balance(account_id, new_balance) # last write wins, one credit vanishes
return {"balance": new_balance}
Both siblings read 5000. Both computed 7500. Both wrote 7500. The account landed at 7500 instead of 10000, and one 2500-cent credit evaporated with no error anywhere. Martin Kleppmann spends a good chunk of the transactions chapter in Designing Data-Intensive Applications on exactly this read-modify-write race (he calls it the lost update, and it predates LLMs by decades). The only new thing here is what pressed the button.
The guard. Serialize the read-modify-write per account, and dedupe on the call id so a retried delivery does not double-apply:
import threading
_account_locks: dict[str, threading.Lock] = {}
_registry_lock = threading.Lock()
def _lock_for(account_id: str) -> threading.Lock:
with _registry_lock:
return _account_locks.setdefault(account_id, threading.Lock())
def apply_credit(account_id: str, amount_cents: int, tool_call_id: str) -> dict:
with _lock_for(account_id):
if db.already_processed(tool_call_id): # edge case: this exact call ran already
return {"status": "duplicate_ignored", "balance": db.get_balance(account_id)}
balance = db.get_balance(account_id) # read
new_balance = balance + amount_cents # modify
db.set_balance(account_id, new_balance) # write
db.mark_processed(tool_call_id)
return {"status": "ok", "balance": new_balance}
In a single process that is enough. Across processes (you will get there) an in-memory lock is theater, so push the atomicity into the store instead: UPDATE accounts SET balance_cents = balance_cents + %(amt)s WHERE id = %(id)s is atomic and needs no read in app code. (If your store cannot do that, a row lock via SELECT ... FOR UPDATE around the same block gets you the same guarantee.)
The reason we caught this one at all is a nightly reconciliation that summed the credit ledger against stored balances and alerted on any drift. If you take one operational thing from this piece, take that: a cheap after-the-fact check that recomputes state from an append-only log will surface every one of these four bugs, usually before a customer does. We had that check for balances. We did not have it for refunds, which is exactly why case 3 ran unnoticed for four days.
2. Read-after-write staleness inside one turn
What we saw. A shopping agent quoted checkout totals that were low. Not always. Roughly 1 in 200 sessions, the number the agent read back to the user was missing the item it had just added. Users noticed before we did, which is its own kind of embarrassing.
The mechanism. The model emitted add_to_cart(item) and get_cart_total() in the same turn. Because the model emits every call in a turn before it sees any result, it cannot hold a data dependency it is aware of. The handlers had one anyway: the total depends on the add. Run concurrently, get_cart_total frequently won the race and read the cart before the add had committed. The model then reported the stale total in perfect good faith. (This is the quiet one, because nothing throws. You just serve a wrong number with total confidence.)
The guard. Do not run a read against state that a sibling call is mutating. The cheap version: partition the tool_calls array into mutating and read-only, run the mutations first (serialized, per case 1), then run the reads. The model never needs to know. You are simply declining to honor its implied ordering with real parallelism when the calls touch the same state.
I will be honest that this one resisted a clean fix for a while, because "which tools touch the same state" is not something the framework knows about your handlers. We ended up tagging each tool as reads, mutates, or pure and letting the executor schedule on those tags. Boring. It held.
If you want the shape of it: the executor groups a turn's calls by tag, awaits the mutates group to completion in a fixed order, then dispatches the reads group against the settled state. Two extra lines of scheduling, and the total the model reads back is always the total after the add, never a snapshot from halfway through.
3. Idempotency-key collision (my own fix caused this one)
What we saw. After I shipped the case 1 guard, refunds started going missing. Over one week, 9 customers who were owed two separate refunds got one. My monitoring for case 1 stayed green the whole time, which is why it took four days to find.
The mechanism. For the case 1 dedupe I had gotten clever and keyed idempotency on the semantics of the operation, hash(account_id, amount_cents, "refund"), so a retried delivery of the same refund would not double-pay. That is correct for retries. It is wrong for two legitimately identical calls in one turn. When a customer was owed two 2500-cent refunds (two separate orders that happened to be the same price), the model issued two identical refund calls, my hash collapsed them to one key, and the second was dropped as a "duplicate." The guard I had written to stop a data race was itself dropping legitimate refunds.
The guard. Split the two questions that "idempotency key" smears together. To dedupe accidental re-delivery (a framework retry, a double-dispatch), use the tool_call.id the API already gives you: it is unique per call in the response, so two distinct refunds get two distinct ids and both go through. To dedupe an intended effect (the user should only ever be refunded once for order X), key on the business fact that makes it unique, the order id, never the amount. If you cannot name the specific field that makes an operation unique, then the key you are hashing is not really identifying the operation. It is identifying a set of arguments that can happen to match, so it will sometimes drop calls that are genuinely distinct.
(Transaction-id collisions are the same bug from the other side: two parallel handlers each run txn_id = max(existing) + 1, both read the same max, both claim the same id. Let the store mint ids, or reuse the tool_call.id, instead of a read-then-increment in app code.)
4. Partial failure across a parallel group
What we saw. A travel agent could, in one turn, emit book_hotel, book_flight, and charge_card. Three cards got charged for hotels that were never booked before we caught it. The model received the three tool results (two ok, one error), apologized to the user, and moved on. The money stayed moved.
The mechanism. Parallel tool calls have no transaction boundary around them. Your executor runs three handlers, one fails after the others have already committed side effects, and there is no rollback because there was never a transaction. The model is not a coordinator. It sees a mixed bag of results after the fact and does whatever its next-token instincts suggest, which is usually to say sorry, not to issue a compensating refund.
The guard. Two options, and I have shipped both. First, do not model a multi-step transaction as a set of independent parallel tools at all. Collapse the atomic unit into one tool (book_trip) that owns its own transaction or saga internally, so the model makes one call and your code owns the all-or-nothing. Second, when you genuinely cannot collapse it, disable parallelism for that toolset so the model has to sequence the work and you can stop after the first failure:
from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
model="gpt-4.1",
messages=messages,
tools=tools,
parallel_tool_calls=False, # model returns at most one tool call per turn
)
Anthropic exposes the same switch through tool_choice with disable_parallel_tool_use: true (as of mid-2026). Neither switch is a real distributed transaction. They just stop the model from opening N side effects you have no clean way to close.
If the work is irreducibly multi-step and refuses to live in one tool, be honest that you are now writing a saga. Each step needs a compensating action (refund_card to undo charge_card), and something durable has to drive those compensations when a later step fails, because the model will not. That is far more machinery than most agents carry, which is itself the strongest argument for collapsing the unit into a single tool and keeping the transaction inside your own code, where a database can actually enforce all-or-nothing.
When this does not bite you
I am not telling you to put a mutex on every tool. Most of an agent's tools are read-only, and for those, parallel calls are pure upside (fan out ten retrievals, good). This class of bug needs three things to line up at once, and if any one is missing you can ignore me:
- Two or more calls in the same turn that mutate state. Purely read-only toolsets are safe, so fan them out freely.
- The calls touch the same key. Two writes to different accounts do not contend at all. The danger is specifically same-row, same-turn.
- Your app code does the read-modify-write. If the mutation is a single atomic statement in a store with real isolation, the database is already doing the coordination and your handler has nothing left to protect.
There is also a lucky case worth naming: naturally idempotent, set-to-value mutations (set_status("shipped")) survive a lost-update race because the last writer lands on the value you wanted anyway. They do not survive read-after-write, though. A sibling can still read the pre-write status and mislead the model. So "we only ever set values, never increment" buys you case 1, not case 2.
And the honest one: at low tool-call fan-out you may never see any of this. If your agent emits more than one mutating call per turn maybe once a week, the race is real but rare, and rare races present as "flaky, could not reproduce, closed." That does not mean the bug is absent. It means the bug is currently cheap enough to ignore, until it is not (ours was a refund, so the day it stopped being cheap arrived with a finance ticket attached).
Where I'd push back on this
Steelman the other side, because it has a case. Most production agents are read-heavy, the model emits conflicting parallel mutations rarely, and bolting locks onto handlers adds latency and a brand new failure mode (deadlocks, an unbounded lock registry, a lock ordering you now have to reason about). You could argue I am importing distributed-systems ceremony into what is, most days, a chatbot that calls a search API twice. That is fair, and I have over-built this before.
Here is the concession. The default I actually ship now is not "lock everything." It is duller than that. Read-only and pure tools run in parallel, untouched. State-mutating tools run serialized per turn, in a defined order, and only the ones that need atomicity get a store-level guard. For genuinely transactional multi-step work, I collapse it into one tool instead of trusting a parallel group to behave. That is slower on the rare turn with two mutations, it is correct, and I stopped losing refunds. If your workload is all reads, you pay none of this cost and you should not adopt any of it.
Objections I'd accept. "Push atomicity into the datastore, not app-level locks." Yes, wherever the store supports it. The in-process lock is a stopgap for single-process deployments, and I said as much. "This is just CS 101 concurrency, there is nothing LLM-specific here." Correct, and that is the point. The mechanism is 40 years old. What is new is that a model now introduces the concurrency invisibly, out of natural language, with nothing annotating that two of its calls collide.
Objections I wouldn't accept. "Just prompt the model not to emit conflicting calls in the same turn." No. You can lower the rate with prompting, but reducing the probability of a data race does not remove it, and a rarer race is harder to reproduce and harder to debug on call than a frequent one. "Then turn parallel tool calls off globally and forget it." That buys correctness and throws away the real latency win on read-heavy fan-out, which is most turns. Scope the disable to mutating toolsets: correctness where you have shared state, parallelism where you do not, and a tag on each tool that says which is which.
The frameworks will get here eventually (some are adding tool-level concurrency hints already). Until they do, it is safer to assume any two mutating calls in the same turn can contend on the same row, because in our workload they did a few times a week.
Top comments (0)