DEV Community

Cover image for I Added a Human Approval Step to My AI Agent. It Charged the Card Twice.
Mothilal M
Mothilal M

Posted on Edited on AI-assisted

I Added a Human Approval Step to My AI Agent. It Charged the Card Twice.

Adding an approval step feels like making an agent safer. But what happens to work that already finished before the pause? A small test of 10xGraph produced this:

$ python parallel_tools.py sqlite start
  charged $49.00  key=f9bb251d-dfc5-40e5-9c02-1d1b3195c021:call_charge
  paused. charges so far: 1
$ python parallel_tools.py sqlite resume
  charged $49.00  key=f9bb251d-dfc5-40e5-9c02-1d1b3195c021:call_charge
  resumed. charges now: 2
Enter fullscreen mode Exit fullscreen mode

One purchase. One human saying yes. Two charges.

Nobody lost money. The card was a text file, and the human approval was a resume command in a second process. But the agent was built on 10xGraph, the open-source Python framework I help maintain, so there was nobody else to blame.

I'd set out to answer a narrow question: what actually happens when an agent stops to wait for a person, and the process dies before the person answers? With an AI assistant, I put together five small scripts and tested them against 10xGraph 0.10.3 and LangGraph 1.2.14. One result turned out to be my own test lying to me, and in one case LangGraph handled things better than we do. It's all below, with the code.

Why the pause has to survive a restart

Human-in-the-loop sounds like the easiest feature in an agent. The agent reaches a risky step, stops, someone clicks Approve, and it carries on.

The trouble is the gap between "stops" and "clicks". Your approver might answer in twenty seconds or on Monday morning. Your Python process won't be sitting there all that time, because you'll deploy, or a pod gets rescheduled overnight. So the pause can't live in memory. It has to be written down somewhere, and a different process has to be able to pick it up.

Both frameworks handle this the same way. You compile the graph with a checkpointer, give each conversation a thread_id, and call interrupt() where you want the human. The first run saves its state and stops. Later, any process that can reach the same database resumes the thread with the human's answer.

That part worked in every test I ran. The paused run came back in a new process every time. The problems were all about what ran twice on the way back.

Test 1: the approver got pinged twice

Here's the approval node from a small refund agent. It tells the approver (a log file standing in for Slack), then waits:

def wait_for_approval(state: RefundState) -> RefundState:
    print("  wait_for_approval: started")
    notify_approver(f"Approve ${state.amount} refund on {state.order_id}?")
    decision = interrupt({"order_id": state.order_id}, message="Approve this refund?")
    state.status = "approved" if decision and decision.get("approved") else "rejected"
    return state
Enter fullscreen mode Exit fullscreen mode

The test used a SQLite checkpointer, let the process exit at the pause, and resumed from a fresh one:

$ python approval_replay.py start
  wait_for_approval: started
  -> #refund-approvals: Approve $49.99 refund on order-1042?
  paused, process exiting
  messages sent to approver: 1
$ python approval_replay.py resume
  wait_for_approval: started
  -> #refund-approvals: Approve $49.99 refund on order-1042?
  issue_refund: approved
  messages sent to approver: 2
Enter fullscreen mode Exit fullscreen mode

The approver got asked twice: once when the agent paused, and again after they'd already said yes.

I wrote the same graph in LangGraph with SqliteSaver. The output was identical, line for line.

The reason is mechanical. Python can't freeze a function halfway through and write it to disk. There's no way to put a half-finished stack frame in SQLite and load it in another process tomorrow. So interrupt() works by raising. The framework catches it, keeps the state from before the node started, and the process is free to go. On resume the node runs again from its first line, and this time interrupt() returns the human's answer instead of raising.

first run                         resume (new process)
---------                         --------------------
wait_for_approval                 wait_for_approval
  print("started")                  print("started")
  notify_approver()                 notify_approver()      <- again
  interrupt()  -> raise, save       interrupt()  -> {"approved": True}
                                  issue_refund
Enter fullscreen mode Exit fullscreen mode

None of this is hidden. LangGraph's interrupts page has a section titled "Side effects called before interrupt must be idempotent", and 10xGraph's docs say the same. It still caught me, because "tell the human, then wait" is the natural order to write it in.

The fix is to give the side effect its own node:

def ask_approver(state: RefundState) -> RefundState:
    notify_approver(f"Approve ${state.amount} refund on {state.order_id}?")
    return state

graph.add_edge(START, "ask_approver")
graph.add_edge("ask_approver", "wait_for_approval")  # this node now only calls interrupt()
Enter fullscreen mode Exit fullscreen mode

When ask_approver finishes, the framework saves a checkpoint, and that checkpoint is the record that the message went out. The resume starts at wait_for_approval:

$ python approval_replay_fixed.py resume
  wait_for_approval: started
  issue_refund: approved
  messages sent to approver: 1
Enter fullscreen mode Exit fullscreen mode

Test 2: two tools in one turn

That was the easy version. The case that actually worries me is tool calls.

Models like to call several tools at once (OpenAI does it by default). So picture a shopping agent where the model, in a single turn, asks for charge_card(49.0) and confirm_shipping("1 Main St"), and confirm_shipping calls interrupt() to check the address with a human. The charge finishes. The confirmation pauses. When the human answers, the tool step resumes. Does the card get charged again?

Setup Charges after resume
LangGraph create_agent, SQLite, two processes 1
10xGraph, in-memory checkpointer 1
10xGraph, SQLite, two processes 2
10xGraph, in-memory, through astream() 2

The two frameworks reach "1" in different ways, and the difference explains the 2s.

LangGraph's create_agent sends each tool call off as its own task (it's Send("tools", [tool_call]) in the source). When a task finishes, its result is saved with the checkpoint. On resume, only the unfinished task, confirm_shipping, runs again. In this tested resume path, the completed charge task was not replayed. A crash between an external charge and saving its result still needs idempotency at the payment boundary.

10xGraph runs all the tool calls from one turn together inside the tool node, using asyncio.gather, each on its own copy of the state. Since they're one step, a resume re-runs all of them, so we keep a ledger. Each finished call is recorded under the id of the assistant message that asked for it plus the tool call id, and on replay a recorded call returns its saved result instead of running. That's why the in-memory row says 1.

But the ledger is only implemented in the in-memory and Postgres checkpointers. SQLite doesn't have it, and the streaming path doesn't look at it at all. Our docs list both gaps on the replay-safe tools page. I still didn't enjoy seeing charges now: 2 in my own terminal.

My test lied to me first

My first LangGraph run also printed two charges, and I came very close to writing "both frameworks get this wrong".

Then I logged every model call. LangGraph had correctly skipped the finished charge. The second charge came from my fake model. It kept a counter of how many times it had been called, the counter started at zero in the new process, and it cheerfully asked for the charge all over again.

If you test resume logic with a scripted model, make it decide from the messages it's given rather than from its own counter. Otherwise your harness is mostly testing itself.

The fix that works on any backend

Frameworks can cut down on double execution. They can't promise zero, because a process can always die between "the payment API said OK" and "the framework wrote that down". The fix that holds up sits at the boundary: give every side effect an idempotency key, and make the receiving side refuse duplicates.

The key has to come from something the checkpoint saved, so it's identical on replay. In 10xGraph a tool can ask for tool_call_id and state as parameters; the framework injects them and leaves them out of the schema the model sees:

def charge_card(amount: float, tool_call_id: str, state: AgentState) -> str:
    """Charge the customer's card."""
    # The assistant message that asked for this call + the call id = one real invocation.
    idempotency_key = f"{state.context[-1].message_id}:{tool_call_id}"
    ...
Enter fullscreen mode Exit fullscreen mode

Same key in both processes:

$ python parallel_tools.py sqlite start
  charged $49.00  key=f9bb251d-dfc5-40e5-9c02-1d1b3195c021:call_charge
$ python parallel_tools.py sqlite resume
  charged $49.00  key=f9bb251d-dfc5-40e5-9c02-1d1b3195c021:call_charge
Enter fullscreen mode Exit fullscreen mode

Hand that key to whatever does the real work. Stripe API v1 accepts an Idempotency-Key header and returns the saved result for a repeat while that key is retained. Keys may be pruned after at least 24 hours, so plan for approvals that arrive later than the provider's retention window. For your own services, record the key and apply the business change in the same database transaction, with a unique constraint on the key. A key recorded separately from the side effect leaves another crash window.

Don't use the tool call id on its own. It isn't guaranteed to be unique across turns. Some models and OpenAI-compatible servers hand out short ids like call_1 every turn, and then tomorrow's legitimate charge gets thrown away as a duplicate of today's. Pairing it with the message id avoids that. (In LangGraph, InjectedToolCallId gets you the call id.)

What 10xGraph still gets wrong here

Since this is a post about my own framework charging a card twice, here's everything I found:

  • The SQLite checkpointer has no tool ledger. If you're on SQLite and your tools have side effects, you get the 2 from the table.
  • stream() and astream() skip the ledger even with the in-memory checkpointer. Streaming is how most UIs talk to an agent, so this is the bigger of the two gaps.
  • With synchronous invoke() and SQLite, the process hangs at exit unless you call checkpointer.release(). Every script in this post does it for that reason.
  • I didn't test the Postgres ledger for this post. The docs say it's there, but I'd rather you check than take my word for it.
  • For parallel tools, LangGraph's one-task-per-call design avoids this whole problem, and I think it's the better design for this case.

10xGraph is pre-1.0 with a small community. Writing this up is partly me saying the gaps out loud so they get fixed.

What I do now

  1. Nothing with a side effect goes above interrupt() in the same node.
  2. Side effects get their own node, so a checkpoint marks them as done.
  3. Anything that touches the outside world carries an idempotency key built from saved state, and the system on the other end enforces it.
  4. Resume tests run in a second process. Running both halves in one process hides too much, like an in-memory cache or a fake model's counter.

All five scripts are here: GitHub gist: five runnable examples. None of them needs an API key. The 10xGraph ones need pip install "10xgraph[sqlite_checkpoint]", and the LangGraph ones need langchain, langgraph and langgraph-checkpoint-sqlite. If you get different numbers, I want to know.

10xGraph is on GitHub at 10xGraph/10xGraph if you want to read the ledger code, or tell me how you'd fix the SQLite side.

How are you handling approvals in your agents: interrupt() inside the graph, or a queue and a webhook outside it? And has a resume ever bitten you in production? I'd like to hear what broke.


Top comments (0)