A reader pointed out a bug in a budget guard I posted: two agents can read the same budget_state.json, both pass the check, both write back, and one charge disappears. They were right. Here is why it happens and a fix you can copy.
The bug: load, check, save is not atomic
import json
def charge(amount, cap, path="budget_state.json"):
state = json.load(open(path)) # agent A and agent B both read spent=9.00
if state["spent"] + amount > cap:
raise RuntimeError("budget exceeded")
state["spent"] += amount # A writes 9.50, B writes 9.40
json.dump(state, open(path, "w")) # B wins, A's 0.50 is gone
With a cap of 10.00, both calls pass, and the file ends up under-counting. Two more holes sit in the same pattern:
- Spend is recorded after the call returns. If the call raises (timeout, 5xx), nothing is counted, but the provider may still bill you.
- Dollars are the only stop condition. A broken call tends to fail fast and cheap, over and over.
The fix: reserve first, in one atomic statement
SQLite gives you atomic updates with the standard library:
import sqlite3
def open_db(path="budget.db"):
conn = sqlite3.connect(path, timeout=10)
conn.execute("CREATE TABLE IF NOT EXISTS budget (id INTEGER PRIMARY KEY, spent REAL NOT NULL)")
conn.execute("INSERT OR IGNORE INTO budget (id, spent) VALUES (1, 0)")
conn.commit()
return conn
def reserve(conn, amount, cap):
cur = conn.execute(
"UPDATE budget SET spent = spent + ? WHERE id = 1 AND spent + ? <= ?",
(amount, amount, cap),
)
conn.commit()
return cur.rowcount == 1 # 1 row changed = reservation succeeded
def refund(conn, amount):
conn.execute("UPDATE budget SET spent = MAX(spent - ?, 0) WHERE id = 1", (amount,))
conn.commit()
The check and the write happen in a single statement, so two agents cannot both slip under the cap.
Wrap the call: reserve before, refund only if it never left
class BudgetExceeded(Exception): pass
def guarded_call(conn, fn, estimate, cap, *args, **kwargs):
if not reserve(conn, estimate, cap):
raise BudgetExceeded(f"cap {cap} reached")
try:
return fn(*args, **kwargs)
except ConnectionError: # request never reached the provider
refund(conn, estimate)
raise
# any other error: keep the reservation, you may still be billed
Add a second stop condition: consecutive failures
class FailureBreaker:
def __init__(self, limit=5):
self.limit, self.count = limit, 0
def ok(self): self.count = 0
def fail(self):
self.count += 1
if self.count >= self.limit:
raise RuntimeError(f"{self.limit} failures in a row, stopping run")
Call breaker.ok() after a success and breaker.fail() in your except block. A runaway retry loop now stops at 5 attempts, not at your whole budget.
Further reading
- SQLite transactions and atomic updates: https://www.sqlite.org/lang_transaction.html
- Python
sqlite3module docs: https://docs.python.org/3/library/sqlite3.html
Do you cap on call rate per minute as well, or is dollars plus consecutive failures enough in your setup?
Top comments (0)