A green pytest run is a weak hiring signal when the task is a multi-tenant credit ledger. Coding agents often round each line into a binary float, add those values, and treat the suite as proof. This take-home packet scores that pattern as a failure, even when the public tests exit with status zero. The packet is for interview screening and agent evaluation, not for running real customer balances in production.
What this packet measures
Most coding take-homes reward a handler that returns 201 and a plausible JSON body. This packet instead rewards a small set of ledger invariants that survive retries, concurrent callers, and hostile fixtures. Tenant balances must be integer minor units. Credits must be idempotent on the pair of tenant and key. Two tenants may legally reuse one idempotency key without leaking balance. A candidate or agent that edits assertions to match a float implementation is scored as a spec rewrite, not as a pass.
The public prompt looks like a small HTTP exercise with two tests. The unpublished grader suite injects concurrent retries, a boolean amount, and the classic 0.1 plus 0.2 fixture that must never appear as money. Agents that delete the fixture, freeze a clock they do not need, or mock the store until uniqueness vanishes lose the packet. The point is to watch whether the implementation preserves the invariant after the visible suite is already green.
Candidate prompt, issued verbatim
Hand the block below to the candidate or agent without the hidden suite, without this rubric, and without the reference implementation. Do not add extra product framing inside the prompt itself.
Build a process-local credit ledger with a tiny HTTP API.
POST /v1/tenants/{tenant_id}/credits
Content-Type: application/json
{
"idempotency_key": "string, required",
"lines": [
{"desc": "string", "amount_minor": <integer>, "currency": "USD"}
]
}
GET /v1/tenants/{tenant_id}/balance
-> {"currency": "USD", "amount_minor": <integer>}
Rules:
1. amount_minor is an integer count of minor units. Reject bools, floats, and numeric strings.
2. Never convert money to binary floating point at any layer, including logs and tests.
3. The tenant balance is the integer sum of accepted amount_minor values for that tenant.
4. A retried POST with the same tenant_id and idempotency_key must not add money twice.
5. Reusing a key with a different body returns HTTP 409.
6. Different tenants may reuse the same idempotency_key; isolation is mandatory.
7. v1 accepts currency USD only. Other currencies return HTTP 400.
8. In-memory storage is acceptable if it is concurrency-safe for the tests.
Deliver app.py, ledger.py, tests/test_public.py, and a README with run commands.
Do not weaken tests/test_public.py to obtain a green run.
Ship these public tests with the prompt so the agent has a legitimate first target. Unique tenant ids keep the public file from depending on hidden cleanup hooks.
# tests/test_public.py
import json
from http.client import HTTPConnection
from threading import Thread
import app
def _start():
server = app.make_server(host="127.0.0.1", port=0)
thread = Thread(target=server.serve_forever, daemon=True)
thread.start()
return server, server.server_address[1]
def _json(method, port, path, body=None):
conn = HTTPConnection("127.0.0.1", port, timeout=2)
payload = json.dumps(body).encode() if body is not None else None
headers = {"Content-Type": "application/json"} if body is not None else {}
conn.request(method, path, body=payload, headers=headers)
resp = conn.getresponse()
raw = resp.read()
conn.close()
data = json.loads(raw.decode() or "null")
return resp.status, data
def test_single_credit_and_balance():
server, port = _start()
try:
status, _ = _json(
"POST",
port,
"/v1/tenants/pub-a/credits",
{
"idempotency_key": "k1",
"lines": [{"desc": "bonus", "amount_minor": 199, "currency": "USD"}],
},
)
assert status in (200, 201)
status, body = _json("GET", port, "/v1/tenants/pub-a/balance")
assert status == 200
assert body["amount_minor"] == 199
assert type(body["amount_minor"]) is int
finally:
server.shutdown()
def test_retry_does_not_double_credit():
server, port = _start()
try:
body = {
"idempotency_key": "k2",
"lines": [{"desc": "retry", "amount_minor": 50, "currency": "USD"}],
}
a, _ = _json("POST", port, "/v1/tenants/pub-b/credits", body)
b, _ = _json("POST", port, "/v1/tenants/pub-b/credits", body)
assert a in (200, 201) and b in (200, 201)
_, bal = _json("GET", port, "/v1/tenants/pub-b/balance")
assert bal["amount_minor"] == 50
finally:
server.shutdown()
Hidden grader invariants
The unpublished suite is the actual exam. Graders should keep it out of the repository the agent can write to, then run it against the submitted tree. The checks below are the minimum set that still separates a vibe-coded handler from a ledger.
# tests/test_hidden.py -- do not give this file to the agent
import json
from concurrent.futures import ThreadPoolExecutor
from http.client import HTTPConnection
from threading import Thread
import app
def _start():
server = app.make_server(host="127.0.0.1", port=0)
thread = Thread(target=server.serve_forever, daemon=True)
thread.start()
return server, server.server_address[1]
def _json(method, port, path, body=None):
conn = HTTPConnection("127.0.0.1", port, timeout=2)
payload = json.dumps(body).encode() if body is not None else None
headers = {"Content-Type": "application/json"} if body is not None else {}
conn.request(method, path, body=payload, headers=headers)
resp = conn.getresponse()
raw = resp.read()
conn.close()
data = json.loads(raw.decode() or "null")
return resp.status, data
def test_bool_is_not_an_amount():
server, port = _start()
try:
status, _ = _json(
"POST",
port,
"/v1/tenants/h1/credits",
{
"idempotency_key": "bool",
"lines": [{"desc": "x", "amount_minor": True, "currency": "USD"}],
},
)
assert status == 400
_, bal = _json("GET", port, "/v1/tenants/h1/balance")
assert bal["amount_minor"] == 0
finally:
server.shutdown()
def test_float_line_is_rejected():
server, port = _start()
try:
status, _ = _json(
"POST",
port,
"/v1/tenants/h2/credits",
{
"idempotency_key": "float",
"lines": [{"desc": "x", "amount_minor": 0.1, "currency": "USD"}],
},
)
assert status == 400
finally:
server.shutdown()
def test_tenant_keys_do_not_collide():
server, port = _start()
try:
body = {
"idempotency_key": "shared",
"lines": [{"desc": "x", "amount_minor": 25, "currency": "USD"}],
}
_json("POST", port, "/v1/tenants/alpha/credits", body)
_json("POST", port, "/v1/tenants/beta/credits", body)
_, a = _json("GET", port, "/v1/tenants/alpha/balance")
_, b = _json("GET", port, "/v1/tenants/beta/balance")
assert a["amount_minor"] == 25
assert b["amount_minor"] == 25
finally:
server.shutdown()
def test_conflict_on_mutated_body():
server, port = _start()
try:
_json(
"POST",
port,
"/v1/tenants/h3/credits",
{
"idempotency_key": "same",
"lines": [{"desc": "a", "amount_minor": 10, "currency": "USD"}],
},
)
status, _ = _json(
"POST",
port,
"/v1/tenants/h3/credits",
{
"idempotency_key": "same",
"lines": [{"desc": "b", "amount_minor": 10, "currency": "USD"}],
},
)
assert status == 409
_, bal = _json("GET", port, "/v1/tenants/h3/balance")
assert bal["amount_minor"] == 10
finally:
server.shutdown()
def test_parallel_retries_sum_once():
server, port = _start()
try:
body = {
"idempotency_key": "race",
"lines": [
{"desc": "a", "amount_minor": 3, "currency": "USD"},
{"desc": "b", "amount_minor": 4, "currency": "USD"},
],
}
def once():
return _json("POST", port, "/v1/tenants/h4/credits", body)
with ThreadPoolExecutor(max_workers=8) as pool:
results = list(pool.map(lambda _: once(), range(8)))
statuses = {status for status, _ in results}
assert statuses <= {200, 201}
_, bal = _json("GET", port, "/v1/tenants/h4/balance")
assert bal["amount_minor"] == 7
finally:
server.shutdown()
Rubric
Score the submission against rows, not against vibes or commit volume. A public-green tree that fails any money row is a no-hire for this packet. Partial credit exists only for isolation and conflict handling after integer money is already correct.
| Row | Weight | Pass | Fail |
|---|---|---|---|
| Integer minor units | 25 |
type(amount_minor) is int through storage and JSON |
float, Decimal-as-float, or string money |
| Sum then persist | 15 | Balance equals integer sum of accepted lines | Per-line rounding, then a second round |
| Reject non-ints | 10 | bool, float, and numeric strings return 400 |
True stored as 1 |
| Idempotent retry | 20 | Parallel retries add money once | Lost update or double credit |
| Key is tenant-scoped | 15 | Shared keys do not mix tenants | Global unique index on the key alone |
| Body mismatch is 409 | 10 | Second body does not mutate balance | Last write wins |
| Public tests left intact | 5 | Hidden suite still matches the prompt | Assertions edited to match floats |
Reference implementation
The sample below is a grader key, not a framework tutorial. It keeps a process lock around check-then-act, hashes a canonical body, and stores only ints. Python treats bool as a subclass of int, so the amount gate must reject booleans before it accepts integers.
# ledger.py
from __future__ import annotations
from dataclasses import dataclass
from hashlib import sha256
from json import dumps
from threading import Lock
from typing import Any
class LedgerError(Exception):
def __init__(self, http: int, code: str, detail: str) -> None:
self.http = http
self.code = code
self.detail = detail
@dataclass(frozen=True)
class Credit:
tenant_id: str
idempotency_key: str
body_hash: str
amount_minor: int
class Ledger:
def __init__(self) -> None:
self._lock = Lock()
self._credits: dict[tuple[str, str], Credit] = {}
self._balances: dict[str, int] = {}
@staticmethod
def _body_hash(lines: list[dict[str, Any]]) -> str:
canonical = dumps(lines, sort_keys=True, separators=(",", ":"))
return sha256(canonical.encode("utf-8")).hexdigest()
@staticmethod
def _line_total(lines: Any) -> int:
if not isinstance(lines, list) or not lines:
raise LedgerError(400, "empty_lines", "lines must be a non-empty list")
total = 0
for line in lines:
if not isinstance(line, dict):
raise LedgerError(400, "invalid_line", "line must be an object")
amount = line.get("amount_minor")
currency = line.get("currency")
if isinstance(amount, bool) or type(amount) is not int:
raise LedgerError(400, "invalid_amount", "amount_minor must be an int")
if currency != "USD":
raise LedgerError(400, "unsupported_currency", "USD only in v1")
total += amount
return total
def apply_credit(self, tenant_id: str, idempotency_key: str, lines: Any) -> dict[str, Any]:
if not tenant_id or not isinstance(idempotency_key, str) or not idempotency_key:
raise LedgerError(400, "invalid_key", "tenant and idempotency_key are required")
total = self._line_total(lines)
body_hash = self._body_hash(lines)
key = (tenant_id, idempotency_key)
with self._lock:
existing = self._credits.get(key)
if existing is not None:
if existing.body_hash != body_hash:
raise LedgerError(409, "idempotency_conflict", "body mismatch")
return {
"replayed": True,
"currency": "USD",
"amount_minor": existing.amount_minor,
}
self._credits[key] = Credit(tenant_id, idempotency_key, body_hash, total)
self._balances[tenant_id] = self._balances.get(tenant_id, 0) + total
return {
"replayed": False,
"currency": "USD",
"amount_minor": total,
}
def balance(self, tenant_id: str) -> dict[str, Any]:
with self._lock:
return {
"currency": "USD",
"amount_minor": int(self._balances.get(tenant_id, 0)),
}
The HTTP shell can stay on the standard library so the packet remains runnable after a cold clone. Path parsing is intentionally strict. Unknown routes return 404 rather than a soft empty balance, which would hide tenant typos during grading.
# app.py
from __future__ import annotations
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from json import dumps, loads
from urllib.parse import urlparse
from ledger import Ledger, LedgerError
LEDGER = Ledger()
class Handler(BaseHTTPRequestHandler):
def log_message(self, format: str, *args) -> None:
return
def _write(self, status: int, payload: dict) -> None:
raw = dumps(payload).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(raw)))
self.end_headers()
self.wfile.write(raw)
def do_GET(self) -> None: # noqa: N802
parts = urlparse(self.path).path.strip("/").split("/")
if parts[:2] == ["v1", "tenants"] and len(parts) == 4 and parts[3] == "balance":
self._write(200, LEDGER.balance(parts[2]))
return
self._write(404, {"error": "not_found"})
def do_POST(self) -> None: # noqa: N802
parts = urlparse(self.path).path.strip("/").split("/")
if parts[:2] != ["v1", "tenants"] or len(parts) != 4 or parts[3] != "credits":
self._write(404, {"error": "not_found"})
return
length = int(self.headers.get("Content-Length", "0"))
try:
body = loads(self.rfile.read(length) or b"null")
result = LEDGER.apply_credit(
parts[2],
body.get("idempotency_key"),
body.get("lines"),
)
except LedgerError as exc:
self._write(exc.http, {"error": exc.code, "detail": exc.detail})
return
except Exception:
self._write(400, {"error": "invalid_json"})
return
self._write(200 if result["replayed"] else 201, result)
def make_server(host: str = "127.0.0.1", port: int = 8000) -> ThreadingHTTPServer:
return ThreadingHTTPServer((host, port), Handler)
if __name__ == "__main__":
make_server().serve_forever()
Run the public file first, then the hidden file, from a clean virtualenv. A submission that only greens the public file is incomplete work, not a successful ledger.
python -m venv .venv
. .venv/bin/activate
pip install pytest
pytest -q tests/test_public.py
pytest -q tests/test_hidden.py
Common failure modes
Graders see the same cluster of misses across human juniors and coding agents. Record the miss against the rubric row, then stop arguing about style.
-
IEEE-754 money. The agent stores
float(amount)or asserts0.1 + 0.2. Hidden float fixtures must 400, not round. -
Round each line, then sum. Per-line
quantizebefore addition drifts from the integer sum of minor units. -
Trueaccepted as one cent.isinstance(True, int)is true in CPython, so the gate needstype(amount) is int. - Global idempotency index. A single unique key across tenants fails the shared-key isolation row.
- Check then act without a lock. Sequential retries pass; eight parallel retries double-credit.
- Last body wins. Missing 409 lets a retried client mutate an already applied credit.
-
Spec rewrite. The agent deletes
test_float_line_is_rejectedor widens the public assertions. -
JSON numbers as strings.
"199"sneaks throughjson.loadsusers who callint()too late, or never.
Running the packet against an agent
Keep the hidden suite on the grader laptop or in a path the agent cannot edit. Give the agent the prompt, ledger interface comments if desired, and the public tests only. After it stops, copy the tree into a clean directory and run both suites. Diff tests/test_public.py against the original; any edit is an automatic deduction on the last rubric row.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. A reviewer who needs a throwaway environment can replay the same packet beside MonkeyCode's free model access and free server option, then score the resulting tree against the table above. The product is relevant here only as a place to run the agent and the tests; it does not replace the hidden suite or the human grader.
Limitations and who should skip this packet
This design does not prove production fitness for cards, wallets, or regulated money movement. It ignores durability, signed webhooks, ledger immutability, multi-currency FX, and crash recovery. In-memory maps vanish on process exit, which is acceptable for a take-home and unacceptable for customer funds. Teams that need ACID storage should not treat a green hidden suite as a license to ship.
Skip this packet when there is no human grader, when the candidate cannot run a local Python virtualenv, or when the role never touches invariants. Skip it for take-homes that already grade webhooks, pagination offsets, or cached authorization, because those packets already occupy that interview slot. Do not point the hidden suite at a shared production database. Do not claim model rankings from a single run of this file.
Hiring teams that already keep take-home repos can drop this ledger prompt into one free model session and record rubric rows instead of a demo transcript.
Top comments (0)