DEV Community: levl

Letting an LLM press buy: idempotency keys, fsync and process groups

levl — Wed, 14 Oct 2026 15:39:48 +0000

We recently shipped a way for AI agents (Claude Code, Codex, anything that takes a prompt and reads stdin, or speaks MCP) to act on the live trades of Polymarket wallets and place orders of their own. The data side streams fills to the agent; the trading side is a command the agent calls:

pmwallets-copytrade order buy <tokenId> --usdc 5 --max-price 0.45 --key <eventId>

The happy path needs little code. Most of what we wrote handles failures that could cost money; those are the cases I'll cover here.

The failures we planned for were ordinary ones:

The agent retries a command after it seemed slow.
The model misspells an argument.
The process crashes, or the machine loses power, halfway through something.
The agent CLI starts child processes that outlive it.

Each of those, unhandled, turns into either a second order or an order without the limit you meant to set.

1. A spend ceiling that's actually a ceiling

--usdc 5 reads as "spend $5". What we wanted was "never spend more than $5".

A BUY is fill-or-kill at a limit price, and the share count is computed at that limit, not at the current ask:

// usdc is a ceiling: sized at the limit, the order cannot cost more than it whatever it matches
shares = roundBuyShares((microToward(req.usdc, 'down') * UNIT) / limit, limit);

If the ask is 0.40 and the agent sets --max-price 0.45, it gets 5 / 0.45 shares. If the book moves up to 0.45 between the check and the match, the cost is still at most $5. A loose --max-price buys fewer shares, but keeps the order within the budget.

The rounding took more thought than expected. Prices and amounts are integers in millionths, and the caller's numbers come in as JavaScript floats. Rounding to nearest turns usdc 1.9999999 into 2.00 and minPrice 0.3800001 into 0.38, both of which loosen the caller's limit slightly. So every caller number is rounded in the direction that can't loosen it: a budget, a max price or a share count down, a min price up. It's done on the number's shortest decimal string, exactly, so no binary floating-point digit decides the direction:

export function microToward(x: number, dir: 'up' | 'down'): bigint {
  const m = /^(\d+)(?:\.(\d+))?(?:e([+-]?\d+))?$/.exec(String(x));
  if (!m) throw new Error(`cannot read ${x} as a decimal`);
  const digits = BigInt(`${m[1]}${m[2] ?? ''}`);
  const shift = Number(m[3] ?? 0) - (m[2]?.length ?? 0) + 6;
  if (shift >= 0) return digits * 10n ** BigInt(shift);
  const div = 10n ** BigInt(-shift);
  const q = digits / div;
  return dir === 'down' || digits % div === 0n ? q : q + 1n;
}

The same rule applies to the market's tick size: a BUY limit is rounded down onto the tick grid, a SELL floor up, and a floor above the highest price the market accepts is refused rather than lowered.

2. The idempotency key is written before the order leaves

A claude -p run that saw a slow response may run the same command again. --key prevents that retry from sending a second order; the fill's eventId is a suitable choice.

The key is claimed on disk and synced before the order is sent:

claim(key: string): boolean {
  mkdirDurable(this.dir);
  let fd: number;
  try {
    fd = openSync(this.file(key), 'wx');        // exclusive create: two callers can't both win
  } catch (e) {
    if ((e as NodeJS.ErrnoException).code === 'EEXIST') return false;
    throw e;
  }
  try {
    writeSync(fd, JSON.stringify({ status: 'sending', key, claimedAt: new Date().toISOString() }));
    fsyncSync(fd);
  } finally { closeSync(fd); }
  syncDir(this.dir);                             // the directory entry too
  return true;
}

These details each came from a specific failure case:

wx (O_CREAT | O_EXCL) means two concurrent calls with the same key can't both claim it.
fsync on the file isn't enough for a new file. Its directory entry also has to reach the disk, or a power cut can lose the file entirely. Hence syncDir, and mkdirDurable, which syncs each directory level it creates into its parent.
If the claim can't be synced, it throws and nothing is sent.
A claim with no recorded result (the process died while sending) comes back to the next caller as status: unknown, verify: true, not as "free". The order may have filled; the agent is told to check trades first.

To record the result, we write a temporary file, fsync it, rename it and sync the directory. A crash between the claim and the send means that key won't order again, even though nothing was sent. We accepted that missed-trade risk to avoid sending the same order twice.

3. Handing fills to the agent at most once

pmwallets watch --exec "<agent command>" holds a WebSocket to the fill stream and runs the agent once per batch of fills. The stream itself delivers each fill once, in order, and replays gaps. The question was what happens between the stream and the agent.

At-least-once is the usual choice for queues. Here, if the agent placed an order and then the process crashed, running the batch again could place it again. We hand each batch over at most once:

A fill is appended to queue.jsonl and synced before the stream cursor moves past it.
To run a batch, the batch is written to batch-<id>.jsonl, the queue is rewritten without it, and the batch's eventIds are appended to handed.txt and synced. Only then is the batch renamed to running-<id>.jsonl and the command started.
If the command exits 0, the file is deleted. If it fails or times out, it moves to failed/.
At startup, any running-* file means the previous process died while the agent ran. It moves to failed/ and is never run again. A batch-* file that was never started goes back to the front of the queue, unless its ids are already in handed.txt, in which case it also goes to failed/ for a human.

A fill whose eventId is in handed.txt is never queued again, even if the stream replays it because the cursor wasn't saved before a crash. handed.txt is trimmed occasionally, but only of ids at or before the stream cursor as it is on disk (synced first), since those are the only ones the stream can never replay.

A crash can leave fills in failed/ that nobody acted on. We log those for manual checking and accept that risk with this design.

4. Stopping the agent means stopping everything it started

An agent CLI is rarely one process. claude -p and codex exec start their own children, and a child left running after a timeout might still be about to call order buy.

The command runs in its own process group:

child = nodeSpawn(this.o.exec!, { shell: true, env, stdio: ['pipe', 'inherit', 'inherit'], detached: process.platform !== 'win32' });

A timeout or a stop sends SIGTERM to the whole group (process.kill(-pid, ...)), then SIGKILL five seconds later. On Windows it's taskkill /T /F. When the command exits normally, the group is killed anyway, because a grandchild may still be running.

Two things went wrong here before they were fixed:

The pending SIGKILL timers weren't cancelled when the command ended on its own. Five seconds later they could hit a different process that had been given the same pid.
A process can leave its group (setsid), and after a crash of the watcher itself, there's no parent left to remember the group.

For the second, every batch's command gets PMW_BATCH_FILE=<absolute path> in its environment. That doubles as a marker. On Linux, startup scans /proc/*/environ for the marker of any running-* batch and stops those groups before handing anything new out. On other systems, where that can't be checked, a running batch with no pid record makes startup refuse with instructions, instead of guessing.

5. Lock files that are never reused

We need one watcher per directory to avoid sending every fill to the agent twice. The classic pid lock file (create it, delete it on exit, take it over if the pid is dead) has a race. Starter A reads the lock, sees a dead pid, and pauses. Starter B also sees it dead, deletes it, creates its own and starts running. A wakes up, deletes B's live lock and creates its own, leaving two watchers running.

The fix was to stop reusing names. Locks are generations, watch.lock.0, watch.lock.1, and so on. The highest number is the lock. A new one is created with exclusive create, as highest + 1, only when the current highest is released or its owner is dead. Releasing writes { released: true } into the file instead of deleting it, and old generations are never removed. A starter that paused on an old scan tries to create a number that already exists and gets EEXIST. Each file is a few bytes, one per start.

6. Rejecting misspelt arguments

Over MCP, a model calls order_buy with a JSON object. If it sends max_price instead of maxPrice, a lenient server ignores the unknown field and the order goes out with the default limit, which is the current ask, whatever that is. MCP hosts don't have to validate against the schema, so the server does, strictly:

for (const [k, v] of Object.entries(args)) {
  const p = props[k];
  if (!p) throw new Error(`unknown argument ${JSON.stringify(k)} (allowed: ${Object.keys(props).join(', ') || 'none'})`);
  // ...type checks; null is rejected with "leave it out instead"
}

The error lists the allowed names so the model can fix the call. The command line also rejects unknown flags such as --max_price. A flag with no value, like --key --all, returns an error; otherwise it could place an order under the key --all.

7. Secrets are removed before anything is serialized

Exchange errors sometimes echo the request back, and the request can contain credentials. Anything written to a key record, a log line or stderr goes through redact() first. It walks the whole value, keys included, and replaces every known secret value plus anything shaped like an API key or a user:pass@ URL. It runs on the object before JSON.stringify, so field names are covered as well as values.

Finding failure cases in review

Many of the failure cases above came out of an adversarial review loop with Codex over the diff. We asked it specifically for crash windows, races and partial writes, and successive rounds found more cases to fix. The data CLI went from 0.6.0 to 0.6.5 on the day it shipped:

0.6.1: cancel pending SIGKILLs once the command is over; keep a batch recorded as handed but never started in failed/.
0.6.2: sync the stream cursor before forgetting handed ids; new-fills moves its cursor only once the answer is complete.
0.6.3: a new queue or handed file is created durably, directory synced, before a fill counts as queued.
0.6.4: cut a half-written last line off the queue and the handed record at start (otherwise the next append is glued onto the fragment and lost with it); one new-fills queue per file however its path is spelt.
0.6.5: on Linux, find a batch's processes by the environment marker even when there's a pid record, and stop any that left the group when the batch ends.

The trading CLI's 0.4.1 was the redaction fix in section 7.

None of those would show up in a normal test run. They need a crash at one specific line, or two processes starting within milliseconds of each other. What made the review useful was asking a narrow question: where can a crash between two lines cause an order to be sent twice, or a fill to be handed over twice?

Does it work

In a production test, 57 real fills reached the agent command between 0.01 and 0.16 seconds after they were pushed. A headless claude -p took about 11 seconds to read a fill, look up the market and place a dry-run order; codex exec took about 27. Most of the time went into the model run. That delay is another reason we start the trading side in dry-run mode, where orders are simulated against the live book and nothing is sent.

For a similar setup, I'd check the following:

Make the money limit a hard ceiling, computed at the limit price, and round caller numbers in the safe direction.
Claim idempotency keys durably before the side effect, and report a claim with no recorded result as "unknown", never as free to reuse.
Pick at-most-once when the consumer has side effects, and keep what fell through somewhere a human will look.
Stop the whole process group and leave a marker you can find after the watcher crashes.
Use a new lock file name for each generation.
Reject unknown arguments so a model's typo can't silently change an order.

The code is MIT-licensed: src/watch.ts and src/mcp.ts in polymarketwallets/pmwallets-node, and src/agent.ts in polymarketwallets/polymarket-copy-trading-bot on GitHub.

Nothing here is financial advice, and Polymarket's regional restrictions apply to placing orders.

I help maintain PMWallets.

Let your AI agent follow Polymarket traders: Claude Code, Codex, Cursor and MCP

levl — Wed, 14 Oct 2026 15:39:41 +0000

When a Polymarket trader I follow makes a trade, I want an AI agent to see it, look at the market, and decide whether to do anything about it.

Agents like Claude Code or Codex can read a fill and reason about a market, but they don't work well as long-running listeners. They exit when their task is done. In this setup, a small process holds the live connection and queues fills. It starts the agent once per batch with the fills on stdin; the agent handles the batch and exits.

Everything here uses two command-line tools that print JSON, and both also run as MCP servers:

pmwallets: the data side. Leaderboard, trader stats, subscriptions, live fills.
pmwallets-copytrade: the trading side. Market and order book lookups, balance, positions, and orders on Polymarket.

Install

npm i -g pmwallets pmwallets-copytrade     # Node.js 20+
# or
pip install pmwallets pmwallets-copytrade  # same commands

You need a PMWallets API key (created on pmwallets.com/keys):

export PMW_API_KEY=pmw_...

Pick traders to follow

pmwallets leaderboard --period 30d --minWinLo 0.55 --category politics --limit 20
pmwallets entity 7KQ2MF9X4B1C        # one trader's stats, by handle

minWinLo is the lower bound of the 95% confidence interval on a trader's win rate. A trader who won 3 bets out of 3 has a 100% win rate and a lower bound well under 0.5, because three trades say very little. Filtering on the lower bound pushes lucky streaks down the list.

Then follow one:

pmwallets subscribe 7KQ2MF9X4B1C     # billed per hour while subscribed
pmwallets subscriptions

Getting fills to the agent

There are two ways, depending on whether you can keep a process running.

Pull: `new-fills`

If your agent runs on a schedule (a cron job, a loop inside the agent, a chat agent you poke now and then), call:

pmwallets new-fills

It returns the fills since the previous call, each one once. The position is kept in ~/.pmwallets/mcp-cursor.json, or wherever PMW_MCP_STATE points. One reader per file: if two agents poll, give each its own PMW_MCP_STATE, or they will split the fills between them.

Push: `watch --exec`

If you can keep a process up, watch holds a live connection and runs your command for each batch of fills:

pmwallets watch --batch-sec 30 --exec 'claude -p "Fills from traders I follow are on stdin. For each BUY above $1000 in a market ending after 2026-12-31, read the market with pmwallets-copytrade market, then buy $5 with --key <eventId> if the price is below 0.6. One line per fill: what you did." --allowedTools "Bash(pmwallets-copytrade:*)"'

The fills arrive on the command's stdin as JSON lines, and the same lines are in a file named by $PMW_BATCH_FILE. Fills that come in while the agent is still running go into the next batch. --batch-sec 30 waits 30 seconds after the first fill so that a burst becomes one agent run instead of ten.

--allowedTools "Bash(pmwallets-copytrade:*)" lets headless Claude run the trading CLI. Without it, Claude either can't run anything or you end up granting far more than it needs.

For other agents, the command looks similar:

# Codex: its default sandbox has no network, so the trading CLI can't reach Polymarket
--exec 'codex exec --sandbox danger-full-access "...same instructions..."'
# Gemini CLI
--exec 'gemini -p "..." --yolo'

Without --exec, watch just prints one JSON line per fill, which is handy for piping into your own script.

A fill looks like this (trimmed):

{
  "eventId": "137:78012345:0x…:0x…:212",
  "side": "BUY",
  "role": "taker",
  "price": "0.570000",
  "tokenId": "1043…",
  "usdcAmount": "1824.000000",
  "sharesAmount": "3200.000000",
  "market": { "question": "…", "outcome": "Yes", "endDate": "2026-11-03T00:00:00Z" },
  "source": "ws",
  "receivedAt": "2026-10-14T09:12:03.417Z"
}

The raw usdc and shares fields are integers in millionths. The *Amount fields are the same values as decimals, so the agent doesn't have to divide by a million (models do get this wrong). The market question and outcome are attached, so the agent can tell what was bought without looking it up.

A batch is handed to the command at most once. If the agent crashes, times out, or the machine reboots mid-run, that batch goes to pmw-watch/failed/ and is never run again, because the agent may already have placed an order. Those batches need to be checked by hand. A restart picks up the stream where it stopped.

Placing orders

First write a config. It starts in dry-run:

pmwallets-copytrade init        # writes config.yaml, mode: dry-run
pmwallets-copytrade check       # checks keys and account, places nothing

In dry-run, every order is simulated against the live order book and nothing is sent. That is the mode to leave the agent in for a while.

The commands an agent needs:

pmwallets-copytrade market <tokenId>      # question, outcomes, prices, order book
pmwallets-copytrade balance
pmwallets-copytrade positions
pmwallets-copytrade order buy <tokenId> --usdc 5 --max-price 0.45 --key <eventId>
pmwallets-copytrade order sell <tokenId> --all --min-price 0.40 --key <eventId>:exit

The order commands follow these rules:

A BUY is fill-or-kill: all of it at --max-price or better, or nothing.
--usdc sets the maximum spend. The share count is usdc / max-price, so the order can't cost more than --usdc whatever it matches. It may spend less than that amount. Without --max-price the limit is the current best ask.
A SELL is fill-and-kill and is capped at what the account holds.
--key is an idempotency key. Pass the fill's eventId. The same key never sends a second order, so an agent that retries after a timeout doesn't double up.
status: unknown (or verify: true) means the order may have filled. The agent should check trades before sending another one. Exit codes: 0 filled or dry-run, 2 not traded, 3 unknown.

I'd explicitly require --key in the prompt, since agents may retry after a slow response. Without a key, that retry sends a second order.

When you switch the config to mode: live, give it a Polymarket account of its own with only what you're willing to lose in it. Polymarket doesn't accept orders from some countries (including the US and the UK); pmwallets-copytrade check tells you whether your machine's location can trade.

MCP instead of a shell

For Claude Desktop, Cursor, Windsurf, Cline and similar, the same tools are available as MCP servers:

{
  "mcpServers": {
    "pmwallets": {
      "command": "npx",
      "args": ["-y", "pmwallets", "mcp"],
      "env": { "PMW_API_KEY": "pmw_..." }
    },
    "copytrade": {
      "command": "npx",
      "args": ["-y", "pmwallets-copytrade", "mcp", "--config", "/abs/path/config.yaml"]
    }
  }
}

The new_fills tool shares its position with pmwallets new-fills, so you can mix the two, but the one-reader-per-file rule still applies.

The trading MCP server rejects arguments it doesn't know. If a model sends max_price instead of maxPrice, the call fails with an error that lists the allowed names, rather than quietly dropping the limit and buying at whatever the ask is.

No shell, no MCP

For Dify, Coze, n8n or a custom GPT, import the OpenAPI spec at https://pmwallets.com/openapi.json and call the REST API with an x-api-key header. Fills can also arrive as signed webhooks. Placing orders from there is up to you; the trading CLI needs a machine to run on.

How fast is it

Two measurements from testing:

A headless claude -p took about 11 seconds to read one fill, look up the market and place a dry-run order. codex exec took about 27 seconds for the same thing.
In a production test, 57 real fills reached the agent command between 0.01 and 0.16 seconds after PMWallets pushed them, each with its market question and outcome attached.

Most of the delay comes from the agent. If you care about getting the same price as the trader you follow, waiting 10 to 30 seconds for a model matters: in fast markets the price will have moved. For that case I'd use a fixed-rule bot (pmwallets-copytrade run). The agent setup is more useful when a task needs reading, such as skipping sports or only acting if the question mentions a specific event. It can also summarise what the people you follow did today.

Things to check before going live

Run in dry-run for a while and read what the agent did.
Put the key rule in the prompt: every order gets --key <eventId>.
Keep --usdc small and set --max-price yourself rather than letting the model pick one.
Look in pmw-watch/failed/ after any crash.
Check whether you're allowed to trade on Polymarket from where you are.

None of this is financial advice, and copying a trader is not the same as being that trader: you buy later, at a different price, and pay fees.

The guide to hand your agent is printed by npx pmwallets skill, and the same material is at pmwallets.com/sdk. Package versions at the time of writing: pmwallets 0.6.5 and pmwallets-copytrade 0.4.1.

I help maintain PMWallets.

Two Polymarket strategies that look fine until you add fees and a queue

levl — Tue, 13 Oct 2026 18:18:49 +0000

Filling at the mid and leaving out fees makes it pretty easy to get a profitable-looking backtest for a short-horizon Polymarket strategy. I wanted to see how much two simple ideas change once the backtest pays the taker fee, walks the book for depth, and makes resting orders wait in the queue.

Both tests ran on one day (2026-09-08) of Polymarket's BTC 5-minute Up/Down markets. The data has order book snapshots and deltas for the UP and DOWN tokens, trade prints, and the Chainlink stream the markets settle on. The book is rebuilt from snapshots and from deltas thinned to a capture cadence, so it approximates the depth at each moment rather than replaying the exchange exactly.

An 82% win rate that loses money

In the last 60 seconds of each market, the first strategy buys whichever side is priced higher, but only between 55¢ and 95¢:

from outcometick import Strategy, Order
from outcometick.backtest import run, load_sample


class Favourite(Strategy):
    def on_market_open(self, ctx, market):
        self.close, self.done = market.close_ts_ms, False

    def on_tick(self, ctx, tick):
        if self.done or self.close - ctx.now > 60_000:
            return None
        book = ctx.book()
        up, down = book.best("UP"), book.best("DOWN")
        if up is None or down is None:
            return None
        self.done = True
        side, price = ("UP", up) if up >= down else ("DOWN", down)
        if not 0.55 <= price <= 0.95:
            return None
        return Order(side=side, notional=50, limit=0.95)


result = run(Favourite, load_sample("polymarket"))
print(result.summary())

  market-days          1
  net pnl              -$15.62
  trades               108
  win rate             82.4%
  brier                0.1397
  edge/contract        0.7¢
  max drawdown         -$284.51
  fees                 -$52.90  (venue schedule, estimated per book level)
  lost to slippage     -$4.22

It won 82.4% of 108 trades, and the edge before fees was 0.7¢ per contract. Polymarket charges takers C × rate × p × (1 − p) with a rate set per market, and that came to $52.90 for the day, more than the strategy made before fees. When you buy at 80–95¢, a high win rate is mostly what you paid for. Here, the 82.4% win rate still left a net loss of $15.62.

Maker quotes that fill at the wrong moments

For the second test, I tried resting orders to avoid the taker fees. This strategy quotes 25 shares of UP at the best bid and 25 of DOWN at one minus the best ask. It requotes at most once a second and exits inventory with a passive order one tick above entry, with 150 ms order latency and 100 ms cancel latency.

To be filled, a resting order has to wait until trades have used up the size queued ahead of it at that price. Once it fills, a markout measures how far the mid has moved against you 1s, 10s and 60s later, and at settlement.

  trades               1105
  net pnl              -$26,133.42
rested 117061 orders, maker fill rate 0.0377 by size
median queue ahead at join 94.57, at fill 0.0
markout     1s: -0.0463 per share over 44594 fills
markout    10s: -0.0408 per share over 43760 fills
markout    60s: -0.0397 per share over 33437 fills
markout settle: -0.0462 per share over 44606 fills

Here a trade is one position from open to close, so the 1,105 trades were built from about 44,600 individual maker fills. The 1s markout averaged −4.6¢ a share and was still about −4.6¢ at settlement. A quote this simple tends to get filled when the other side knows something, for example when BTC has just moved on spot and the 5-minute market hasn't repriced yet. Those fills lost far more than the spread earned.

Only 3.8% of the quoted size filled, and the median fill had about 95 shares ahead of it when the order joined the queue. If the backtest had filled each quote as soon as the price touched it, this would have looked like a strategy that earns the spread on every fill.

A checklist for any Polymarket backtest

When I look at a Polymarket backtest, these are the things I check:

Fees per market. The taker fee depends on price and on each market's own rate. Assuming a flat fee, or no fee at all, overstates the returns of any strategy that crosses the spread.
Book depth. Fill an order level by level against the book as it stood at that moment, and report the size the book didn't have instead of filling it at the best price.
Latency. Match the order against the book as it was when the order would have arrived, not when the signal fired.
A queue for resting orders. A price touch is not a fill: trades still have to use up the size ahead of your order. Skipping that wait makes maker strategies look far better than they are.
Markouts. For maker orders, I also check how the mid moves after a fill. The 1s markout helps explain the PnL by showing whether the price moved against the order almost immediately.
Settlement from the official outcome. These markets settle on a Chainlink stream, and the market's own config says which one. Settling on a spot price you picked yourself can mislabel close markets.
Out-of-sample days. I wouldn't draw conclusions from this one day. Rank parameters on earlier days, then test the winners on later days that weren't used for ranking.

Reproducing this

The code for these tests is in quickstart.py and maker_quotes.py in polymarket-backtest. Install it with pip install outcometick; load_sample() downloads the same day used here. Each run took about two minutes on an 8-core machine.

The backtester is MIT-licensed, and there's also a JS version. The sample data is free under CC BY 4.0, requires no key or signup, and is also available on Kaggle and Hugging Face.

For the out-of-sample check above, use sweep(..., holdout=0.3).

If you find a maker variant with a non-negative 1s markout on a different day, I'd like to see it in the comments.

I help maintain this backtester and the outcometick dataset it reads.