Every cross-venue "arb scanner" for prediction markets does three things: fetch both venues, decide which contracts are the same question, subtract. The first and third are trivial. The second one ate my afternoon.
The data
Both venues expose keyless market data.
- Kalshi:
GET /trade-api/v2/events?status=open&with_nested_markets=true, cursor-paged, 200 events per page. Two gotchas. Nested markets reportstatus: "active"rather than"open", and they reportliquidity_dollars: "0.0000"across the board, so you gate onvolume_fpand a two-sided book instead. Also skip everything with an event ticker startingKXMVE: those are multivariate parlay shards, and there are tens of thousands of them. - Polymarket:
GET https://gamma-api.polymarket.com/markets?active=true&closed=false&order=volume24hr&ascending=false, 100 per page. Offsets past a couple thousand return HTTP 422, so sort by volume and accept that you get the liquid top of the book. That is where the arbs are anyway.
After filtering: about 28,000 Kalshi contracts with real volume, about 1,600 Polymarket contracts with real liquidity.
Naive matching fails in three specific ways
I started with token Jaccard plus a difflib sequence ratio. The top "opportunities" were:
- "Highest temperature in Miami: 92° to 93°" vs "between 90-91°F". Same city, same day, adjacent bracket. A 70-cent "gap."
- "Closest Senate race in 2026: Maine" vs "Will the Democrats win the Maine Senate race?" Shares four tokens, different question entirely.
- "When will Xi Jinping visit Taiwan? Before 2027" vs "Will Xi Jinping visit US before 2027?" The naive tokenizer dropped "us" as a two-letter word, leaving the two questions nearly identical.
Each one is a class, not a one-off.
Three vetoes fixed most of it
Before any scoring, reject the pair if:
- Numbers differ. Extract every number from both raw strings. If either side has any, the sets must be equal. Kills the bracket problem and most date confusion.
- A pivot word is on one side only. A short list: closest, margin, first, neither, before, after, not, win, lose, Democrats, Republicans, above, below, and so on. If the symmetric difference of the token sets hits the list, it is a different question.
-
Places conflict. A list of countries, states, and cities. If both sides mention a place and they differ, reject. This needed a "keep short tokens" exception for
us,uk,eu,xi,10y.
Then require containment (shared / smaller set) of at least 0.6, and score 0.4 Jaccard + 0.3 containment + 0.3 sequence ratio. One-to-one assignment, greedy by score.
What survived
On a mid-September afternoon: 21 pairs above a 3-cent gap and $500 of liquidity. The clean ones were things like Senate "Clarity Act" vote markets per senator (3 to 5 cent gaps), a Trump-speech word market (75c vs 71c), a Warsh press-conference word market. And one liquid pair, whether the US reopens its embassy in Iran before 2027, priced 3c on Kalshi and 16c on Polymarket with $223k behind it.
Also one 80-cent gap on "who leads Venezuela at end of 2026: Maduro". Both books tight, both liquid. That is not free money; that is two venues resolving on different rules, and the scanner's most useful output was flagging it as "VERIFY RULES" instead of ranking it first.
Performance note
The first version segfaulted. Candidate generation was a token inverted index, and tokens like "win" fanned out to tens of thousands of Kalshi markets. Capping index tokens at 5% document frequency fixed it and the whole match runs in seconds.
The lite version of the scanner is free on GitHub: https://github.com/myfirstcodeo/pm-divergence-scanner-lite. The full one with watch mode and Telegram alerts is a one-time purchase at https://instaverb.gumroad.com/l/pm-scanner. Not financial advice. Read both rule sets.
Top comments (1)
The most useful output being a "VERIFY RULES" flag matches what I saw last week. I pulled the October Fed markets off Gamma and resolutionSource came back empty on all five, while the description named the FOMC statement. A scanner reading only the convenience field would've called them unsourced. My matcher is a different shape, news articles against a macro release calendar rather than venue against venue, but your bracket problem showed up the same way: a market wrap that mentions CPI kept outranking the note about the print itself, until titles got their own tier. On the low-importance releases it sits at precision 0.35 with recall 1.0 and I've left it there, a spare match is cheaper than a missed one in that use. Did the number veto ever bite you on dates written out as words?