We pre-registered a Polymarket insider score. It passed, and we still didn't ship it

Polymarket insider trading makes headlines every few weeks, so an insider score sounds like an obvious feature for a Polymarket analytics site. We built one, called it the Early score, wrote the rules down before running it, and tested it on 355,732 wallets. It passed its own test, and we did not release it. The full results are on our blog and the aggregated data and a script are on GitHub.

This post is about the method: how to set up a wallet score so the test can actually fail, the traps we hit before the test, and the check that changed our mind.

1. Write the rules down first

Before computing a single score, we put the whole method in a pre-registration file: what an entry is, how the move is measured, the shrinkage constant, which markets are out, which periods are compared, and what counts as a pass. Every later change went in as a dated line with the reason.

The pass rule was simple: score wallets on October 2025 to March 2026 (period A), and the top 10% must still be positive on April to September 2026 (period B), also without the 5 largest events.

Without this file it is very easy to keep adjusting until something works. With it, every change you make has to be justified in writing before you see the outcome.

2. The score

If a wallet knows something first, the price should move its way after it trades. So for each entry we measured the price of the side the wallet holds 24 hours later, minus the price it paid, in cents. A wallet's score is the size-weighted average, pulled toward zero for wallets with few entries:

def shrunk_mean(moves, weights, k=5.0):
    """Size-weighted mean move, shrunk toward 0.
    weights are entry sizes capped at $10,000, so n_eff counts 'effective entries'."""
    total_w = sum(weights)
    if total_w == 0:
        return 0.0
    raw = sum(m * w for m, w in zip(moves, weights)) / total_w
    n_eff = total_w ** 2 / sum(w * w for w in weights)
    return raw * n_eff / (n_eff + k)   # 5 effective entries keep half the raw mean

Wallets within 1 cent of zero (one price tick) get a neutral 50; the rest become a percentile. Sports and short Up or Down markets are out: a day after a match the price is the result, so the score would just rank sports bettors.

3. Trap: wallets trading with each other at made-up prices

On one test day the top of our list was a group of related wallets passing the same shares back and forth in finished tennis matches. One bought at 0.02 and sold to a partner at 0.96 four minutes later. Read trade by trade, that is a +94 cent entry.

The fix was to define an entry as a wallet's net position change in one market over one UTC day, not as a single trade:

from collections import defaultdict

def daily_entries(trades):
    """Simplified. trades: dicts with wallet, market, day, side ('YES'/'NO'), shares (+buy/-sell), price.
    Selling NO counts as buying YES. A same-day round trip nets to zero."""
    net = defaultdict(lambda: {"yes": 0.0, "no": 0.0, "cost_yes": 0.0, "cost_no": 0.0})
    for t in trades:
        k = (t["wallet"], t["market"], t["day"])
        s = "yes" if t["side"] == "YES" else "no"
        net[k][s] += t["shares"]
        net[k]["cost_" + s] += t["shares"] * t["price"]
    for k, v in net.items():
        delta = v["yes"] - v["no"]
        if delta == 0:
            continue
        side = "yes" if abs(v["yes"]) >= abs(v["no"]) else "no"
        avg_price = v["cost_" + side] / v[side] if v[side] else None
        yield k, delta, avg_price

If you rank Polymarket wallets by anything trade-based, expect these pairs near the top of your list.

4. Trap: a thin "price after"

In one market a handful of trades of $1.50 to $15 at 0.15 set the price 24 hours later, though the market settled at 1. One tiny trade can fake a move. We now require at least $100 of other wallets' trades in that hour before using a price, and fall back to the final price only if the market has closed.

5. The test passed

Period A groups, measured on period B, with t-statistics clustered by event (many entries in one event are not independent):

Group from period A	24 h move in B, cents per entry	t
Top 10%	+0.83	7.9
Bottom 10%	-0.76	-4.7
All wallets	-0.01	-0.2

It held without the 5 largest events, without weather markets, and for every shrinkage constant from 0 to 100. By the rule we wrote, a pass.

6. The check that changed our mind: look at the mechanism

A pass tells you the score finds something that repeats. It does not tell you what. An insider and a wallet that simply buys cheaper both show a positive move, but with a different shape:

• news spreading: the price keeps moving for hours;
• a cheap fill: the gap is there at once and stops growing.

So we split each wallet's move into the first hour and the drift from 1 to 24 hours, and asked which part repeats between the two periods:

from statistics import correlation  # Python 3.10+

def ranks(xs):
    order = sorted(range(len(xs)), key=xs.__getitem__)
    r = [0] * len(xs)
    for i, j in enumerate(order):
        r[j] = i
    return r

def spearman(a, b):
    return correlation(ranks(a), ranks(b))

# per wallet, period A vs period B
rho_first_hour = spearman(first_hour_A, first_hour_B)   # 0.138
rho_drift      = spearman(drift_A, drift_B)             # 0.050

The top 10% were already +0.75 cents ahead after one hour and +0.83 after 24. The first-hour part repeats; the drift, the part an insider score exists to catch, barely does. The score was ranking wallets by fill quality.

This check was not in the pre-registration, and we label it exploratory everywhere. But it is the one that decided whether a label like "insider" would tell users the truth. It would not, so the score is not on the site, in the API or in MCP.

7. Money, not moves

A 24-hour move is not profit, so we also held every entry in a resolved market to settlement and subtracted the fee each wallet actually paid on-chain. The top 10% made +1.17 cents per entry after fees (t 2.9), on both mostly taker and mostly maker entries. One detail that matters if you repeat this: compare groups with all wallets, not with zero. On longer-dated markets every group drifts up, so zero is the wrong baseline.

Reproduce it

The repo has the aggregated tables behind every number and scripts/early_score_check.py (standard library only), which rebuilds entries, scores and the period test from the OrcaLayer API:

export ORCALAYER_API_KEY=sk_orca_...        # never commit it
python3 scripts/early_score_check.py --csv entries.csv        # first 1,000 markets of data/markets.csv
python3 scripts/early_score_check.py --load-csv entries.csv   # rerun the analysis, no API calls

The API's trade history starts on May 1, 2026, so the script scores on May and June and checks on July to September, on a fixed list of 1,000 markets. The script writes wallets only as salted hashes, and the repo publishes groups only, no addresses.

What this does not show

Sports, individual wallets, and copying these wallets (a copy trades after them and does not get their price). On the shorter API check, the groups did not separate when held to settlement, so treat the money part as the weakest result.

---

Originally published on orcalayer.com. OrcaLayer is an independent Polymarket analytics provider, not affiliated with Polymarket.