This came up in Chinese AI developer communities recently, sparked by a new paper from Gu, Kagan, Sun, Wu, and Xu (arXiv:2607.06166v3) titled When do prophets profit in prediction markets? The question sounds simple. The answer will cost you if you assume it is.
Here is the failure mode that matters most in practice: you build a forecasting model, it outperforms the market on a proper scoring rule, you deploy it on Kalshi or a similar limit-order-book platform, and after a month you are net negative. Not because your predictions were wrong. Because you conflated predictive accuracy with extractable profit, and those two things are separated by at least four distinct friction layers. If you have shipped something like this and gotten burned, the paper gives you language for exactly where the edge disappeared.
The Gap Nobody Warns You About
Most introductory writing on prediction markets treats prices as probabilities and leaves it there. If the market says 40% and you believe 60%, you buy. Your model is better, so you profit. This is the intuition. It is not wrong exactly, but it is dangerously incomplete.
The older theoretical literature mostly models market makers using automated market makers (AMMs), where a single price-update rule governs the whole book. Real platforms increasingly use limit-order books. That distinction matters because in a limit-order book, the question is not just whether your probability is better than the last trade price. The question is whether there is a counterparty willing to fill your order at a price that still makes the trade positive in expectation after spread, fees, and price impact.
Gu et al. formalize this. Given your probability estimate p, the market's current price q, and a proper scoring rule S, they ask: can you construct a betting strategy that depends only on p and q, such that being more accurate than the market guarantees positive expected profit? Their answer is a conditional yes, and the conditions are the whole story.
What "Proper Betting" Actually Means
The paper constructs what it calls a proper betting strategy. The key property is that if your p is better than q under the scoring rule, and the market has sufficient liquidity at the prices you need, then your expected profit is positive. They also prove this strategy is essentially unique among those with this kind of robustness guarantee.
That word "essentially unique" is doing real work. It means there is not a family of equivalent strategies you can tune. There is one structure that has the theoretical guarantee, and deviations from it will either lose the guarantee or require additional assumptions you probably cannot verify in live trading.
The profit decomposes into two components. One piece comes from your scoring-rule advantage over the market, essentially how much better your probability is by the metric the scoring rule defines. The second piece is related to the raw distance between p and q. These are not the same thing. You can have a large gap between your estimate and the market price while having only a modest scoring advantage, and vice versa. Conflating them is another way practitioners lose money they thought they had coming.
Four Behavioral Archetypes and Why They Produce Different Returns
The empirical section of the paper compares thousands of AI model predictions against several betting heuristics. To explain the variation in outcomes, the authors introduce four behavioral categories: conservative, aggressive, diversified, and fragile.
The conservative bettor has a small average deviation from market prices. The aggressive bettor has large deviations. The diversified bettor spreads position across many events. The fragile bettor concentrates on a small number of high-conviction trades.
Here is the counterintuitive result: at the same overall scoring-rule accuracy level, these four archetypes produce meaningfully different profit distributions. A fragile bettor with identical aggregate accuracy to a diversified bettor can end up with worse realized returns simply because the structure of how their estimates deviate from market prices interacts badly with liquidity constraints and price impact.
Figure 1 in the paper (page 12) illustrates these archetypes synthetically. It is not a live trading returns chart. This distinction matters because it is easy to look at that figure and think you are seeing backtest results. You are not. You are seeing a schematic of the behavioral space, which is useful for diagnosis but not for calibrating return expectations.
The Kalshi Pilot: What Small-Scale Live Trading Actually Showed
The paper includes a one-month live trading pilot on Kalshi. The purpose was not to demonstrate profitability at scale. It was to test whether the theoretical framework survives contact with real-world friction: bid-ask spreads, platform fees, discrete order fills, and thin books.
The authors are explicit on the first page that this pilot should not be read as trading advice. That disclaimer is not boilerplate. It is a precise statement about scope. The pilot is evidence that the strategy is executable under realistic conditions, not that it is profitable at any given scale.
The practical friction points the pilot surfaces are worth listing:
- Bid-ask spread: On low-liquidity events, the spread alone can consume a significant fraction of your theoretical edge. If the scoring advantage gives you 3 cents per contract in expected value and the spread is 4 cents, the trade is negative before fees.
- Platform fees: Kalshi charges per contract. On small-probability events where you need large position sizes to move the needle on expected profit, fees scale with position, not with edge.
- Discrete fills: Limit orders do not always fill at the price you modeled. Partial fills are common, especially on events with thin books.
- Limited liquidity: The theoretical guarantee requires sufficient liquidity. The paper does not give you a formula for what "sufficient" means in dollar terms for a given event, because it depends on the event's book depth at the moment you trade.
The Claim That Should Change How You Instrument Your System
The core argument of the paper, translated into practitioner terms, is this: predictive accuracy and profitable trading are not the same metric, they are not even on the same scale, and conflating them is the primary failure mode in this space.
This is not a novel philosophical point. What makes the paper useful is that it gives you a decomposition. If you are logging your model's performance and your trading performance as the same number (say, Brier score or log-loss against realized outcomes), you are hiding the information you need to debug your system.
The authors suggest, and I think correctly, that the more valuable instrumentation separates: your scoring-rule performance against the market, the distribution of p-minus-q deviations across your trades, the per-trade cost of spread and fees, and the fill rate at your target prices. When you have those four numbers separately, you can see whether your edge is real and where it is leaking. When you collapse them into one accuracy metric, a month of losses looks like a model problem when it might actually be a liquidity problem.
What the Theory Does Not Cover
Positive expected profit is not the same as acceptable drawdown, manageable variance, or a strategy that scales beyond the position sizes the Kalshi pilot used. The theoretical guarantee is about expectation under the model's assumptions. It does not speak to:
- How often you will be down significantly before the expectation resolves
- Whether the scoring advantage persists as you take larger positions and move the market yourself
- How the guarantee degrades as market prices become more efficient in response to your trading
These are not criticisms of the paper. The paper is clear about its scope. They are reminders that the distance between a well-specified theoretical result and a deployed trading system is not closed by the theory alone.
The honest question the paper leaves me with: if you have a model that demonstrably outperforms market prices on a proper scoring rule, what is the minimum liquidity depth you need in the order book before the theoretical edge survives all four friction layers? The paper gives you the framework to ask that question precisely. I am not sure it gives you a number you can use before your first trade.
What has your experience been with the gap between model accuracy and actual P&L in prediction market deployments? Specifically: which of the four friction layers hits first in practice?
Top comments (0)