I backtested a forecast-based model against Kalshi's temperature markets. The market won. Here's the data.

I built a tool that takes the NWS forecast high for a city, wraps it in that city's measured forecast-error distribution, prices every Kalshi daily high-temperature bracket from it, and compares to the Kalshi price at the close of the day before. Then I backtested it on every settled US high-temp market from Jul 25 to Sep 22, 2026: 24 cities, 1,317 city-days, 7,902 markets.

The short version: the market beat the model. Here are the numbers, and the one thing in the data that was genuinely useful.

Setup

• Markets and ground truth: Kalshi's public API. Settlement value is Kalshi's own expiration_value, which is The Weather Company's reading at the station named in each market's rules (for NYC, "New York City (CLINYC)", the Central Park climate station). I cross-checked it against the NWS climate report: it matched on 1,316 of 1,317 city-days. The one miss was Miami on Aug 29, 90 vs 85.
• Forecast: Open-Meteo's previous-runs archive, the hourly temperature_2m_previous_day1 series, max over the local day. I tried GFS, ECMWF, and their average; the blend had the lowest pooled sigma so it won.
• Market price: the close of Kalshi's daily candle for the last full day before the target date. Mid for scoring, ask for the P&L.
• Model: actual high ~ Normal(forecast + city bias, city sigma), bracket probability is the mass inside the bracket with a half-degree continuity correction.

Forecast error by city (actual minus day-1 forecast, F)

city	n days	bias	sigma	MAE	skew	ex. kurt	within 1sd	within 2sd	normal ok?
NYC	60	-1.074	2.078	1.861	-0.35	0.14	0.7	0.983	yes
MIA	60	3.657	1.916	3.667	0.11	-1.18	0.583	0.983	yes
CHI	60	0.26	2.407	1.837	-0.74	1.57	0.733	0.95	yes
AUS	60	2.458	1.455	2.622	-1.29	4.69	0.75	0.95	no
HOU	60	-0.025	2.267	1.75	0.64	1.44	0.717	0.967	yes
DC	60	-0.593	2.546	2.113	-0.02	-0.49	0.7	0.95	yes
SAN	35	-3.354	2.543	3.669	0.51	-0.18	0.657	0.971	yes
NOLA	60	1.159	2.201	1.926	-0.64	2.02	0.717	0.967	no
MIN	60	-0.116	2.976	2.366	-0.38	-0.09	0.617	0.967	yes
SATX	60	0.526	1.592	1.204	-0.39	2.06	0.8	0.933	no
DEN	60	0.624	1.946	1.641	-0.52	0.32	0.733	0.917	yes
LAX	60	-3.961	3.064	4.346	0.19	-0.15	0.717	0.917	yes
BOS	60	0.309	2.446	1.901	-0.71	0.79	0.683	0.983	yes
ATL	60	0.326	2.128	1.647	0.4	-0.5	0.7	0.967	yes
PHL	60	1.901	1.884	2.224	-0.08	-0.56	0.6	0.95	yes
OKC	60	1.263	2.603	2.26	-0.39	2.47	0.85	0.933	no
PHX	60	0.972	1.832	1.608	0.33	0.46	0.717	0.933	yes
LAS	60	0.616	2.232	1.587	-1.4	6.02	0.85	0.933	no
TTN	28	0.546	1.651	1.425	-0.72	-0.18	0.679	0.964	yes
EWR	28	0.873	2.236	1.87	0.17	-0.53	0.643	0.929	yes
SFO	60	-1.095	3.434	2.527	-1.42	2.8	0.767	0.967	no
SEA	60	0.809	2.151	1.912	-0.76	0.76	0.8	0.95	yes
DFW	60	0.45	1.926	1.527	-1.49	4.11	0.733	0.967	no
SDF	26	-0.285	2.465	1.573	-0.76	3.81	0.846	0.923	no
pooled	1317	0.321	2.755	2.136	-0.72	1.57	0.733	0.948	yes

Pooled: bias +0.3F, sigma 2.76F, MAE 2.1F. The station biases are the interesting part. Miami runs 3.7F warmer than forecast; LAX runs 4.0F colder. Those are large, stable, and exactly the kind of thing a normal-distribution model should be able to exploit if the market weren't already pricing them.

Out of sample: fit on the first 30 days, test on the last 30

	Brier (lower is better)
Model (quoted markets)	0.1466
Market mid (quoted markets)	0.1195
Base rate (constant)	0.1541
Model (all markets incl. unquoted)	0.1285

The market wins by 0.027 Brier. It won in every city and in every price band.

Calibration in 10% bins:

bin	model n	model mean p	observed	market n	market mean p	observed
0.0-0.1	1240	0.034	0.079	1407	0.042	0.026
0.1-0.2	614	0.147	0.179	498	0.143	0.131
0.2-0.3	681	0.249	0.266	411	0.244	0.238
0.3-0.4	491	0.345	0.263	347	0.348	0.34
0.4-0.5	142	0.444	0.423	384	0.447	0.427
0.5-0.6	53	0.535	0.358	162	0.545	0.556
0.6-0.7	24	0.641	0.25	63	0.634	0.683
0.7-0.8	18	0.734	0.556	10	0.736	0.7
0.8-0.9	15	0.84	0.533	3	0.847	1.0
0.9-1.0	7	0.945	0.571	0

Where the model said 50 to 70 percent and disagreed with the market, the event happened 25 to 36 percent of the time. The model's lowest bin (0 to 10 percent) resolved yes 7.9 percent of the time against the market's 2.6 percent: fat tails that a normal distribution under-prices.

What happens if you trade it

strategy	trades	wins	staked $	fees $	P&L $	ROI
Buy YES (as specified)	689	74	6850.81	420.8	-2583.61	-0.3771
Buy NO (mirror: model NO prob >= NO ask + 0.08)	787	454	7630.54	215.34	-550.88	-0.0722

Every price band loses. The worst are the sub-10-cent tails, which is where an 8-cent "edge" shows up most often and where the normal assumption is most wrong.

What I take from this

1. A public forecast plus a normal error model does not beat the day-before Kalshi price. The people trading these markets are already doing at least that, with fresher model runs than a previous-day archive.
2. The per-station bias table is real information. If you trade weather markets and are not adjusting for the station, that is the edge you are leaving, and it is the one thing the tool computes fresh each day.
3. Honest caveats: 60 summer days only; the backtest forecast is archived GFS/ECMWF while the live tool uses NWS; the market close has slightly fresher information than the archived run, so the benchmark is a little favored.

The tool

Free, MIT, standard-library Python, no API keys: https://github.com/myfirstcodeo/kalshi-weather-fair-value

python weather_edge.py scans all 24 open series against the live NWS forecast and prints model probability, market price, edge, and a quarter-Kelly size per bracket. python backtest.py reproduces every number above from public data. python test_smoke.py checks the pricing math offline.

The cross-venue tool that does pay for itself is the Kalshi vs Polymarket divergence scanner, a one-time $29 purchase: https://instaverb.gumroad.com/l/pm-scanner

Not financial advice. Read the market rules before you trade anything.