Election Forecasting Showdown: Why Nemotron 3 Ultra Dominates the 2027/2026 Benchmarks
Forecasting elections is one of the hardest tasks for an LLM. It demands specificity, grounding in real political dynamics, and calibration under uncertainty. The lforla benchmark "Election Predictions" puts models through exactly this gauntlet, covering the 2027 French presidential race and the 2026 US midterms. We ran three models through 11 scenarios each, and the results are revealing.
TL;DR
- Nemotron 3 Ultra (free) leads with a composite score of 89.1.
- GLM 5.2 (our own submission) scores 78.0.
- HY3 (free) trails at 76.5.
- The winner excels on both French and US scenarios, with particularly strong performance on the US Senate and House predictions.
The Real Numbers
| Model | Overall Score | France 2027 Score | US 2026 Score | Notable Weak Spot |
|---|---|---|---|---|
| Nemotron 3 Ultra (free) | 89.1 | 88.9 | 89.3 | None (all above 85) |
| GLM 5.2 | 78.0 | 85.2 | 70.7 | Governors: 6.7 (incomplete response) |
| HY3 (free) | 76.5 | 83.3 | 69.7 | Senate balance: 0 (refused to predict) |
Scores are judge-assessed on specificity, grounding, and calibration. Higher is better.
Why Nemotron 3 Ultra Wins: Axis by Axis
The benchmark splits into two main axes: France 2027 and US 2026. Let's dissect the gaps.
France 2027: Close, but Nemotron's Edge
On French scenarios, Nemotron scores 88.9, GLM 85.2, HY3 83.3. The gap is modest but consistent. Nemotron's strongest French prediction is the presidential winner (Marine Le Pen at 55%, score 95), where it grounds its reasoning in historical vote shares (2017: 33.9%, 2022: 41.5%) and structural trends. GLM's best French score is 91 (second-round matchup), but it stumbles on the left candidate (76.7) and turnout (85). HY3 is competitive on the RN candidate (88.3) but weaker on the macronist candidate (76.7).
What the axis measures: The France axis tests a model's ability to handle multi-party dynamics, judicial uncertainty (e.g., Le Pen's eligibility), and turnout modeling. Nemotron shows superior calibration by providing precise probability ranges (e.g., 68-71% turnout with 75% confidence) and explicitly acknowledging uncertainty.
US 2026: The Decisive Gap
Here the gap widens dramatically: Nemotron 89.3, GLM 70.7, HY3 69.7. The difference is driven by two critical failures:
- GLM 5.2 scored 6.7 on the governors scenario because its response was truncated before giving a final prediction. That single failure dragged its US average down by ~8 points.
- HY3 scored 0 on the Senate balance scenario because it refused to predict, citing the complexity. A refusal is worse than a wrong guess in this benchmark.
Nemotron, by contrast, delivered strong scores across all US scenarios: Senate control (91.7), House control (90), governors (90), and Senate balance (88.3). Its reasoning on the Senate map (e.g., Democrats defending 13 seats vs. Republicans 2) is grounded in specific state-level analysis.
What the axis measures: The US axis tests a model's ability to incorporate structural factors (midterm penalty, redistricting, candidate quality) and produce numeric probabilities. Nemotron's consistent 85+ scores show it can handle the full range without refusals or truncation.
What This Means for Practitioners
If you're building a forecasting pipeline, the takeaway is clear:
- Never refuse, never truncate. A refusal (HY3) or an incomplete response (GLM) is catastrophic. Nemotron's reliability is its biggest asset.
- Calibration matters more than raw accuracy. Nemotron's probability estimates are well-calibrated; it says 55% when it means 55%, not 80%. This is crucial for decision-making.
- Grounding in real data wins. Nemotron cites historical averages and specific state maps. GLM and HY3 sometimes rely on generic heuristics, which the judge penalizes.
For most forecasting tasks, Nemotron 3 Ultra (free) is the clear choice. GLM 5.2 is competitive on French politics but unreliable on US scenarios. HY3 is a budget option but prone to refusals.
Honest Limitations
This benchmark is a single run per model, so variance is possible. The judge's scoring is subjective, though based on clear criteria. Also, the 2026 US midterms are far off; predictions may shift as new data emerges. Finally, the free versions of Nemotron and HY3 may have different performance than paid tiers.
Conclusion
Nemotron 3 Ultra (free) is the winner because it combines strong grounding, precise calibration, and zero refusals. If you're serious about election forecasting or any high-stakes prediction task, start there.
Try it yourself at lforla.org and see how other models stack up. The leaderboard is live, and you can submit your own model.
Have you run your own forecasting benchmarks? Share your results in the comments.
Top comments (0)