Election Forecasting Benchmarks: How LFORLA Scores Models Before the Votes

TL;DR: LFORLA's election forecasting benchmark asks models to predict the winners of the 2027 French presidential election and the 2026 US midterms. Because correctness is only verifiable after the vote, a fixed judge scores each response on specificity, grounding, and calibration. The current leaderboard shows Nemotron 3 Ultra (free) at 89.1, GLM 5.2 at 78, and HY3 (free) at 76.5.

Election forecasting is hard for humans and models alike. You cannot check a prediction until the ballots are counted. So how do you evaluate a model's forecasting ability today? LFORLA's approach is to elicit concrete, verifiable scenarios and score their quality, not their accuracy. This article explains the mechanism behind the numbers.

How the evaluation works

The benchmark, called Election Predictions (FR 2027 · US 2026), is a forecast elicitation task. Models are prompted to predict the winners of two upcoming elections: the 2027 French presidential election and the 2026 US midterms. The prompt requires models to commit to concrete scenarios, not vague hedges. For example, a model might state which candidate wins the French presidency and which party controls each chamber of the US Congress.

Responses are public verbatim. That transparency is key: anyone can inspect what each model actually said.

Since the elections have not happened, correctness cannot be scored. Instead, a fixed judge evaluates each response on three axes:

• Specificity: Does the model name candidates, parties, and outcomes clearly? Vague statements like "the incumbent party faces headwinds" score poorly.
• Grounding: Does the prediction reference relevant facts, such as polling, historical trends, or institutional rules? Ungrounded guesses are penalized.
• Calibration: Does the model express appropriate confidence? Overconfident long shots and underconfident favorites both lose points.

The judge is fixed, meaning the same scoring rubric and model are used for every submission. This ensures consistency across models and over time. The judge does not know the election results; it only assesses the quality of the forecast as a forecast.

The real numbers

Here is the current leaderboard:

Model	Score	Provider
Nemotron 3 Ultra (free)	89.1	opencode-zen
GLM 5.2	78	opencode-zen
HY3 (free)	76.5	opencode-zen

We also submitted our own model, GLM 5.2, to this benchmark. Its overall score is 78.0, matching the leaderboard entry.

Why the ranking looks this way

Nemotron 3 Ultra leads by a wide margin. In practice, a score of 89.1 suggests its responses were highly specific, well-grounded in available evidence, and appropriately calibrated. It likely named candidates and outcomes clearly and supported them with references to polling or historical patterns.

GLM 5.2 and HY3 are close, at 78 and 76.5. The gap between them is small, meaning both produce forecasts of similar quality under the judge's rubric. They may be specific but less grounded, or well-grounded but less calibrated. Without inspecting individual responses, we cannot say which axis dragged the score down, but the closeness indicates neither model dominates the other.

The scores are not probabilities of being correct. They measure forecast quality as judged by the rubric. A model can score well and still be wrong; a model can score poorly and still be right. The benchmark rewards clear, evidence-based reasoning, not lucky guesses.

What this means for practitioners choosing a model

If you need election forecasting for research, journalism, or scenario planning, this benchmark offers a signal. A higher score means the model is more likely to produce forecasts that are specific, grounded, and calibrated, which are desirable traits for any forecasting task. However, the benchmark covers only two elections, so do not overgeneralize. Use it as one input among many.

How to read a leaderboard

1. Check what is being scored. Here, it is forecast quality, not accuracy. Accuracy is impossible before the vote.
2. Look at the axes. Specificity, grounding, and calibration tell you what the judge values. A model that excels on one axis may lag on another.
3. Compare scores, not ranks. A 1.5-point gap between second and third is small; a 11-point gap is meaningful.
4. Inspect the public responses. Verbatim responses let you see why a model scored as it did.
5. Consider the judge. A fixed judge ensures consistency but may have its own biases. No judge is perfect.

Honest limitations

This benchmark has clear limits. First, it covers only two elections, so results may not generalize to other forecasting tasks. Second, the judge is a model, and model judges can have biases or blind spots. Third, the scoring axes are proxies for forecast quality; they do not guarantee accuracy. Finally, the leaderboard is a snapshot; models and prompts change over time.

Conclusion

Election forecasting benchmarks like LFORLA's provide a structured way to evaluate models before the votes are cast. By scoring specificity, grounding, and calibration, they reward clear, evidence-based reasoning. Nemotron 3 Ultra currently leads, but the real test comes after the elections. Explore the full leaderboard and methodology at https://lforla.org.