The Problem
Every election cycle, politicians make hundreds of promises. Some are specific: "Increase pensions by 50 euros by Q2 2027." Others are vague: "Strengthen our healthcare system." Most voters never know which candidates are actually committing to measurable goals versus just saying what sounds good.
This matters because specificity correlates with implementation capability. A candidate who names exact figures, timelines, and legislation can be held accountable. A candidate who speaks only in aspirations often cannot.
The question: Can we systematically measure this gap?
Introducing the Program Specificity Index
We built a framework that categorizes every policy claim into three tiers:
Concrete Plans (🟢): Specific numerical targets, exact timelines, named legislative acts, or measurable milestones. Example: "Cut corporate tax rate to 15% by 2027."
General Direction (🟡): Policy intent without implementation detail. Commitment exists but lacks numbers, timelines, or methods. Example: "Support small business growth."
Empty Slogans (đź”´): Uncosted aspirations or emotional appeals too vague to be actionable. Example: "Build a brighter future."
Every claim in a political program gets assigned to one tier. The result: a "specificity score" (percentage of concrete claims) for each candidate or party.
What the Data Showed
We analyzed over 1,000 policy claims across a full slate of political programs and found:
- Specificity varies wildly: Top performers scored 56% concrete claims; bottom performers scored 7%. This gap is meaningful.
- Some sectors are always vague: Healthcare and education promises were systematically less specific than tax or defense promises, regardless of who made them.
- Family benefits dominate concrete language: Pension and child-benefit pledges consistently included exact euro amounts. When politicians want to show specificity, this is where they do it.
- Party age matters: Newer, single-leader parties averaged much higher specificity scores than long-established coalition parties, which relied heavily on abstract phrasing.
- Universal boilerplate exists: Across all parties, identical defense spending commitments ("5% of GDP") appeared verbatim. Same language, different lips.
How We Built This: The Technology Stack
The analysis pipeline relied on three layers of AI:
1. Text parsing and claim extraction: We fed entire political programs to Claude Sonnet (Anthropic's mid-tier model, optimized for reasoning) to parse natural language and extract distinct policy statements. The model filtered out meta-commentary, introductions, and repeated phrasing to identify unique claims worth categorizing.
2. Multi-tier classification: Each claim was classified by Sonnet using the three-tier framework. The model applied consistent logic: Does this claim contain at least one concrete element (a number, date, percentage, or specific action)? If yes: 🟢. If it names intent but lacks implementation detail: 🟡. If it is pure aspiration: 🔴.
3. Sector tagging and aggregation: Every claim was also tagged by policy domain (Economy, Healthcare, Education, Defense, Social Protection, etc.). This allowed us to measure specificity not just by party, but by sector within each party. We aggregated counts, percentages, and patterns across all 1,000+ claims to generate summary statistics.
The entire workflow ran in a single pipeline: parse → categorize → tag → aggregate → summarize. Sonnet's reasoning capability was essential because political language is slippery. The model had to distinguish between "we will raise pensions" (general direction) and "we will raise pensions by 50 euros" (concrete plan). That nuance requires understanding, not just keyword matching.
Why This Matters
This framework demonstrates that AI can be applied to civic accountability. Instead of relying on journalist instinct or voter intuition, we used systematic analysis to measure a concrete property of political communication: specificity.
The method is replicable. Any country running an election can apply this same framework to their own candidates or parties. The three-tier system is universal. The AI pipeline is straightforward. The results are transparent (every claim can be reviewed).
More broadly, this shows how LLMs can be used for data journalism and civic tech. The model does not make judgments about right or wrong policy. It measures an objective property: Is this claim specific enough to implement and verify?
Apply This to Your Elections
If you are running an election in your country, you can replicate this analysis. The framework needs:
- Full text of candidate or party programs (in any language Claude understands).
- Clear definitions of what "concrete," "general," and "vague" mean in your political context (adapt the examples).
- A few minutes of AI API time to run the classification.
The output: A ranked list of candidates or parties by specificity score, plus sector-level breakdowns showing which policy areas get concrete treatment and which are left vague.
This is useful for voters. It is also useful for researchers studying how political communication changes over time or across regions.
What would you measure if you built this for your own elections?
Top comments (0)