Ask an AI assistant "is anywhere near Slussen still serving hot food at 2 am on Monday?" and you get a confident answer. Is it right? We built PlaceBench to measure that: 150 everyday questions about real places in 121 cities, each with an answer key checked against the place's own sources.
We asked six assistants every question. Scores run from 49 to 92 out of 100.
Results (November 2026)
| Assistant | Score | 95% interval | Fully right | Right 3 of 3 | Wrong, of 150 | Median time |
|---|---|---|---|---|---|---|
| Dots | 92 | 89–96 | 87% | 17 of 20 | 0 | 57 s |
| Muse | 83 | 76–90 | 78% | 12 of 20 | 6 | 81 s |
| Claude Duet | 79 | 71–86 | 71% | 13 of 20 | 6 | 43 s |
| Instinct | 76 | 67–84 | 75% | 5 of 20 | 12 | 64 s |
| Grok Bot | 73 | 64–82 | 71% | 8 of 20 | 12 | 29 s* |
| Ask Maps (Google Maps) | 49 | 36–61 | 61% | 12 of 20 | 30 | 8 s |
*Grok Bot's times come from 53 timed answers.
Each first answer earns 1 point if it's right, 0.75 for a small error, 0.25 if it's partly right, 0 for no answer and −1 if it's wrong. So a wrong answer costs more than no answer.
What we found
- Dots leads. It scores 92, ahead of every other assistant at 95% confidence, with no wrong answers.
- Four assistants are close. Muse, Claude Duet, Instinct and Grok Bot sit within each other's margin of error.
- Ask Maps is the fastest and the least accurate. It answers in a median 8 seconds and gave 30 wrong answers out of 150.
- Answers change between asks. We asked 20 questions three times. Dots was right all three times on 17, Instinct on 5.
- Closed places still get recommended. 12 answers sent people to a place that a public source showed had closed, moved or changed its name.
The questions
Five kinds of request: finding a place nearby (54), place details (41), planning a route or a time (33), booking and contact details (11) and sharing with friends (11). 46 are written in the city's own language. Most name a day and a time between 15 and 21 November 2026, so every answer can be checked against what was true then.
An example from Mexico City: "Which Sanborns is closest to the Ángel de la Independencia and still open at 10:30 pm on Tuesday 17 November?" The key says none is: the stores near the Ángel close at 10 pm.
How we decided what's true
Every key cites its sources, in this order: the place's own website and notices, the operator's timetable for transit, the place's Google Maps listing where it has no page of its own, then OpenStreetMap, official registers and dated press. Where sources disagree, the place's own page wins. Every key was drafted twice, independently.
Claude Opus 5.5 grades each answer against its key. As a cross-check, OpenAI's gpt-6.1-sol regraded 60 answers without seeing the grades and agreed on 48.
Limits
This is one run, in November 2026, and assistants change. The keys hold for the week the questions name. The judge is a Claude model, and one assistant, Claude Duet, is built on Claude. 27 keys cite Google Maps; without those questions, Dots is still first and Ask Maps still last.
Data
All 150 questions, keys, sources and grades are on GitHub under CC BY 4.0, with a script to recompute the scores: github.com/voygr-tech/placebench. Full write-up with charts: voygr.tech/placebench.
If you build an assistant or agent that answers questions about places and want it tested on the same questions, get in touch.
Top comments (0)