DEV Community

XM Yang
XM Yang

Posted on

I ranked AI models using only public benchmarks — and stopped publishing scores

Most "best AI" lists publish scores nobody can reproduce. Different agent harnesses, different prompts, vendor-reported numbers mixed with independent runs — the numbers look precise and mean almost nothing.

So on Tools Hub's rankings pages I made two rules. First: order only, no scores. Second: every board names its source on the page — SWE-bench Verified (independent runs compiled by Epoch AI) for coding, HealthBench for medical questions, LMArena's blind human votes plus EQ-Bench for writing, and Arena's text-to-image votes for image models.

The interesting part is where the boards disagree. EQ-Bench's judged board puts GPT first for creative writing; LMArena's human voters put Claude first. I follow the humans and say so on the page. That disagreement is more useful to a reader than a fake-precise 9.4/10.

Four boards are live (coding, medicine, writing, image). Happy to hear what other public benchmarks are actually worth ranking from — that's the bottleneck, not the page template.

https://tools-hub.sweetai.work/rankings

Top comments (0)