While benchmarking free AI APIs I ran the identical model — nemotron-3-super-120b, NVIDIA's own — on two endpoints:
- NVIDIA's own API: 7.6 s to first token, 36.7 tok/s
- OpenRouter's free tier: 2.7 s to first token, 45.7 tok/s
Same weights. Same request. Different serving infrastructure.
I'd assumed the first-party endpoint would be the fastest path to a vendor's own model. It wasn't, at least not on the free tier that day. Free capacity is shared and unevenly provisioned, and a reseller with spare headroom can beat the source.
Two practical takeaways:
- If a specific model matters to you, test every endpoint that serves it rather than assuming the vendor's own is best.
- Treat any single measurement as a snapshot. Re-running the same request on NVIDIA days apart gave anywhere from 27 to 100 tok/s. The swing is itself the finding.
Cross-platform numbers for a dozen free APIs: https://toolfreebie.com/free-ai-api-speed-test/
Top comments (0)