Its input price was half of GPT-4o's, but BenchLM's quality score for it is only an estimate.
Sources: BenchLM, Activepieces and Puter pricing breakdowns.
Quality evidence is thin, so this post covers cost and limits.
- What does a long prompt really cost?
- Is there independent evidence on quality?
📌 THE 128K PRICE JUMP
Gemini 1.5 Pro (Google)
- Cost per token doubled once a prompt exceeded 128,000 tokens (Activepieces).
- Its $1.25 input price was the figure cited for it, labeled Previous in BenchLM's pricing table.
📌 PRICE VS RIVALS
GPT-4o (OpenAI)
- $2.50 per 1M input tokens, double Gemini 1.5 Pro's input price (Activepieces). Claude 3.5 Sonnet (Anthropic)
- $3.00 per 1M tokens; Gemini 1.5 Pro was 58% cheaper (Activepieces, citing Compute Comparison).
📌 WHAT THE QUALITY DATA SAYS
BenchLM score (BenchLM.ai)
- Gemini 1.5 Pro scores 27.16/100 as an estimate, with public rank #191.
- It shares 0 benchmark results with its comparison model, so BenchLM gives no quality verdict.
What this means for your team:
- Model your bill at your longest prompts, since past 128K tokens the per-token rate doubles.
- Compare price against GPT-4o and Claude 3.5 Sonnet, but don't assume quality from price alone.
- Ask for independent test results before you commit a production workload.
- Check current Google pricing, since newer Gemini models list separate rates.
Pull your largest prompt sizes this week and check them against the 128K line before you choose a model.
P.S. Newer Gemini 3.1 Pro lists $2.00 input and $4.00 above 200K tokens, so the long-prompt jump is still there.
Originally shared on LinkedIn.
An AI interview question every day, leadership for tech people and hands-on AI learning:
Top comments (0)