DEV Community

Cover image for DeepSeek V4.1 Flash 1017 vs V4 Pro: Is the Flash Enough Now?
AIHubMix
AIHubMix

Posted on

DeepSeek V4.1 Flash 1017 vs V4 Pro: Is the Flash Enough Now?

#ai #deepseek #llm #api

DeepSeek's cheaper model now beats its pricier one. On every coding and agent benchmark DeepSeek publishes for both, V4.1 Flash 1017 (October 17) scores higher than V4 Pro 0813: 42.3 against 12.4 on Terminal-Bench 4.0, 77.0 against 62.7 on DeepSWE v1.1. Pro still costs 3.3 times as much per output token, and on a costed agent session it comes out at about four times the Flash's bill.

On paper that settles it. In practice, if you run a pipeline that was tuned on deepseek-v4-pro-0813, those are DeepSeek's numbers, not yours. The engineering problem is verifying the switch on your own workload: pass rate, output length, and cost per run, measured the same way on both models. This post collects the published numbers, gives a short script that calls both models with one key, and lists what skews the comparison.

The published numbers

Price

Model Input, peak Output, peak Cache hit, peak
V4.1 Flash 1017 $0.30 $1.20 $0.006
V4 Pro 0813 $1.32 $3.96 $0.044

USD per 1M tokens, from the DeepSeek pricing page. Off-peak is half price for both models. AIHubMix lists Flash 1017 at $0.3098 / $1.2392 and Pro 0813 at $1.3944 / $4.1832 (input / output, peak).

The ratios: Pro is 4.4× on input, 3.3× on output, and over 7× on cache hits. On DeepSeek's own API, Pro's concurrency limit is also a fifth of the Flash's: 500 against 2,500.

Benchmarks

Pro figures come from Table 3 of DeepSeek's V4.1 Flash paper, all at max effort; 1017 figures come from the model card. All of them are DeepSeek's measurements, and nobody has reproduced them.

Benchmark V4 Pro Flash (Sept) Flash 1017
Terminal-Bench 4.0 12.4 31.2 42.3
DeepSWE v1.1 62.7 74.2 77.0
ProgramBench 15.5 20.3 29.1
NL2Repo-Bench 61.5 65.4 68.2
Automation-Bench 43.2 54.8 60.5
Agents' Last Exam 25.7 31.8 35.4
HLE, text-only subset 42.7 39.1 44.0
GPQA Diamond 92.4 90.9 —

A dash means DeepSeek has not published that score for 1017.

The September Flash had already passed Pro on every coding and agent row; 1017 widened the gap. On the text-only HLE subset, 1017 (44.0) is now ahead of Pro (42.7) too. Pro's one possible remaining lead is GPQA Diamond. Artificial Analysis reached the same verdict independently on its general index in September, calling V4.1 Flash DeepSeek's "new flagship" ahead of V4 Pro 0813.

One session, costed

No published test gives per-task token counts for either model, so this is an estimate from stated assumptions: an agent bug-fix session with 40 model calls, 1.2M input tokens in total (90% cache hits), and 60K output tokens including thinking. Same token counts on both models, AIHubMix list prices:

Model Peak Off-peak
V4.1 Flash 1017 $0.118 $0.059
V4 Pro 0813 $0.469 $0.233 
Pro, peak:   0.12M × $1.3944 + 1.08M × $0.0465 + 0.06M × $4.1832 = $0.469
Flash, peak: 0.12M × $0.3098 + 1.08M × $0.0062 + 0.06M × $1.2392 = $0.118
Enter fullscreen mode Exit fullscreen mode

Pro costs about four times as much per session. To break even, Pro would have to finish the same job with roughly a quarter of the Flash's tokens. The only published verbosity measurement says otherwise: Artificial Analysis found the September Flash used 1.62 times Pro 0813's output tokens on its index, so Pro needed about 62% as many, not 25%.

Prerequisites

  • An AIHubMix API key exported as AIHUBMIX_API_KEY. The AIHubMix quick start walks through creating one.
  • The OpenAI Python SDK (pip install openai), pointed at https://aihubmix.com/v1.
  • Both model IDs: deepseek-v4.1-flash-1017 and deepseek-v4-pro-0813. One key covers both.
  • An eval set from your real workload, with a pass/fail check you already trust.

The script uses Chat Completions (/v1/chat/completions), the endpoint AIHubMix's model listing shows for 1017.

Run both models on the same prompt 

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["AIHUBMIX_API_KEY"], base_url="https://aihubmix.com/v1")

for model in ("deepseek-v4-pro-0813", "deepseek-v4.1-flash-1017"):
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Find the bug: def mean(xs): return sum(xs) / len(xs) - 1"}],
        max_tokens=16000,
    )
    print(model, resp.usage.completion_tokens, resp.choices[0].message.content[:200])
Enter fullscreen mode Exit fullscreen mode

Each line of output is the model ID, its completion_tokens, and the first 200 characters of the answer. Replace the single prompt with your eval set and record three things per model:

  • Pass rate on your own checks. The benchmarks say the Flash should win on coding and agent work; your eval set says whether that holds for you.
  • completion_tokens per prompt. This is the number that decides cost. 1017 was trained for shorter outputs, but nobody has measured it against Pro yet, so your logs are the first measurement that applies to your workload.
  • Cost per run, read from the AIHubMix request logs rather than estimated from list prices.

Failure modes that skew the comparison

The Flash prints more tokens, so it looks more expensive

Token counts alone don't decide cost. At the prices above, Pro only breaks even if it uses about a quarter of the Flash's tokens. Multiply by price before you compare.

The same eval costs twice as much on one run as on another

Off-peak is half price for both models. If one model's run lands at peak and the other's off-peak, the cost gap halves or doubles. Run the two back to back, in the same price window.

Image cases can't be scored on Pro

Pro does not take images at all; 1017 does. Split screenshot, chart, and UI cases out of the shared eval set. If your workflow has any of them, that decides the question on its own.

No 1017 score for your science-QA workload

GPQA Diamond is the one benchmark where Pro (92.4) beat the September Flash (90.9), and DeepSeek has not published a 1017 score. If science QA is your main workload, test it on your own questions before moving.

What nobody has verified yet

  • Every benchmark number above is DeepSeek's own; no third party has reproduced them.
  • The paper's Pro column does not name the 0813 build. Its Terminal-Bench 2.1 score matches the 0813 card, which is why it's read as 0813 here.
  • 1017's output length against Pro. Artificial Analysis's 62% figure is for the September Flash, and no third party has re-measured 1017.
  • Per-task token counts for either model. The session cost above is an estimate from assumptions, not a measurement.

Decision rules

  • Keep Pro if you have a production pipeline validated on Pro 0813 and no time to re-run evaluations this month, or a science-QA workload where the GPQA gap has shown up in your own tests.
  • Move to the Flash for coding agents, terminal automation, tool-calling pipelines, anything with images, and anything high-volume where concurrency matters.
  • Test before moving long-form writing and analysis where you have relied on Pro's shorter, denser answers.

There is no deadline forcing the move: DeepSeek's changelog says it will keep serving V4 Pro, and no retirement date has been announced. The AIHubMix compare page lists both models' specs and prices side by side while you run the evals.

Keep reading: the DeepSeek V4.1 Flash 1017 series

Sources

Top comments (0)