- Book: Observability for LLM Applications
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
If your code calls deepseek-flash, it has been talking to a different model since October 17. DeepSeek's release note promised "shorter outputs than the previous version", and its own docs say the name "is now served by DeepSeek-V4.1-Flash-1017". The only benchmark table was DeepSeek's. The Decoder pointed out that day that no independent results were out yet.
Now there is one. Artificial Analysis scores the reasoning version of V4.1 Flash 1017 at 45 on its Intelligence Index. The previous V4.1 Flash scored 39 on the same index. On October 19 the page was posted to Hacker News, where it drew more than 40 comments.
The cost row moved more than the score did.
What Artificial Analysis measured
Both pages use Intelligence Index v4.3.2, which bundles 10 evaluations, among them Terminal-Bench 4.0, SciCode, Humanity's Last Exam and AA-LCR (1017 page). Both test the "Max" reasoning variant, priced at DeepSeek's API rates. Side by side, from the V4.1 Flash page and the 1017 page:
| V4.1 Flash (Max) | V4.1 Flash 1017 (Max) | |
|---|---|---|
| Released | September 10, 2026 | October 17, 2026 |
| Intelligence Index | 39 | 45 |
| Output tokens to run the index | 250M | 210M |
| Cost per index task | $0.49 | $0.26 |
| Price per 1M input / output | $0.30 / $1.20 | $0.30 / $1.20 |
| Output speed | 231.2 tokens/s | 230.4 tokens/s |
| Time to first token | 0.91s | 1.08s |
The page puts 1017 at #2 of 121, and its FAQ calls the score "well above average among other open weight models of similar size (median: 19)." The license is MIT, and the context window is 1M tokens (1017 page).
The cost drop came from tokens, not price
Look at the price row. It didn't move. DeepSeek charges the same per token for 1017 as it did for the model it replaced, and Artificial Analysis's cost per task still fell from $0.49 to $0.26, a drop of about 47%.
The output-token row moved too, from 250M to 210M, about 16% fewer. Artificial Analysis doesn't break the per-task cost into input, cached input and output, so I can't tell you how much of the 47% came from shorter answers and how much from somewhere else. The page does show a model at an unchanged list price getting cheaper to run on the same 10 evaluations.
It is still a talker. The 1017 page calls 210M output tokens "very verbose in comparison to the median of 140M." A low price per token and a long answer can cancel out, so read the per-task row.
Two settings behind the $0.26
The $0.26 assumes two things your traffic may not match.
The first is the price. Artificial Analysis uses $0.30 per million input tokens and $1.20 per million output. On DeepSeek's pricing page, those are the peak rates for deepseek-flash. Off-peak is half: $0.15 and $0.60, with cached input at $0.003 instead of $0.006. Peak hours are 01:00 to 04:00 and 06:00 to 10:00 UTC, Monday through Friday, and Chinese public holidays count as off-peak. A batch job that runs outside those windows, say between 10:00 and 01:00 UTC on a weekday or any time at the weekend, pays half of what the table assumes.
The second is effort. Artificial Analysis tested Max. DeepSeek's thinking-mode guide says thinking is on by default "with the default effort being high", and the documented values are low, high and max. If you never set reasoning_effort, you are not running the configuration that scored 45, and you are probably not paying its token bill either. In the Hacker News thread, one commenter testing earlier Flash versions (not 1017) wrote that max "used a lot more tokens (almost twice as many IIRC) for little to no benefit over high." That's one person's experience on older models, not a measurement.
Price it per task on your own traffic
You don't need a leaderboard to get your own cost per task. DeepSeek's chat completion reference returns prompt_cache_hit_tokens, prompt_cache_miss_tokens and completion_tokens in every response's usage object. Add those up per task, then apply the rate for the hour the request ran:
// deepseek-flash, USD per 1M tokens (DeepSeek pricing page)
const PRICES = {
peak: { hit: 0.006, miss: 0.3, out: 1.2 },
offPeak: { hit: 0.003, miss: 0.15, out: 0.6 },
};
type Usage = {
prompt_cache_hit_tokens: number;
prompt_cache_miss_tokens: number;
completion_tokens: number;
};
// Peak: 01:00-04:00 and 06:00-10:00 UTC, Monday-Friday.
// Chinese public holidays are off-peak; not handled here.
function isPeak(at: Date): boolean {
const day = at.getUTCDay(); // 0 = Sunday, 6 = Saturday
if (day === 0 || day === 6) return false;
const h = at.getUTCHours();
return (h >= 1 && h < 4) || (h >= 6 && h < 10);
}
export function costUSD(u: Usage, at: Date): number {
const p = isPeak(at) ? PRICES.peak : PRICES.offPeak;
const scaled =
u.prompt_cache_hit_tokens * p.hit +
u.prompt_cache_miss_tokens * p.miss +
u.completion_tokens * p.out;
return scaled / 1_000_000;
}
Sum costUSD over every call that belongs to one task, and log it next to whether the task passed. Run your eval set at high and at max. The number you want is the cost of a passing task at each effort.
Race it against the model you run today
An index score tells you how a model does on Artificial Analysis's tasks. Your repo is a different set of tasks. DeepSeek documents how to point Claude Code at its API through an Anthropic-format endpoint at https://api.deepseek.com/anthropic (docs), so you can give the same ticket to the setup you use now and to one running on 1017. That guide sets CLAUDE_CODE_EFFORT_LEVEL to max, the expensive setting from the section above, so set it to high for a second run if you want to compare efforts.
I build Hermes IDE, so weigh this with that in mind. Hermes runs Claude Code, Codex and "any command-line agent you add" in a real terminal, and ⌘N (Ctrl+Shift+N on Windows and Linux) starts each task on its own hermes/<name> branch in its own worktree (README). Two runs of one ticket can sit in split panes on one repo without touching each other's files.
Two agent sessions in split panes in Hermes 2.3.4, each on its own branch, with status, context use, cost and branch in the sidebar.
Hermes shows the cost an agent reports, or an estimate from its transcript marked as estimated, with token totals. For a model it wasn't built to price, compare the token totals and do the dollars with DeepSeek's price sheet. I haven't run 1017 in it myself. Hermes is free, source-available under BSL 1.1, runs on macOS, Windows and Linux, and isn't affiliated with DeepSeek, Anthropic, OpenAI or Nous Research.
Your deepseek-flash calls already moved
The swap needed no change on your side. The legacy names deepseek-v4-flash and deepseek-v4-flash-vision-exp still work, but DeepSeek's pricing page says their models are retired and the requests go to 1017 at the Flash price. If you want a version that can't change under you, the MIT-licensed weights are one way to hold one still.
Whether 1017 is cheaper for you depends on the hour and the effort. Do you set reasoning_effort explicitly, or have you been running on the default? Tell me in the comments.
If this was useful
The table above is a cost-per-task argument, and so is most of running LLM features in production. Observability for LLM Applications is the book I wrote about tracing LLM calls, attaching cost and quality to each one, and catching the regression before your users do.
If you want to race models on the same ticket, download Hermes IDE or star it on GitHub.
Top comments (0)