DEV Community

Cover image for Kimi K3.1 Pro vs Qwen 4 vs Claude Fable 5.5 for Front-End Work: Benchmarks, Cost per Build, and a Three-Model Test Script
AIHubMix
AIHubMix

Posted on Edited on

Kimi K3.1 Pro vs Qwen 4 vs Claude Fable 5.5 for Front-End Work: Benchmarks, Cost per Build, and a Three-Model Test Script

TL;DR. One of the three has shipped. Moonshot released Kimi K3.1 Pro on 23–24 October, and AIHubMix already serves it as kimi-k3.1-pro. Alibaba hasn't released Qwen 4 yet (its tiers were named at Apsara in September), and Fable 5.5 has only appeared in leaks. For them there's an anonymous Arena model (Kalmia, tied to Qwen by tokenizer fingerprint) and a claude-fable-5-5 catalogue listing that was removed a day later. What the evidence says so far:

  • Moonshot's numbers: K3.1 Pro ahead of Fable 5.1 on agentic coding. 65.7 to 57.9 on Terminal-Bench 4.0 and 75.5 to 67.4 on DeepSWE, but 31.5 to 53.5 behind on ProgramBench. Self-reported, harnesses differ, and the rival is Fable 5.1, not 5.5.
  • Independent numbers, mostly last generation: Fable. Claude Fable 5.1 scores 53 to Kimi K3's 44 on Artificial Analysis's Intelligence Index v4.3.2, and 52% to 13% on its Terminal-Bench 4.0 run.
  • Cost: Kimi. $2.00 per index task for K3 against $7.63 for Fable 5.1. K3.1 Pro is cheaper per token than K3: Moonshot lists it at $2.00 / $10.00 per million, a third below K3's $3.00 / $15.00.
  • Front end: split. K3 topped Arena's Frontend Code board at launch, ahead of Fable 5. On the LMArena WebDev ratings AIHubMix shows today, Fable 5.1 leads K3, 1,742 to 1,655.
  • Token price: Qwen. Qwen3.8-Max's list output price, $6.00 per million, is 60% of K3.1 Pro's and 40% of K3's.
  • One one-prompt build, at official list prices and the token counts of AIHubMix's own K3 run: about $1.14 on K3, $0.76 on K3.1 Pro, $0.46 on Qwen3.8-Max and $3.80 on Fable 5.1, if all of them used the same tokens. That's an assumption, not a measurement.

The engineering problem for launch week: neither price sheets nor vendor tables will settle this; one measured run per model will. This post collects what's known, checks the loudest claims, and gives a script that sends the same Three.js prompt to all three.

Why these three, and the task that decides it

  • Qwen 4 is the same-week, same-tier rival. Kibitz and Kalmia ran side by side in Code Arena's battle mode. K3 and Qwen3.8-Max were the summer's two Chinese trillion-parameter flagships, both with downloadable weights.
  • Fable 5.5 is the model both are chasing. Moonshot measured K3 against Claude Fable 5 at launch and K3.1 Pro against Fable 5.1. Developers who use Claude ask the same thing of every Kimi release: can it do Fable's job for a fraction of the price?
  • One task decides it: a one-prompt 3D front-end build in Three.js. Kimi's teaser shows exactly that, and Moonshot lists 3D creation among K3.1 Pro's main gains. Arena testers gave Kalmia prompts like "a 3D cycle riding a cow using three.js". Front end is also where Kimi has independent evidence on its side.

What's known about each model

Kimi K3.1 Pro Qwen 4 Claude Fable 5.5
Status Shipped, 23–24 Oct Named, 22 Sep Leaked, 13 Oct
Codename seen Kibitz (likely) Kalmia (likely) Onigiri (likely)
Parameters 3T Not announced Not announced
Open weights By 4 Nov (announced) 27B shown as open No
Context 1M Not announced Not announced
Price $2.00 / $10.00 (official) Not announced 5.1's (placeholder)
Model ID seen kimi-k3.1-pro None claude-fable-5-5

The Kimi column comes from Moonshot's launch post and API docs. K3.1 Pro is a post-trained K3 with 3T parameters (K3 has 2.8T), image and video input, and a 1M context priced flat. For your client code, the rules match K3's: thinking is always on, reasoning_effort takes low, high or max and defaults to max, temperature is fixed at 1.0, and max_completion_tokens defaults to 131,072 with a ceiling of 1,048,576. Moonshot says "the full model weights will be released by Nov 4".

The price cell is Moonshot's official list, per million tokens: $2.00 input, $10.00 output, $0.20 cached, a third below K3's $3.00 / $15.00 / $0.30 (¥13 / ¥68 against ¥20 / ¥100 on the CNY list). AIHubMix serves Kimi K3.1 Pro under Moonshot's ID; the cost model below uses the official lists.

The other two columns are still leaks or stage announcements. Qwen 4's four tier names (Max, Plus, Flash, 27B) come from attendee reports of Alibaba's Apsara keynote. Its architecture preview, Qwen3.8-Flash-Next, has open weights: 125B main parameters plus 51B of n-gram embeddings, about 6B active per step.

The Fable column is the shakiest. On 13 October a claude-fable-5-5 entry was added to LiteLLM's price list from an Azure catalogue, with its price, context and output fields copied from Fable 5.1 ($10 / $50, 1M context, 128K output). It was removed a day later, when nobody could find a deployment. TestingCatalog reported on 17 October that partners have the model. A test model called claude-onigiri-eap was briefly visible on Artificial Analysis and is the likeliest candidate for it.

The hype, checked

Claim Status
Qwen 4 has 5–10 trillion parameters Wrong: that target is for Qwen 4.5 and 5
K3.1 built 13,707 Minecraft blocks for $0.38 One leaker, unreproduced
Kalmia beat GPT 6.1 Sol Ultra on a prompt One tester's anecdote
Fable 5.5 is live on Azure Listing removed a day later
Code Arena's "korrine" was K3.1 Wrong: tokenizer says Qwen-family
K3.1 costs the same as K3 Wrong: Moonshot's list is a third lower

The same-price claim came from a September registry listing that showed K3's price, likely a placeholder; Moonshot's own list now prices K3.1 Pro a third below K3 ($2.00 / $10.00 against $3.00 / $15.00). The Minecraft figure is from @ChrisGPT on X: 13,707 blocks for $0.38 from K3.1, against 5,738 blocks for $5.31 from a GPT-6 model. No prompt or token counts were published, so there's nothing to rerun. And anonymous labels have been wrong before: in August a Code Arena model was taken for K3.1 until a tokenizer fingerprint placed it in the Qwen family. K3.1 Pro has shipped, but Moonshot hasn't said it was Kibitz, so treat Kibitz, Kalmia and Onigiri as strong guesses until the labs confirm.

Baseline: K3, Qwen3.8-Max and Fable 5.1

Apart from a first, preliminary Arena score (below), K3.1 Pro's benchmark numbers are Moonshot's, so the independent baseline is still the previous generation.

Kimi K3 Qwen3.8-Max Claude Fable 5.1
Released Jul 2026 Aug 2026 Sep 2026
Weights Open Open Closed
Context / max output 1M / 1M 1M / not stated 1M / 128K
Official in / out $3.00 / $15.00 $2.00 / $6.00 $10.00 / $50.00

Prices are official list prices per million tokens; AIHubMix sells Qwen3.8-Max for less, at $1.69 / $5.07. Both open-weight models come with custom licences.

Kimi vs Fable. From Artificial Analysis, Intelligence Index v4.3.2, both models at max effort:

AA result Kimi K3 Fable 5.1
Intelligence Index 44 53
Terminal-Bench 4.0 13% 52%
Humanity's Last Exam 47% 59%
AA-LCR (long context) 89% 85%
AutomationBench 58% 59%
Cost per task $2.00 $7.63
Output tokens per task 48k 78k

Fable 5.1 is clearly ahead on hard reasoning and terminal work. Kimi leads only on long-context retrieval and ties on business automation. Fable also spent more tokens per task, so each task cost almost four times as much, a wider gap than the list prices suggest.

Kimi vs Qwen. These independent figures are older: Artificial Analysis results from August 2026 (Intelligence Index v4.1.1), as compiled by Qubrid. K3 scored 57 and Qwen3.8-Max 56, and K3 was cheaper per task ($0.86 to $1.14) because Qwen spent more tokens. In TrilogyAI's blind-reviewed StackPerf test, each model got an hour to analyse the same 269 files from two codebases (Qwen ran as the 3.8-Max preview). K3 scored 83 and Qwen 80, and the reviewers called the two "complementary". Don't compare the v4.1.1 and v4.3.2 numbers with each other: Artificial Analysis rescaled the index between them.

K3.1 Pro vs Fable 5.1: Moonshot's numbers

Figures are Moonshot's self-reported launch numbers; harnesses differ by model and nobody has reproduced them yet. Most of K3.1 Pro's coding runs used a Claude Code harness, and Fable 5.1 ran at max effort with fallback. Moonshot's table has Fable 5.1, not 5.5, and no Qwen model.

Moonshot's result Kimi K3.1 Pro Fable 5.1
Terminal-Bench 4.0 65.7 57.9
DeepSWE v1.1 75.5 67.4
SWE-Marathon v1.1 51.7 45.6
ProgramBench 31.5 53.5
Humanity's Last Exam, no tools 51.1 57.6
OSWorld V2.1 79.6 80.7
Hallucination rate 8.3 23.3

The hallucination row is Moonshot's internal Kimi Hallu Bench (hard, with tools); lower is better.

K3.1 Pro wins the long agentic coding runs and makes far fewer factual errors on Moonshot's own test. It loses badly on ProgramBench, loses on Humanity's Last Exam without tools, and roughly ties on OSWorld. On the same test version as its predecessor, DeepSWE went from K3's 67.5 to 75.5.

Don't chain the Terminal-Bench rows across tables. Artificial Analysis's harness put K3 at 13% on Terminal-Bench 4.0; Moonshot's 65.7 for K3.1 Pro comes from a different harness. It's not a jump from 13 to 65.7, and it doesn't overturn Fable's independent lead until someone reruns it.

Front end: what the evidence says

Arena battles this week. Arena doesn't publish scores for anonymous models, so the only evidence on the new pair is one-shot battles that testers posted on X. TestingCatalog collected the Kibitz reports on 23 October; the Kalmia rows come from the testers' own posts:

Battle report What it showed
Same 3D helicopter prompt, Kibitz vs Kalmia Both built it in 15–20 minutes
Kibitz vs Fable 5.1, 3D test Kibitz quicker; rated near Opus 5.5 Medium
Kibitz, voxel castle "Feels identical to GPT-6.1 Sol Max"
Kalmia, drivable 3D car About 20 minutes, "still has some issues"

LuminaBench's helicopter test is the closest to this post's task. From the same prompt, both models built takeoff, landing and exploded-part views; Kibitz added X-ray and wireframe modes, Kalmia a component index and flight telemetry. The tester called both "very strong". Each report is one prompt and one attempt, so none of it ranks the models. No anonymous Fable 5.5 has been reported on Arena; its likely codename, Onigiri, showed up only in Artificial Analysis's site code.

Kimi vs Qwen: Kimi, narrowly. K3 took first place on Arena's Frontend Code leaderboard at launch with 1,679 points, ahead of Claude Fable 5 (1,631), and led six of the seven frontend domains; Fable 5 kept only Gaming. Sources disagree on Qwen3.8-Max's standing on that board, so it gets no number there. On Arena's WebDev board on 24 October, K3.1 Pro (Max) has a preliminary 1,692 (±14, about 2,450 votes) against 1,673 for Qwen3.8-Max's September build; K3 itself, at 1,655, sits below both Qwen builds.

Kimi vs Fable: unclear. On the LMArena WebDev ratings AIHubMix shows today, Fable 5.1 scores 1,742 to K3's 1,655. That's a different board from the Frontend Code arena K3 topped in July, so the two results don't make a trend, but Fable is ahead on the board AIHubMix tracks. K3.1 Pro's first, preliminary score there, 1,692, narrows the gap but leaves Fable 5.1 about 50 points ahead. Moonshot's own front-end number is its internal Design Bench, an Elo rating: K3.1 Pro 957, GPT-6 Astra 1,076, Claude Opus 5.5 927, and no Fable 5.1 entry. Where K3.1 Pro lands against Fable 5.5 is the first thing to check once both have independent results.

A cost model for one build

AIHubMix's own K3 run of a one-prompt game build (single HTML file, one reference image) used 74,994 completion tokens, 73% of them thinking, over about 42 minutes. Input wasn't recorded; assume 5,000 tokens. Official list prices:

K3:          5,000 × $3.00/M  + 74,994 × $15.00/M ≈ $1.14
K3.1 Pro:    5,000 × $2.00/M  + 74,994 × $10.00/M ≈ $0.76   (same token counts)
Qwen3.8-Max: 5,000 × $2.00/M  + 74,994 × $6.00/M  ≈ $0.46   (same token counts)
Fable 5.1:   5,000 × $10.00/M + 74,994 × $50.00/M ≈ $3.80   (same token counts)
Measured K3.1 Pro / Qwen 4 / Fable 5.5: AIHubMix will run this once all three are available
Enter fullscreen mode Exit fullscreen mode

The shared token count is the weak spot, and real counts move the result both ways. On Artificial Analysis's tasks, Fable 5.1 used more output tokens than K3, and so did Qwen3.8-Max. In one hands-on build by Chase AI, K3 used about six times Fable 5's tokens. K3.1 Pro doesn't think less than K3: in an AIHubMix spot check at low effort, both reasoned several thousand tokens on the same prompt (K3.1 Pro 5,561–7,356, K3 3,013–7,051), though K3.1 Pro finished about twice as fast. Replace these lines with measured completion_tokens.

Prerequisites

  • An AIHubMix key in AIHUBMIX_API_KEY.
  • The OpenAI Python SDK: pip install openai.
  • Kimi, Qwen and Claude models answer on the same OpenAI-compatible endpoint, https://aihubmix.com/v1, so only the model ID changes.
  • kimi-k3.1-pro is live on AIHubMix. AIHubMix serves it on Chat Completions and Messages (Responses returns 404 for the ID); the script uses Chat Completions. qwen4-max and claude-fable-5-5 are guesses until those models ship, and AIHubMix rejects both today (see the failure modes below). To run the current generation now, use qwen3.8-max and claude-fable-5-1.

The script

import os
from openai import OpenAI

client = OpenAI(base_url="https://aihubmix.com/v1", api_key=os.environ["AIHUBMIX_API_KEY"])
prompt = ("Build a single-file HTML page with Three.js: a low-poly cow riding a bicycle "
          "around a track, with orbit controls. Return only the HTML.")

for model in ["kimi-k3.1-pro", "qwen4-max", "claude-fable-5-5"]:  # Qwen 4 and Fable 5.5 IDs are guesses, not on AIHubMix yet
    r = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        timeout=3600,
    )
    with open(f"{model}.html", "w") as f:
        f.write(r.choices[0].message.content)
    print(model, r.usage.prompt_tokens, r.usage.completion_tokens)
Enter fullscreen mode Exit fullscreen mode

Each model writes <model>.html and prints its prompt and completion tokens; each call gets up to an hour, which K3.1 Pro may need, since it thinks at max unless told otherwise. Open the three files side by side, judge the scenes, then price each run from the printed counts instead of the estimate above.

Failure modes and fixes

A 400 on temperature. K3.1 Pro fixes temperature at 1.0, like K3. When an AIHubMix test sent 0.2 to kimi-k3.1-pro, it came back as 400 invalid temperature: only 1 is allowed for this model. The script leaves the field out; if your client or framework injects a default, strip it for Kimi models. The migration post linked below covers the K3.1 Pro 400s.

A first token that takes minutes. Artificial Analysis measured Fable 5.1's first token at 255 seconds, because its reasoning isn't streamed, against 3.6 seconds for K3. Yet Fable finished an average task in 722 seconds against K3's 1,231. A watchdog tuned to Kimi's first token will cut Fable off early; size timeouts and progress indicators for both. If Kimi's wall-clock time is the problem, Moonshot says K3.1 Pro Highspeed (kimi-k3.1-pro-highspeed) has identical capabilities at 2–3× the speed (138–150 tokens a second in an AIHubMix spot check on the main IDs, about 1.6–1.75× K3.1 Pro's rate at low effort over three runs each, with tool calls no faster); the Highspeed post below weighs that against its price.

A 200 that isn't an answer. Fable runs safety classifiers that can decline a request with a normal 200 response and stop_reason: "refusal". Anthropic's stop-reason docs recommend retrying on another Claude model. Check that a reply actually contains HTML before writing it out. In Moonshot's own runs, Fable 5 fell back on 35% of SWE-Marathon tasks; that's a rival's number, but test security-adjacent prompts early.

Long outputs cut short. Fable 5.1 caps output at 128K tokens. K3.1 Pro allows up to 1,048,576 but defaults max_completion_tokens to 131,072, so raise it for whole-file generation and long reports.

Choosing on per-token price. List prices, per-task cost and per-build tokens point in different directions. Compare completion_tokens from real runs on your own prompt before choosing on price.

Trusting a codename. "korrine" was the wrong model. Kibitz, Kalmia and Onigiri are guesses, and K3.1 Pro shipping doesn't confirm Kibitz, so don't wire anything to them until the labs confirm.

A 400 on the Qwen and Fable IDs. qwen4-max and claude-fable-5-5 are guesses. On 24 October AIHubMix answered both with a 400, "This model ID is incorrect", so the loop stops after writing the Kimi file. Swap in qwen3.8-max and claude-fable-5-1 to compare against the current generation, and check the real Qwen 4 and Fable 5.5 IDs on AIHubMix when they launch.

What this can't tell you yet

  • Independent benchmarks for K3.1 Pro beyond a first, preliminary Arena score; every other K3.1 Pro score here is Moonshot's.
  • Prices, context and open-weight status for Qwen 4 and Fable 5.5, which haven't been announced.
  • Qwen3.8-Max's maximum output, which the sources used here don't state.
  • Whether Kimi keeps its front-end lead over Qwen, and where K3.1 Pro lands against Fable 5.5.
  • Real token counts on the Three.js prompt for any of the new models.
  • Qwen3.8-Max's Frontend Code standing: sources disagree.

Which one to pick

These calls rest on the independent K3, Qwen3.8-Max and Fable 5.1 record, weighed against Moonshot's K3.1 Pro numbers and K3.1 Pro's first, preliminary Arena score. Re-check them once K3.1 Pro has more independent results and the other two have launched.

  • Hardest agentic and terminal work, where a failed run costs more than the tokens: Claude Fable 5.5, until independent tests back Moonshot's numbers. Moonshot puts K3.1 Pro ahead of Fable 5.1 on Terminal-Bench, but the last independent check had Fable 5.1 far ahead of K3 there.
  • Front-end and UI generation at volume: start with Kimi K3.1 Pro, and run the Three.js test against Fable 5.5 before committing. Kimi beat Qwen there, but Fable 5.1 leads on WebDev today, and Moonshot's Design Bench didn't rate Fable.
  • High-volume, short tasks: Qwen 4, if it keeps Qwen3.8's pricing.
  • Long documents and retrieval across a 1M-token context: Kimi K3.1 Pro, priced flat across the context. K3 led Fable 5.1 on long-context retrieval at a quarter of the cost per task.
  • Self-hosting or data residency: Kimi or Qwen. K3's weights are about 1.56 TB and need a large multi-accelerator cluster, and K3.1 Pro's are due by 4 November; Qwen 4 27B is the tier shown as open, and its predecessor fits on one 80 GB GPU. Fable has no weights and runs only on Anthropic's API and the clouds that resell it.

AIHubMix's Kimi K3.1 Pro vs Claude Fable 5.1 comparison page puts the current pair side by side.

Keep reading: the Kimi K3.1 Pro series

Sources

Top comments (0)