DEV Community

Anguo
Anguo

Posted on Edited on

World First: GenAI Benchmarks for QCS6490 — Qwen3-0.6B at 1,555 tok/s Prefill, 24.6 tok/s Decode

Three days ago I asked a simple question: how fast does a small LLM run on a Qualcomm QCS6490? Qualcomm's model zoo publishes GenAI numbers for flagship phones and X Elite laptops. For the industrial chip in the Dragonwing RB3 Gen 2 kit — nothing. A blank page. So I measured it myself, with Qwen3-0.6B (official pre-quantized w4a16 package, context 512).

It took four attempts and three distinct dead ends.

Dead end 1: the plumbing. The stock export's built-in downloader stalled at zero bytes for fifteen minutes. A manual resumable download fixed that (3.1 GB), then the unpack step wrote into a 512 MB tmpfs and died. Annoying, fixable, not interesting yet.

Dead end 2: the compiler rejection. The AI Hub compile job came back with: --output_names is not supported for AIMET ONNX models. My first suspicion was the chip — maybe QCS6490 just isn't a GenAI target. So I ran the identical package against a Galaxy S25, a device Qualcomm officially supports for this model. Identical rejection, word for word. Not the chip. Reading the client source afterwards: the export helper appends --output_names unconditionally to every compile. The compiler was never broken; the script was feeding it an option it cannot accept for this model type. I filed it (issue #366 in qualcomm/ai-hub-models); the maintainer confirmed it the next day and scheduled the fix for v0.67.0.

Dead end 3: the precision that doesn't exist. The llama.cpp runtime path needs q4_0. The model's manifest lists q4_0 as supported — but there is no quantization recipe for it anywhere in the repo, for this model or any other. Supported on paper, unreachable in practice. Filed as #367.

What worked: stop arguing with the script and drive the API by hand. I read the failed job's own compile options and its already-uploaded model straight from the API, removed the single offending option, pinned the input shapes myself (the split graphs have dynamic axes; the original job showed exactly how they were pinned), and submitted the compiles directly. Four graph compiles, all green. One link job consolidated them into a single QNN context binary. Then per-graph profile jobs on the real device.

The structure was a surprise in itself: this model's "2 parts" are the embedding layer alone (one Gather) and the entire 28-layer transformer plus lm_head.

The numbers — Qwen3-0.6B, w4a16, context length 512, Dragonwing RB3 Gen 2 (QCS6490):

  • Prefill: 1,555 tok/s (a 128-token prompt processes in 82.3 ms)
  • Decode: 24.6 tok/s (40.7 ms per token)

Method note, because it matters: these are derived from per-graph profile timings on real silicon (embedding graph + transformer graph, prompt and token variants), not from an end-to-end chat demo. Every compile, link and profile job is linked in the public record below, so the whole chain is auditable and reproducible.

These are the first GenAI tokens/second numbers ever published for QCS6490. First — as in nobody, including Qualcomm, had put a number on this chip before. The flagship phones have had their scorecards for a while; the board that actually ships in industrial products was a blank page until today.

Full record, all job links, and the other models measured on this chip (vision models included): https://github.com/anguoyang/qcs-bench
Model card: https://huggingface.co/anguoyang/qwen3-0.6b-qcs6490

When v0.67.0 lands I'll re-run the stock one-command export and publish the comparison. The hand-built chain gives it something to be checked against — which is really what a benchmark is for.

Top comments (0)