I build a browser-based image-to-SVG vectorizer. Every tool in this category publishes a comparison table where it wins every row, which is why nobody believes any of them.
So I published the opposite: a reproducible benchmark, scripts and raw data included, where my own tool loses on speed, loses on file size, and loses outright on monochrome work.
Repository: raster-to-vector-benchmark — MIT, re-runnable.
The setup
One real logo, 1535x621 RGBA, 11,697 distinct colours. Two tracers:
- potrace 1.16 — the standard, and the engine behind a lot of things you have used
- 12pixa — my tool. VTracer compiled from Rust to WebAssembly, running client-side in a Web Worker
The numbers
| potrace 1.16 | 12pixa (colour) | 12pixa (B&W) | |
|---|---|---|---|
| time | 0.10 s | 7.21 s | not measured |
| SVG bytes | 3,331 | 24,119 | 6,503 |
| colours | 1 | 224 | 1 |
| MAE vs original (lower is better) | 10.34 | 3.47 | 13.31 |
potrace is roughly 70x faster and produces a file 7x smaller. That is not a rounding difference, it is two orders of magnitude on time.
And the row that hurts: my own black-and-white preset is the worst result in the table. MAE 13.31 against potrace's 10.34 on the same monochrome job. potrace beats me at the thing potrace is for.
Where my tool wins is colour — MAE 3.47, the most accurate reproduction of the three, and the only one that produces a palette at all. But "wins the row it was built for" is a much smaller claim than the marketing table usually makes.
The methodology mistake I made first — and why it matters
My first attempt compared each output against a thresholded version of the original. potrace scored 97%. My tool scored 21%.
That result was garbage, and the reason is worth internalising: potrace only accepts bitonal input, so prepare.py thresholds the image before handing it over. Comparing potrace's output to the thresholded reference compares it to its own input. It cannot lose. The benchmark was measuring "did potrace faithfully reproduce the file we gave it", which is a question with a known answer.
The fix is to score every candidate against the original colour raster — the thing a user actually had before they started. That single change flipped the outcome and produced the table above.
If you write a benchmark, the first question to ask is: does one contestant get to be measured against a reference derived from its own preprocessing? If yes, throw the numbers away.
A detail that costs people an hour
potrace 1.16 reads PBM/PGM/PPM and BMP. It cannot open a PNG. Every "potrace failed on my file" thread reduces to this. scripts/prepare.py converts first:
# 1. prepare the input (potrace cannot read PNG)
python scripts/prepare.py corpus/logo-1535x621.png
# 2. trace
potrace --svg -o results/potrace--logo.svg out/logo-1535x621.gray.pgm
Timing a browser tool has its own problem: there is no binary to wrap in time. The page is reloaded per run and measured in-page, so the number includes WASM instantiation. That is the honest way to report it — it is what the user waits through.
What I would actually tell you to use
- Monochrome art, logos, line work, laser or vinyl cutting → potrace. Faster, smaller, and more accurate than my own B&W preset. Use it.
- Anything with colour → potrace is not a candidate; it has no mechanism for colour at all.
- 7.2 seconds is slow, and I am not going to describe it as "instant".
Why publish a benchmark you lose
Two reasons.
The first is that an unfalsifiable claim is worth nothing. "Best vectorizer" is not a testable statement. "MAE 3.47 versus 10.34 on this file, here is the script" is, and you can prove me wrong in ten minutes.
The second is that this is how I would want to be sold to. When a tool tells me where it is weak, I trust the rest of the page.
The corpus, the scripts, the raw JSON and the SVG outputs are all in the repo. If you re-run it and get different numbers, open an issue — I would rather fix the benchmark than defend it.
The tool is 12pixa — free, no upload, no account, runs entirely in the browser through WebAssembly. Built by AiJoLabs in Amman, Jordan.
Top comments (0)