A client asked me for a simple thing.
Not ChatGPT.
Not an agent.
Not a multimodal assistant that can explain invoices, generate React components, and write poetry in three languages.
Just a small classifier embedded into a website.
The job sounded boring in the best possible way:
take some text, classify it, return a result, keep it fast.
So I started looking at the usual solutions.
And then I had one of those moments where you stop reading documentation, lean back, and ask:
Are we seriously doing this?
Because the answer I kept running into looked like this:
download a huge runtime
download a huge model
initialize a big ML stack
then classify one small piece of text
In one setup, the path was getting close to something like 250 MB per user.
For a simple classifier.
On a website.
From a server.
Every time.
No. Sorry. That is insane.
The problem
The web has a strange habit now.
You ask for one small AI feature, and the answer is often:
bring the entire construction company.
But sometimes I do not need a construction company.
I need one person on the construction site.
One task.
One tool.
One result.
This is especially true for simple classification, embeddings, semantic search, routing, filtering, ranking, small local decisions.
Not every AI problem needs an LLM.
Not every website needs a full inference engine.
Not every user should pay a 250 MB download tax because we were too lazy to think smaller.
So I started digging
I wanted something simple:
- runs in the browser
- does not require a server for inference
- small enough to actually ship
- works with transformer-style models
- can tokenize text
- can run BERT-like forward inference
- can produce embeddings or classification input
- does not bring ONNX Runtime, Candle, ndarray, or half the internet with it
At first I thought:
“Surely someone already made the tiny version.”
There are great tools out there.
Transformers.js is powerful.
ONNX Runtime Web is powerful.
Candle is powerful.
But that was exactly the problem.
They are powerful because they are general.
I did not need general.
I needed narrow.
I needed small.
So I built one.
Introducing wasmicro
wasmicro is my attempt at a tiny transformer inference runtime for the web.
Current WASM bundle size:
~94 KB
Not 94 MB.
94 KB.
The project is here:
https://github.com/Xzdes/wasmicro
Live demo:
https://xzdes.github.io/wasmicro/
It is not perfect.
It is not finished.
It is still being tested.
But it already does the thing I needed: run a small transformer-style pipeline in the browser without shipping a giant runtime.
What it does today
Right now wasmicro supports:
- tiny owned tensors
- safetensors loading
- WordPiece tokenizer
- BERT encoder forward pass
- mean pooling for embeddings
- WASM bindings
- SIMD128 matmul path
- i8/u8/q4 quantized weight types
- converter tool for HuggingFace models
The design rule is simple:
if it does not make the WASM bundle smaller, faster, or useful for a transformer architecture, it probably does not belong.
No training.
No autograd.
No optimizer.
No general tensor framework.
No “module zoo”.
Just forward inference.
Why not just use a big runtime?
Because sometimes the runtime is bigger than the problem.
If I am building a serious AI application, yes, I will use serious AI infrastructure.
If I need WebGPU, many architectures, image models, audio models, generation, pipelines, fallback backends, and broad model support, then I should use the big tools.
But if I need a small classifier embedded into a website?
I do not want the entire AI ecosystem.
I want the smallest useful thing.
The difference is like hiring:
- a whole construction company
- or one worker with the correct tool
The web keeps giving me the company.
I needed the worker.
The current numbers
The WASM bundle is currently around:
94 KB after wasm-opt -Oz
The hard size ceiling I set for myself is:
250 KB
The default library dependency set is intentionally tiny.
The project avoids pulling in things like:
- ndarray
- candle
- rayon
- serde_json
- chrono
- getrandom
The converter CLI can use heavier dependencies, because it runs on a desktop machine and never ships to the browser.
The browser runtime stays small.
Is it faster?
That depends what “fast” means.
If you mean maximum throughput on GPU against a fully optimized WebGPU runtime, probably not.
That is not the fight.
The fight I care about is:
- cold start
- first useful result
- small runtime
- simple embedding/classification tasks
- CPU/WASM path
- no huge framework download
I want to compete on:
time to load
time to first embedding
runtime size
simple integration
Not on “who supports 200 model architectures”.
That is a different game.
What is missing
A lot.
This is still early.
The project still needs:
- better public benchmarks
- easier model download story
- more optimized attention path
- less allocation during forward
- better q4 loading for BERT
- cleaner zero-config examples
- more browser measurements
- more real-world classification demos
Also, the live demo currently expects you to provide model files.
That is not ideal.
But it is already enough to prove the point:
You do not always need a massive runtime to do a small AI job.
The real lesson
This started as a simple client task.
“Add a classifier to a website.”
Then I looked at the common path and saw the cost.
And I could not accept that the answer to a small feature was:
ship hundreds of megabytes and hope nobody notices.
Users notice.
Browsers notice.
Mobile connections notice.
Cold start notices.
So I built the smaller thing.
Not because it is perfect.
Because the alternative felt wrong.
Final thought
AI tooling is amazing right now.
But we are also getting lazy.
We reach for the biggest tool because it is convenient.
Sometimes that is correct.
Sometimes it is absurd.
If the job needs a crane, use a crane.
If the job needs one person with a hammer, do not send a construction company.
wasmicro is my attempt at the hammer.
Small.
Narrow.
Still rough.
But already useful.
GitHub:
https://github.com/Xzdes/wasmicro
Demo:
Top comments (3)
The 250KB hard ceiling is the part I respect most — it forces actual architectural decisions instead of just throwing more at the problem.
One thing I'd push back on slightly: WordPiece was the move in 2018, but SentencePiece / Unigram is what most modern models actually ship with. If you're targeting BERT-family models that migrated to SentencePiece (most of what you'd pull from HuggingFace today), you'd need a different tokenizer path. Not a dealbreaker, just means the converter has more work than expected.
On mean pooling — have you measured how much the [CLS] token approach saves vs mean pooling for classification tasks? In my experience, mean pooling wins on semantic search (which is why sentence-transformers use it), but [CLS] is often competitive on classification and avoids the extra pass.
Thanks, this is a really thoughtful comment — and you're right on the tokenizer side. WordPiece covers the English MiniLM / BGE / E5 models I was targeting, but the moment you go multilingual or SentencePiece-based (DeBERTa, T5, and friends) you need a different path. I actually ran into exactly that adding a T5 path, so it's firmly on the list.
On pooling — fair nudge. I default to mean pooling since that's the sentence-transformers convention for embeddings, but I haven't benchmarked CLS head-to-head for classification, so that's worth actually measuring rather than assuming. Appreciate the careful read!
Glad the tokenizer point landed — SentencePiece vs WordPiece is one of those things that's obvious in hindsight but easy to miss until you hit a non-English model.
On CLS vs mean pooling for classification: worth measuring indeed, but my gut says CLS wins for classification tasks where the model was pretrained with [CLS] as the classification head. Mean pooling spreads signal across all tokens, which helps for semantic similarity but can dilute the "this is category X" signal. Would be curious to see your benchmarks if you get around to it!
Good luck with the T5 path — curious what use case pushed you toward T5 specifically?