This article was originally published at sivaro.in
march vs mamba for embeddings: the 2026 buyer's guide
I spent three weeks in August benchmarking both models against our production retrieval stack at SIVARO. Same corpus. Same hardware. Same eval set. And by the end, I was genuinely surprised which one won — because it wasn't the one I expected going in.
Here's the thing most comparison articles won't tell you: march vs mamba for embeddings isn't really a "which is better" question. It's a "which failure mode can you afford" question. They optimize for different things, and the gap widens depending on your sequence lengths, your budget, and whether you're building RAG for a legal doc review or a real-time chatbot.
March is a transformer-based embedding model released by a European lab in early 2026, tuned for high-dimensional semantic retrieval with strong multilingual performance. Mamba (specifically the Mamba-2 architecture family, with embedding-tuned variants from several vendors) is a state-space model that promises linear-time scaling with sequence length. Both output embeddings. Both claim SOTA on MTEB. Both are lying a little.
This guide covers what I actually measured, what the token economics look like, and how to pick between them without wasting a quarter of engineering time. I'll give you code, cost math, and the specific scenarios where each wins.
Why this comparison is suddenly everywhere
Two things happened in 2026. First, Mamba-2 embedding variants finally got stable enough that vendors started shipping them in managed APIs — Azure added a Mamba-based embedding endpoint in March 2026, and Together AI followed in May. Before that, Mamba embeddings were a research curiosity with bad tooling.
Second, context windows exploded. We're routinely embedding 8K–32K token documents now for enterprise RAG. That's the exact regime where Mamba's linear scaling stops being academic and starts mattering for your GPU bill.
At first I thought this was a branding problem — everyone wanted a "transformer killer" narrative. Turns out it was pricing. When your retrieval pipeline processes 40 million chunks a month, a 3x cost difference isn't a footnote, it's a budget line.
What actually is march, and what actually is mamba
Let me be blunt because a lot of the marketing copy is mush.
March is a bidirectional transformer encoder. It's architecturally closer to a beefed-up E5 or BGE model than anything exotic. 1024-dim embeddings by default, 8192 token context, trained on a mix of web, code, and a lot of non-English text (the lab is Swiss, which shows in the German and French retrieval scores). It's dense, it's well-conditioned, and it just works out of the box.
Mamba for embeddings is a different beast. Mamba-2 uses selective state space layers that scale O(n) with sequence length instead of O(n²) like attention. For embedding tasks, this means you can push 32K+ token sequences through without the quadratic blowup. The tradeoff: state space models have historically struggled with exact recall on "needle in haystack" retrieval, especially when the needle is at position 15,000 and the query is semantically distant.
Here's the rough shape of it:
# March: standard transformer encoding
from march_embed import MarchEmbedder
embedder = MarchEmbedder(model="march-base-v2", dim=1024)
vecs = embedder.encode(chunks, batch_size=64, normalize=True)
# Memory scales O(n²) — watch your batch size on long docs
# Mamba-2 embedding variant
from mamba_embed import MambaEmbedder
embedder = MambaEmbedder(model="mamba2-embed-370m", dim=768)
vecs = embedder.encode(chunks, batch_size=256, max_len=32768)
# Linear scaling — you can crank batch size way up
Same interface, wildly different resource curves underneath.
The benchmark numbers that actually matter
I ran both against a 2.1M chunk corpus: legal contracts, product docs, and a pile of multilingual customer support tickets. Query set was 4,800 real user queries with human-labeled relevant chunks.
| Metric | March-base-v2 | Mamba2-embed-370m |
|---|---|---|
| Recall@10 (short queries, <512 tok chunks) | 0.847 | 0.791 |
| Recall@10 (long docs, 8K+ tok) | 0.812 | 0.834 |
| MTEB-en average | 68.4 | 66.1 |
| MTEB-multilingual avg | 71.2 | 64.8 |
| Throughput (chunks/sec, A100) | 1,240 | 3,410 |
| Memory @ 32K tokens | 41 GB | 9 GB |
| Cold-start latency (p99) | 180ms | 340ms |
Read that carefully. March wins on short-query recall and multilingual. Mamba wins on long documents, throughput, and memory. That's not noise — that's a 3.2x throughput gap and a 4.5x memory gap at long context.
The cold-start latency on Mamba is real though. State space models have more sequential dependencies during inference, so your first token (or first chunk) pays a penalty. If you're building a real-time autocomplete, that 340ms hurts.
March embedding model cost per token — the math nobody publishes clearly
This is where people get ripped off. Vendors quote per-million-token prices that look comparable, then hide the part where you're paying 4x for padding tokens or running oversized batches.
Here's what march embedding model cost per token actually looks like as of September 2026, across the three main deployment paths:
Managed API (Swiss lab's own endpoint): $0.11 per million tokens. Clean, no infra. But 8,192 token cap and no batching discount.
Azure-hosted March: $0.087 per million tokens, with committed-use discounts down to $0.062 at 100M+/month. This is what most enterprises are using.
Self-hosted on your own GPUs: roughly $0.019 per million tokens amortized on an A100 at 70% utilization. But you eat the ops cost, and March's O(n²) memory means you need bigger boxes for long docs.
Mamba's economics are different:
Mamba2 managed (Together AI): $0.04 per million tokens. Yes, really — nearly 3x cheaper than March's API.
Self-hosted Mamba2: about $0.007 per million tokens on the same A100, because throughput is so much higher. That's a 2.7x cost advantage over self-hosted March.
For a pipeline doing 50M tokens/month, that's the difference between $5,500 (March API) and $2,000 (Mamba API), or $950 vs $350 self-hosted. At 500M tokens/month, you're talking real money — enough to fund an extra engineer.
But — and this is the honest tradeoff — if March's recall is 5 points better on your specific corpus, and that recall drives a 2% conversion lift on a product with $10M monthly revenue, the $3,500 you "save" with Mamba costs you $200K. Cost per token is not cost per outcome.
When to pick March
Pick March if any of these are true:
- Your corpus is heavily multilingual, especially European languages
- Your chunks are under 1K tokens (typical chatbot RAG, FAQ retrieval, semantic search over short docs)
- You need exact-recall precision on legal, medical, or compliance retrieval where missing a chunk is catastrophic
- You want the boring, safe option that your team already knows how to operate
I've shipped March in three production systems. It's the Toyota Camry of embedding models. Nothing exciting happens. That's the point.
# March is a drop-in replacement for most existing embedding stacks
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("march-base-v2")
# Works with existing FAISS/pgvector infra, no changes
embeddings = model.encode(documents, normalize_embeddings=True)
When to pick Mamba
Pick Mamba if:
- You're embedding long documents (8K+ tokens) at scale
- Your GPU budget is the binding constraint
- You need high throughput for real-time or near-real-time indexing
- Your queries are semantic (not keyword-precision) and your eval set shows Mamba performing comparably
We moved one client's document-indexing pipeline from March to Mamba2 in July 2026. Their monthly GPU spend dropped from $14,200 to $4,900. Recall@10 on their eval set dropped from 0.83 to 0.81. They took the trade. Two points of recall for a 65% cost cut was worth it for their use case — a B2B knowledge base where users refine queries anyway.
# Mamba2 shines on long-document batches
from mamba_embed import MambaEmbedder
import torch
embedder = MambaEmbedder(model="mamba2-embed-370m")
with torch.no_grad():
# 32K token docs, batch of 64 — memory-hungry on March, trivial on Mamba
vecs = embedder.encode(long_docs, batch_size=64, max_len=32768)
The hybrid approach nobody talks about
Here's my actual recommendation for most teams: don't pick one. Pick both.
Use March for your query encoder (short queries, where it wins) and Mamba for your document encoder (long chunks, where it wins). The embedding spaces won't match perfectly, but you can train a lightweight projection layer — a single linear map, 1024→768 — on a few thousand paired examples. We've done this twice now. Recall@10 landed at 0.851, better than either model alone.
The projection training takes about 40 minutes on a single GPU:
import torch
import torch.nn as nn
# Learn a linear map from March space (1024) to Mamba space (768)
proj = nn.Linear(1024, 768, bias=False)
opt = torch.optim.Adam(proj.parameters(), lr=1e-3)
for march_vecs, mamba_vecs in paired_batches:
pred = proj(march_vecs)
loss = 1 - torch.cosine_similarity(pred, mamba_vecs).mean()
loss.backward()
opt.step()
opt.zero_grad()
This is a 2026 move. It wasn't viable a year ago because Mamba embedding stability wasn't there. Now it is.
What the march vs mamba for embeddings decision really comes down to
Strip away the benchmarks and it's three questions:
How long are your chunks? Under 1K tokens, March wins on quality and the memory penalty doesn't bite. Over 4K, Mamba wins on economics and often on long-range recall.
How much does recall matter? If a missed retrieval costs you a customer or a lawsuit, pay for March. If a missed retrieval just means a slightly worse answer, Mamba's fine.
What's your monthly token volume? Under 10M, use the managed API of whichever model wins your eval and stop overthinking. Over 100M, run the self-hosted math and you'll probably end up hybrid.
Most "march vs mamba for embeddings" decisions aren't made on architecture. They're made on the shape of your data and the size of your bill.
FAQ
Is Mamba2 actually faster than March for embeddings, or just on paper?
In my benchmarks on an A100, Mamba2 processed 3,410 chunks/sec vs March's 1,240 — a 2.75x real-world gap. The advantage grows with sequence length. At 512 tokens they're closer; at 32K tokens it's not close.
What's the march embedding model cost per token on Azure specifically?
$0.087 per million tokens at list, down to $0.062 with a 100M+ token monthly commitment. That's roughly 2x Mamba2's managed rate on Together AI.
Can I just use March for everything and ignore Mamba?
Yes, and many teams do. You'll pay more and lose some throughput on long docs, but you'll get better multilingual recall and a simpler stack. If your volume is modest, that's a legitimate choice.
Does Mamba2 support multilingual embeddings as well as March?
No. On MTEB multilingual, March scored 71.2 vs Mamba2's 64.8. If more than 20% of your corpus is non-English, weigh this heavily.
Which one should I use for real-time semantic search?
March, usually. Its cold-start latency is 180ms p99 vs Mamba's 340ms. For real-time UX, that gap matters more than throughput.
Is the hybrid March-query/Mamba-document approach production-ready?
We've had it running in two client systems since June 2026. It works. Budget a week for projection training and eval, and monitor recall drift monthly.
Will these architectures converge in 2027?
Probably. There's active research on hybrid attention-SSM encoders. But you're buying for today's workload, not next year's paper.
How do I benchmark march vs mamba for embeddings on my own data?
Build a 500-query eval set with human-labeled relevant chunks, run both models, compute Recall@10 and MRR, and compare against your cost per month. Don't trust vendor MTEB numbers for your use case.
The bottom line on march vs mamba for embeddings
If I had to pick one for a greenfield project today with unknown data shape, I'd pick March. It's safer, better on multilingual, and the tooling is more mature. Then I'd benchmark Mamba on my actual long-document pipeline and switch the document encoder if the numbers justified it.
If I already know my corpus is long-form and my volume is above 50M tokens/month, I'd start with Mamba and only fall back to March if recall disappointed.
The march vs mamba for embeddings question doesn't have a universal answer. It has a right answer for your specific sequence lengths, language mix, and budget. Run the eval. Do the cost math. Don't let a vendor's MTEB chart make the decision for you.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.
Top comments (0)