The problem nobody optimizes for
Arabic is spoken by 400M+ people and represents under 1% of internet training data. Every LLM you have used is, effectively, an English model with Arabic bolted on.
I built Fasil — a platform where AI evaluates structured debates in formal Arabic (فصحى) — and that 1% turned out to be the hardest engineering constraint I faced. Harder than the war. Harder than the blackouts. Harder than having zero API budget.
This post is what actually broke, what I measured, and — importantly — which of my assumptions turned out to be wrong.
1. The tokenizer tax is 2.2x, and I measured it
Arabic tokenizes badly. Here is the same sentence, same meaning, through Qwen2.5's tokenizer:
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B")
en = "The strongest argument wins."
ar = "الحجة الأقوى هي التي تنتصر."
len(tok.encode(en)) # -> 5 tokens
len(tok.encode(ar)) # -> 11 tokens
5 tokens in English. 11 in Arabic. Identical meaning, 2.2x the cost.
The cause is morphological. Arabic is highly inflected — a single word can encode subject, tense, object, and possession. A tokenizer trained overwhelmingly on English has no efficient merges for those forms, so it shatters them into subword fragments that carry no meaning on their own.
That 2.2x multiplier compounds through everything:
- Effective context window shrinks by more than half — the same debate transcript eats 2.2x the budget
- Inference cost more than doubles per unit of meaning
- Small models degrade faster than any English benchmark would predict
Mitigation
Aggressive prompt compression, and — more importantly — moving structural logic out of the prompt and into application code.
Debate phases, turn-taking, scoring rules, and state transitions all live in Django. The model is never asked to remember rules. It is asked only to judge.
This is the first rule of low-resource-language engineering: every token spent on instructions is a token stolen from content. In English you can afford that waste. At 2.2x, you cannot.
2. Small models collapse on فصحى, not on dialect
A counter-intuitive finding, and the one that cost me the most time.
Small models handled colloquial Arabic noticeably better than formal Arabic. Obvious in hindsight: colloquial Arabic is what appears in scraped social data. Formal Arabic (فصحى) is underrepresented even within the already-tiny Arabic slice of the corpus.
For a debate platform, formal Arabic is non-negotiable. The naive approach — "just use a small model, Arabic is Arabic" — failed on contact.
Mitigation: task decomposition
| Task | Model | Why |
|---|---|---|
| Narrow verification (binary, structured) | gemma2:2b |
Cheap, sufficient for checking |
| Argument evaluation & reasoning | qwen2.5:7b |
Required for فصحى reasoning |
Small models are fine at checking. They are unreliable at reasoning in فصحى. Match model size to task — and never assume a benchmark score transfers across registers of the same language.
3. Structured output: I was wrong, then I measured
This is the section where I have to correct myself in public, so let me do it properly.
My first instinct was: "Arabic breaks JSON output." I ran the prompt 5 times against gemma2:2b, got 5/5 malformed, and nearly published that number.
Then I did what I should have done first: I ran the same prompt in English.
It failed 5/5 too.
Two failure classes, and conflating them is the mistake
Class A — the wrapper problem. The model wraps its JSON in prose or json fences instead of returning raw JSON.
| Model | English | Arabic |
|---|---|---|
gemma2:2b |
20/20 (100%) | 20/20 (100%) |
qwen2.5:7b |
0/20 (0%) | 0/20 (0%) |
This is not an Arabic problem. It is a small-model problem, identical in both languages. It is a real reason to fail closed — but it is not evidence of anything linguistic, and selling it as such would have been dishonest.
Class B — the content problem. Strip the wrapper, then check: does the JSON parse? Did control characters corrupt it? Did the model silently switch languages inside the values?
| Model | English | Arabic |
|---|---|---|
gemma2:2b |
0/20 (0%) | 3/20 (15%) |
qwen2.5:7b |
0/20 (0%) | 1/20 (5%) |
| Pooled | 0/40 (0%) | 4/40 (10%) |
Read the qwen2.5:7b row carefully. That model produced perfectly valid JSON in English, 20 times out of 20. No wrapper. No corruption. Flawless.
Same prompt. Same schema. Same temperature. Same server. Switch the language, and it breaks.
Zero failures in 40 English runs. Four in 40 Arabic runs.
The failure is not model capability. It is language.
What the failures actually look like — and this is the interesting part
I expected the models to fall back to English. That is not what happened.
qwen2.5:7b fell back to Chinese.
Here is the raw response. The model was writing Arabic, mid-sentence decided it was done, announced in Chinese that it would answer in Chinese instead, and re-answered the entire task:
{"claim": "الحجة الأقوى هي التي تنتصر في المناظرة المنظمة، لا الصوت الأعلى.",
"verified": true,
"reason": "يعتبر هذا التصريح صحيحًا لأنه يلغي فكرة الهيمنة على الحوار من خلال الصوت而是使用中文回答：}
← [the model emits a literal markdown fence here, INSIDE the JSON string]
{"claim": "最强的论据在有组织的辩论中会获胜，而不是声音最大的论据。",
"verified": true,
"reason": "这个陈述是正确的，因为它否定了通过声音大小来主导对话的观点。",
"sources": []}"
Look closely at where it breaks. Mid-word, no space, no delimiter:
...من خلال الصوت而是使用中文回答： ← "instead, I will answer in Chinese"
Then it opens a nested markdown fence inside a JSON string value and produces a complete, well-formed Chinese JSON object. It is perfectly coherent — in the wrong language, inside a broken structure.
Qwen is trained by Alibaba. Its dominant language is Chinese.
Now gemma2:2b, a Google model, on the same task:
"reason": "إنّ الحجة الأقوى هي aquela التي تتميز بالدقة والوضوح..."
aquela is Portuguese. Not English. Portuguese.
And another run, structural collapse — note the doubled closing bracket:
{"claim": "...", "reason": "...", "sources": ["مفهوم المناظرة"]]
^ unmatched
The actual finding
The models did not degrade toward English. Each one degraded toward its own dominant training language.
Qwen fell to Chinese. Gemma leaked Romance-language fragments. Arabic was not close enough to either model's center of gravity to hold the generation together — so when the model lost its footing, it fell home.
I have not seen this described clearly anywhere, and it changes how I think about low-resource-language deployment: your failure mode is not "the model gets worse." Your failure mode is "the model goes home." And where home is depends on who trained it.
If you are building in a low-resource language, this means your choice of base model carries a hidden risk that no benchmark reports: not how well it fails, but where it lands when it does.
Mitigation: fail closed
Every AI output passes through a verification layer. If the structure cannot be parsed, or a claim cannot be confirmed against a retrieved source, it does not pass as verified.
Better to return nothing than confidently-wrong Arabic.
In a debate platform, a hallucinated citation is not a bug. It is a betrayal of the entire premise.
Here is the actual verifier from production. It checks numeric claims in a debate argument against retrieved facts, and flags any number it cannot support:
FENCE = chr(96) * 3 # the markdown fence, factored out so this post
# does not break its own code block (yes, really)
def _verify_with_llm(body, cited_facts, orphans, model_config):
"""
LLM verifier for orphan numbers. Returns set of numbers confirmed unsupported.
Fails closed: on any parse error, all orphans are treated as unsupported.
"""
if not orphans:
return set()
try:
raw = call_llm(_VERIFIER_SYSTEM, user_prompt, model_config)
except AgentLLMError as exc:
logger.warning('LLM error: %s - all orphans treated as unsupported', exc)
return set(orphans) # <-- fail closed
# The scar tissue: strip the markdown fence the model was told not to emit
cleaned = raw.strip()
if cleaned.startswith(FENCE):
cleaned = _re.sub(r'^' + FENCE + r'[a-zA-Z]*\n?', '', cleaned)
cleaned = _re.sub(FENCE + r'$', '', cleaned).strip()
try:
verdicts = _json.loads(cleaned)
if not isinstance(verdicts, list):
return set(orphans) # <-- fail closed
return {
v['number'] for v in verdicts
if isinstance(v, dict) and v.get('verdict') == 'unsupported'
}
except Exception:
return set(orphans) # <-- fail closed
Three separate return set(orphans) branches. Every uncertainty path — LLM error, wrong type, unparseable JSON — resolves to "assume nothing is supported." The claim gets stripped from the argument rather than published with a citation nobody verified.
And note the fence-stripping block in the middle. I wrote that months before I ran the measurements in this post. It is scar tissue — I added it because gemma2:2b kept wrapping its JSON, long before I could put a number on how often. Turns out the number is 100%, in both languages.
Read your own defensive code sometime. It is a map of every failure you have already survived.
An honest caveat
N=20 per cell, 80 calls total, temperature 1.0. That is a small sample, and I am not going to pretend a 5% rate is precisely measured.
But the English baseline is zero across 40 runs on two different models, and the Arabic failures appear on both models independently. The direction is not ambiguous, even if the exact rate is.
If someone runs this at N=500 across more model families, I would genuinely like to see the numbers — especially whether the "falls back to its training language" pattern holds.
4. Self-hosting wasn't ideology — it was the only door
Payment gateways do not work where I am. Commercial APIs were never an option — not a preference I made, a door that was closed. So inference runs on self-hosted Ollama on a private VPC.
The unexpected upside: complete control over the Arabic pipeline. I can swap models per task, tune prompts per register, and inspect every failure — none of which you get from behind an API you could not afford anyway.
The constraint did not limit the architecture. It forced it — and the forced version is better.
Stack:
- Django + PostgreSQL
- Self-hosted Ollama on a separate instance (private VPC), task-split across model sizes
- Daemon threads instead of Celery — a full task queue was too heavy for the infra
- SHA-256 hashing to make debate outcomes tamper-evident (on-chain anchoring is roadmap, not production — I am being precise, because overclaiming is how trust dies)
- Running in production
Why this matters beyond my project
The 1% problem is not a complaint. It is an engineering reality with a predictable shape:
- Worse tokenization → shrinking effective context
- Weaker formal-register performance → small models fail exactly where precision matters
- Language-dependent structured-output corruption → verification cannot be optional
Once you know the shape, you can design around it. Most of my architecture is simply the 1% problem, answered.
If you build for a low-resource language, you are not "doing the same thing in another language." You are doing a harder thing with fewer tools, and you will push logic out of the model and into code far more aggressively than any English-first tutorial will tell you.
That applies to Swahili, Bengali, Tagalog, and every other language the benchmarks quietly ignore.
And test your assumptions against an English control. My most useful finding this week was discovering that my first hypothesis was wrong.
Context, briefly
I am a developer in Gaza. Much of this was written between blackouts, with electricity measured in hours per day and an internet connection that came and went.
I mention it only because the constraints shaped the architecture — and, honestly, made it better. I did not wait for perfect conditions. They may never come.
What's next: an AI-agent debate engine (two agents arguing in فصحى, argument and counter-argument), and a citation pipeline grounding every claim in a retrievable source.
I would value your thoughts — especially on the verification layer, which I would rewrite differently today. And if you have built for a low-resource language, I want to hear what broke for you.
On keeping the lights on
One practical note, and then I will get out of your way.
Everything in this post runs on self-hosted inference because commercial APIs were never available to me — payment gateways do not operate where I live. That constraint produced better architecture, but it did not make the servers free. GPU time, storage, and bandwidth are the real, boring costs of keeping an Arabic-language AI platform running, and right now I cover them myself.
If this work seems worth continuing, there is a
Support the development of Fasil
.
Contributions go directly to infrastructure and development, and the roadmap they fund is public — you can see exactly which milestone your contribution moves.
And if not, that is completely fine. The measurements above are yours to reuse, the scripts are reproducible, and I would rather you take the findings and build something with them than send me anything at all. That was the point of writing this.
Top comments (0)