Darija Buddy: A Local Gemma-Powered Tutor for a Friend

Soufiane Zaari — Sat, 03 Oct 2026 01:02:48 +0000

The Friend & The Problem

Moving to Morocco as an international student comes with a unique linguistic hurdle: Moroccan Darija. While formal Arabic (MSA) or French might get you through official paperwork, everyday social life happens in Darija.

My university classmate, Alex, has been struggling to fit into group banter and daily conversations. He understands some basics, but when people talk fast or switch between Arabizi (Latin-script Darija using numbers like 3, 7, 9) and spoken phrases, he freezes.

Traditional language apps don't support Moroccan Darija well—they usually default to Modern Standard Arabic, which locals rarely speak on the street. Human tutoring is expensive, and practicing with friends can feel intimidating when you're afraid of making mistakes.

He needed a safe, patient, and conversational practice partner that speaks genuine Darija, understands phonetically typed Arabizi, and explains nuances in French.

What I Built: Darija Buddy 🇲🇦

Darija Buddy is a lightweight, local conversational tutor designed to help non-Moroccan beginners practice real-life Darija speech.

Key Capabilities:

Speaks authentic Darija: It avoids rigid Modern Standard Arabic and replies in natural Moroccan expressions.
Multilingual input handling: It comprehends Latin-script Darija (Arabizi), French, and standard Arabic.
Bilingual feedback loop: Whenever it introduces colloquial vocabulary or idioms, it appends concise pedagogical explanations in French.
Gentle error correction: If the learner makes a grammar or lexical slip, Darija Buddy reformulates the phrase constructively before keeping the conversation flowing.

Technical Architecture & How It Works

1. The Core Model: Google Gemma 3 (4B)

I selected Gemma 3 (4B) as the reasoning engine. For a compact 4B-parameter open-weight model, Gemma 3 showcases remarkable multilingual understanding, easily deciphering transliterated Arabizi and grasping cultural Moroccan idioms.

2. Local Inference with Ollama

Instead of relying on remote APIs with variable latency and pay-per-token pricing, the model runs via Ollama. The quantized 4B weights run smoothly on consumer hardware (CPU-only), keeping RAM consumption under ~4 GB.

3. Gradio Interface

The frontend is encapsulated in a single Python script using gradio.ChatInterface, making it instantly accessible in any local browser window at http://127.0.0.1:7860.

4. Pedagogical System Prompt Engineering

The behavior is governed by a system prompt (full prompt in darija_buddy.py on GitHub) enforcing: authentic Darija instead of MSA, short French explanations, Arabizi/French/English input handling, gentle error correction, and short conversational replies.

Demo & Interaction

Here is a practice session where the user initiates in Arabizi, receives natural conversational responses, and gets immediate French vocabulary breakdowns:

Why Open Innovation Matters for This Project
This project highlights why open-weight models and local inference triumph over proprietary closed APIs:

Zero Operational Cost: Language practice requires repetitive, daily micro-conversations. Running Gemma 3 locally means zero token costs, no API credits expiring, and no credit card requirements for students.
Total Privacy for the Learner: Practicing a new language involves vulnerability and personal conversations. All inference happens in-memory on the laptop; no conversation logs or personal data are ever uploaded to remote commercial servers.
Offline Reliability: University Wi-Fi and mobile data can be unpredictable. Darija Buddy works entirely on a plane, on a train, or in a cafe without an active internet connection.
Customizability: With open weights, I'm not locked into proprietary censorship or forced model upgrades. I can easily fine-tune Gemma on dialectal corpora or swap weights as newer open models release.
What My Friend Said
When I showed it to Alex on a video call:

"C'est magnifique... et ça va beaucoup m'aider à apprendre le Darija."

Prize Categories

Best Use of Gemma

Code Repository
The code is completely open-source and easy to reproduce:

GitHub Repository: https://github.com/SoufianeZaari/darija-buddy

DarijaBench: Do AI Models Actually Understand Moroccan Darija?

Soufiane Zaari — Thu, 01 Oct 2026 16:39:56 +0000

Update (Oct 2): Following reviewer feedback, the 0.95 vs 0.90 gap is not statistically significant (exact McNemar p = 0.25) — the honest reading is a four-way tie at n = 60.

DarijaBench: Do AI Models Actually Understand Moroccan Darija?
As a Moroccan student, I use AI assistants every day. They are brilliant in English and French — but I kept noticing something: ask them something in Darija (Moroccan Arabic dialect, spoken by 35+ million people), and the confident answers start wobbling. So I decided to stop guessing and start measuring. I built DarijaBench, a 60-item benchmark that tests whether today's frontier models truly understand Moroccan Darija — and ran it on four of them.
🔗 Benchmark: https://www.kaggle.com/benchmarks/soufianzaari/darijabench
What DarijaBench tests
Darija is a low-resource dialect: it is barely present in training data compared to English, French, or even Modern Standard Arabic. DarijaBench probes three practical skills, 20 items each:
Darija → French translation — can the model convert everyday Moroccan sentences into French?
Sentiment analysis — can it tell whether a Darija text is positive, negative, or neutral?
Moroccan cultural knowledge — can it answer questions asked in Darija about Morocco (the capital, couscous ingredients, the 2030 World Cup hosts...)?
Each item is graded automatically with keyword checks, and the final score is the average of the three categories. The whole thing was built with the Kaggle Benchmarks SDK, entirely on the free quota.
The results
Model
DarijaBench score
Gemini 3.7 Flash
0.95
Claude Sonnet 5
0.95
Claude Opus 4.7
0.95
GPT-5.6 Luna
0.90
What I learned

The top models have mostly cracked basic Darija. A three-way tie at 0.95 was not what I expected — I assumed Darija would still be a weak spot. Frontier labs are clearly doing better on dialectal Arabic than their reputation suggests.
Sentiment is the easiest skill. Gemini scored a perfect 20/20 on sentiment. Emotional tone ("هاد المطعم رائع!" vs "الخدمة خايبة") seems to transfer across dialects and languages with little friction.
Translation is the hardest. At 85%, Darija→French was the lowest category for Gemini. Darija is full of French loanwords used differently than in French, plus idioms with no literal equivalent — exactly where word-level understanding breaks down.
Small gaps need bigger samples. GPT-5.6 Luna's 0.90 vs 0.95 is ~3 more failures on 60 items — but a paired test shows this isn't statistically significant (p = 0.25). Detecting a 5-point gap reliably needs ~200–300 items. That's the main lesson for v2.
Honest caveats. Keyword-based grading is brittle — a correct answer phrased unexpectedly can fail. Sixty items is small. And the three-way tie tells me the benchmark is currently too easy to separate the best models. The next version needs harder items: idioms, heavy code-switching (Darija/French/Spanish mixes are everywhere in the north), and regional variants. Try it yourself The benchmark is public — run more models on it, or fork the task and make it harder. If you speak a low-resource dialect, consider building your own version: the Kaggle Benchmarks SDK makes it surprisingly painless, and measuring is always better than vibing.
Built for the DEV x Kaggle Benchmarking Challenge.

DEV Community: Soufiane Zaari