DEV Community

Abdelrhafar Naouri
Abdelrhafar Naouri

Posted on AI-assisted

🇲🇦 Building SILMA-Darija: Open LLMs for Moroccan Arabic & Arabizi

#ai #llm #nlp #opensource

🇲🇦 Building SILMA-Darija: Open LLMs for Moroccan Arabic & Arabizi

Most Arabic models struggle with Moroccan Darija. When you talk to them in Darija, they usually switch back to Gulf Arabic or formal Modern Standard Arabic (MSA), and they almost always break when people write in Arabizi using numbers (3, 7, 9).

To fix this, I fine-tuned two open models on real Moroccan conversations and cultural data:

SILMA-9B-Darija: For complex tasks, history, and natural back-and-forth conversation. Runs in ~6 GB VRAM with 4-bit quantization.
SILMA-2B-Darija: A lightweight version that runs fast on consumer laptops and phones (< 3 GB VRAM).
Moroccan Darija Dataset: 46,000+ cleaned Darija instruction and conversation pairs.
Enter fullscreen mode Exit fullscreen mode

Benchmark Comparison

I tested these models alongside existing Moroccan models on a 100-question test set covering local history, proverbs, and everyday code-switching:

        SILMA-9B-Darija (Ours)
        Gemma-2 9B
        2.94%
        24.8
        Full
        ~6 GB (4-bit)


        GemMaroc-Qwen2.5-7B
        Qwen-2.5 7B
        2.51%
        18.4
        Partial
        ~15 GB


        MoroccanDarija-Llama-3.1-8B
        LLaMA-3.1 8B
        2.38%
        17.5
        Poor
        ~16 GB


        SILMA-2B-Darija (Ours)
        Gemma-2 2B
        2.77%
        16.2
        Full
        < 3 GB


        Base SILMA-9B (Untuned)
        Gemma-2 9B
        1.58%
        11.2
        None (Defaults to MSA)
        ~6 GB
Enter fullscreen mode Exit fullscreen mode

What Works Better

Natural Vocabulary: It sticks to actual Darija words (*"كيداير"*, *"واخا"*, *"بزاف"*, *"ديال"*) instead of mixing in Egyptian or Gulf phrases.
Handles Arabizi: You can prompt it in Arabic script or in Latin letters (salam, kif dayer labas 3lik?) and it responds properly.
Local Context: Better recall on Moroccan history (Almoravids, Almohads, Saadians) and common proverbs without hallucinating.
Enter fullscreen mode Exit fullscreen mode

How to Run It

You can load and test the 9B model with Transformers and PEFT on a standard GPU (T4 / RTX 3060):

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "SILMA-AI/SILMA-9B-Instruct-v0.1"
adapter_id = "abdnaouri/silma-darija-9b-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

messages = [{"role": "user", "content": "اشرح ليا كيفاش كتخدم الـ Blockchain بالدارجة؟"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Links

9B Model: huggingface.co/abdnaouri/silma-darija-9b-lora
2B Model: huggingface.co/abdnaouri/silma-2b-darija-lora
Training Data: huggingface.co/datasets/abdnaouri/moroccan-darija-llm-dataset
Enter fullscreen mode Exit fullscreen mode

Feel free to try them out, run your own benchmarks, and share feedback.

Top comments (0)