DEV Community: islam al-nashera

Arabic-First AI: Why Dialects, Datasets, and Deployment Matter

islam al-nashera — Sat, 31 Oct 2026 13:39:32 +0000

Arabic-First AI: Why Dialects, Datasets, and Deployment Matter

Companion models: MasryGPT-Chat-1.5B · datasets · portfolio

Arabic is spoken by ~400 million people across 20+ countries, yet most "Arabic" AI is really Modern Standard Arabic (MSA) — the language of news, academia, and religion. Nobody texts in MSA. Egyptians say "ازيك؟", not "كيف حالك؟". Moroccans say "labas", not "كيفك". The gap between the Arabic that models are trained on and the Arabic that people actually speak is one of the largest unaddressed problems in applied NLP.

This essay argues that Arabic-first AI requires three things: dialect-aware models, dialect-rich datasets, and deployment paths that fit local infrastructure. I write from the perspective of a full-stack developer who has built and shipped both the models and the products around them.

1. The MSA Trap

Most Arabic LLMs (Qwen, Jais, AceGPT, ALLaM) are trained and evaluated on MSA-heavy corpora. The result:

Fluency mismatch. A model that scores well on MSA benchmarks sounds stiff and translated to a dialect speaker.
Cultural loss. Idioms ("عيش وملح"), humor, and register are not MSA. You cannot translate "يا باشا" into a formal register without losing the warmth.
User trust. A chatbot that replies in MSA to an Egyptian user feels like talking to a bureaucrat, not a person.

In my own MasryGPT evaluation, base Qwen scored 4.7/5 MSA fluency but only 3.1/5 Egyptian fluency. After one epoch of dialect fine-tuning, Egyptian fluency rose to 4.6/5 — at the cost of a small MSA regression (−10%). That tradeoff is real and worth naming honestly.

2. The Dataset Problem

Dialect data is scarce, noisy, and legally messy:

Volume. MSA corpora (OSCAR, Arabic Wikipedia, news) are large. Dialect corpora are tiny and fragmented.
Quality. Dialect text on social media is full of typos, code-switching, and transliteration ("3ajibni" instead of "عجبني"). Cleaning it is non-trivial.
Provenance. Real chat logs (WhatsApp, Facebook) are the gold standard but require anonymization and documented consent. Synthetic generation (lexicon-based) is safer but risks producing "textbook dialect" — a dialect that no one actually speaks.
Coverage. Egypt's Cairo dialect dominates; Upper Egypt (Saidi), Levantine, Gulf, and Maghrebi dialects are even more underrepresented.

What I built to address this:

Dataset	Purpose
`arab-dialects-20-countries-3m`	Broad dialect coverage across 20 Arab countries
`arabic-history-and-dialects`	Historical + dialectal Arabic
`documents-Egyptian-Arabic`	Egyptian-Arabic document text
`arabic-to-code-8-langs-3m`	Arabic instructions → code in 8 languages

The 3M-row dialect dataset is the backbone; the MasryGPT training set (80k) is a curated subset focused on Cairo speech.

3. Dialect-Aware Modeling: Lessons from MasryGPT

Key takeaways from fine-tuning Qwen2.5-1.5B-Instruct on 80k Egyptian examples:

Small models can learn dialect. A 1.5B model reached 4.6/5 human fluency in one epoch. You do not need 70B to sound Egyptian.
QLoRA is enough. 18.5M trainable parameters (1.18%) on 4-bit base, trained in 2h13m on free T4s. The barrier to entry is low.
Loss plateaus tell the truth. Loss flattened at step 1,500 → the dataset was saturated. More epochs would have memorized, not generalized. This is the most important diagnostic I check.
Idioms are the hard part. Overall fluency was high, but idiom correctness ("عيش وملح") stayed at 40%. Idioms need dedicated, high-quality data — roughly 5k pairs — not more volume.
Tradeoffs must be reported. MSA fluency dropped 10%. Hiding this would be dishonest and would break downstream users who need both.

4. The Arabic-to-Code Frontier

A second, less discussed problem: Arabic as an instruction language for code generation. Most code LLMs are trained on English prompts. An Arabic-speaking developer who thinks in Arabic must mentally translate before prompting — friction that slows adoption.

My arabic-to-code-8-langs-3m dataset targets this directly: Arabic instructions mapped to code in 8 programming languages. Combined with a dialect model, this enables a fully Arabic developer experience: think in Masry, get code.

5. Deployment: Local Realities

Arabic-first AI must run where Arabic speakers are:

GPU scarcity. Many Egyptian teams have no A100s. A 1.5B model in 4-bit (~1.2 GB VRAM) runs on a T4, a laptop, or even a phone via GGUF/Ollama.
Latency & cost. Self-hosting a small model beats paying per-token API fees for high-volume Arabic chatbots.
Offline & privacy. Government, banking, and healthcare use cases in the region often require on-prem deployment. Small models make this feasible.
Integration. As a full-stack developer, I wire models into real products: Next.js frontends, Prisma/Postgres backends, LangChain for RAG, vector search for grounding. The model is only 20% of the work; the other 80% is product engineering.

6. What's Missing (Honest Gaps)

Evaluation standards. There is no widely accepted Arabic-dialect benchmark. My "4.6/5" is a 3-speaker internal review, not an independent benchmark. The field needs shared eval sets.
Reproducibility. I have not yet published the 80k training JSONL. Until the dataset, generation script, and filtering rules are released, the numbers are an internal pilot.
Dialect breadth. Cairo dominates my data. Saidi, Levantine, Gulf, and Maghrebi need the same treatment.
Safety & bias. Dialect models can amplify regional stereotypes and vulgarity. Filters and red-teaming are underdeveloped for Arabic.

7. A Call to Action

If you are building Arabic AI:

Report dialect fluency separately from MSA fluency. Do not let one number hide the other.
Publish your datasets (or at least their generation recipe). Reproducibility is the bottleneck.
Start small. A well-fine-tuned 1.5B beats a poorly-prompted 70B for dialect tasks, and it deploys anywhere.
Build the product, not just the model. RAG, validation, UX, and monitoring matter more than the checkpoint.

Arabic-first AI is not a translation problem. It is a culture problem, and culture lives in dialects. The models that win will be the ones that sound like the people they serve.

Author: Islam Al-Nashar — Full-Stack Developer & AI builder. GitHub · Hugging Face · Portfolio

Glitch-Img-1.0: RGBA Image Generation & Editing with Qwen-Image-2.1

islam al-nashera — Sat, 31 Oct 2026 13:39:27 +0000

Glitch-Img-1.0: RGBA Image Generation & Editing with Qwen-Image-2.1

Try it: huggingface.co/ISLAM-PO/Glitch-Img-1.0 · image-gen demo Space

Image generation has moved past "pretty pictures" toward controllable, editable, transparent assets. Glitch-Img-1.0 is my re-upload of Qwen/Qwen-Image-2.1 under the name ISLAM-PO/Glitch-Img-1.0, aimed at making a strong unified text-to-image + editing model easy to discover and use from a single, well-documented repository. All weights and configs are unchanged from the original; only the README and repo IDs were updated, and the model is Built with Qwen under the Qwen Research License (non-commercial).

This report covers what the model does, its key capabilities, a quick-start, memory tips, and licensing notes.

1. What It Is

Base model: Qwen/Qwen-Image-2.1
Visual generation component: ~7B parameters (32 Single-Stream DiT layers)
Pipeline tag: text-to-image (diffusers: QwenImage21Pipeline)
License: Qwen Research License Agreement (non-commercial)
Modifications in this re-upload: README renaming, repo-ID updates, attribution notes. Weights/configs unchanged.

Four headline improvements define the Qwen-Image-2.1 family this build inherits:

Compact & efficient — mixed-granularity attention + prefix KV-cache reuse for strong quality at low cost.
Native transparency (RGBA) — generate regular or transparent images, edit transparent layers, extract subjects from photos — all in one model.
Versatile editing — up to 10 reference images, local edits via circles/painted annotations/masks, identity preservation for people and products.
Realistic textures & refined aesthetics — improved typography, portrait lighting, and fine details.

2. Capabilities in Practice

Capability	Details
Text-to-image	Prompts up to 2048×2048 and beyond; strong text rendering
Image editing	Inpainting-style edits, background swaps, style transfer
RGBA transparency	Native alpha channel generation with a documented prompt format
Multi-reference	Up to 10 reference images for identity/subject consistency
Local edits	Circles, painted annotations, separate masks
Aspect ratios	1:1, 4:3, 3:4, 3:2, 2:3, 16:9, 9:16 presets

3. Quick Start

Install:

pip install torch>=2.4.0
pip install transformers>=5.17
pip install git+https://github.com/huggingface/diffusers
pip install accelerate pillow

Text-to-image:

import torch
from diffusers import QwenImage21Pipeline

pipe = QwenImage21Pipeline.from_pretrained(
    "ISLAM-PO/Glitch-Img-1.0", torch_dtype=torch.bfloat16
).to("cuda")

image = pipe(
    prompt='A neon shop sign that reads "GLITCH IMG 1.0", rainy night, reflections on wet pavement',
    width=2048, height=2048,
    num_inference_steps=40,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]

image.save("t2i_example.png")

Image editing:

import torch
from PIL import Image
from diffusers import QwenImage21Pipeline

pipe = QwenImage21Pipeline.from_pretrained(
    "ISLAM-PO/Glitch-Img-1.0", torch_dtype=torch.bfloat16
).to("cuda")

input_image = Image.open("input.png")
image = pipe(
    prompt="Change the background to a sunset beach",
    image=input_image,
    num_inference_steps=40,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]
image.save("edit_example.png")

Transparent (RGBA) generation — use the recommended prompt format:

image = pipe(
    prompt="This is an RGBA image with transparency. A cute cartoon dragon sticker. "
           "The image has alpha channel and the background is transparent.",
    width=2048, height=2048,
    num_inference_steps=40,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]
image.save("transparent_example.png")

Aspect ratios:

aspect_ratios = {
    "1:1":  (2048, 2048), "4:3":  (2400, 1792), "3:4":  (1792, 2400),
    "3:2":  (2528, 1696), "2:3":  (1696, 2528), "16:9": (2752, 1536),
    "9:16": (1536, 2752),
}

4. Memory Optimization

pipe = QwenImage21Pipeline.from_pretrained(
    "ISLAM-PO/Glitch-Img-1.0", torch_dtype=torch.bfloat16
)
pipe.enable_model_cpu_offload()

This offloads model parts to CPU between steps, letting you run on smaller GPUs at the cost of speed.

5. Where It Fits in My Stack

I use Glitch-Img as the visual layer behind my portfolio and product work:

Generating marketing assets and transparent stickers/logos for client sites.
Editing product photos (background swap, identity-preserving edits).
Producing RGBA assets that drop straight into web UIs without manual masking.

It complements MasryGPT (text) as the two halves of an Arabic-first AI product stack: one generates the words, the other the visuals.

6. Licensing & Attribution

Licensed under the Qwen Research License Agreement — non-commercial use only.
Original work © 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd.
This re-upload is Built with Qwen and based on Qwen/Qwen-Image-2.1.
For commercial use, request a license at model-business@notice.qwencloud.com.
See NOTICE and LICENSE in the repo for required attribution.

7. Links

Model: huggingface.co/ISLAM-PO/Glitch-Img-1.0
Demo Space: huggingface.co/spaces/ISLAM-PO/image-gen-demo
Original: huggingface.co/Qwen/Qwen-Image-2.1
Upstream GitHub: QwenLM/Qwen-Image-2.1

Author: Islam Al-Nashar — Full-Stack Developer & AI builder. GitHub · Hugging Face · Portfolio

MasryGPT-Chat-1.5B: Teaching Qwen2.5 to Speak Egyptian Arabic (Masry)

islam al-nashera — Sat, 31 Oct 2026 13:27:58 +0000

MasryGPT-Chat-1.5B: Teaching Qwen2.5 to Speak Egyptian Arabic (Masry)

Try it: huggingface.co/ISLAM-PO/MasryGPT-Chat-1.5B · Gradio demo

Most Arabic LLMs speak Modern Standard Arabic (MSA) — the language of news and textbooks. But 105 million Egyptians speak Masry (العامية المصرية) every day: "ازيك يا باشا؟" not "كيف حالك؟". That gap is cultural, not just linguistic, and it is the reason MasryGPT exists.

MasryGPT-Chat-1.5B is an open-source, instruction-tuned chat model built on Qwen2.5-1.5B-Instruct, fine-tuned with QLoRA + Unsloth on 80,000 Egyptian-Arabic examples. It is small enough to run on a single T4 GPU (~1.2 GB in 4-bit) or even edge devices, and it is released under Apache 2.0.

This article is a technical deep-dive: motivation, model details, dataset, training pipeline, loss analysis, evaluation, usage, limitations, and roadmap.

1. Motivation

MSA vs. Dialect gap. Existing Arabic LLMs (Qwen, Jais, AceGPT) optimize for MSA. Egyptian users get stiff, translated-sounding replies. "ازيك" vs "كيف حالك" is the difference between a friend and a textbook.
Lightweight deployment. 7B models need 14 GB+ VRAM. A 1.5B model runs on mobile, T4, or edge — perfect for Egyptian startups with limited GPU budgets.
Unsloth efficiency. 2× faster fine-tuning on a free Kaggle T4×2, no A100 required.

2. Model Details

Attribute	Value
Model ID	`ISLAM-PO/MasryGPT-Chat-1.5B`
Base model	`Qwen/Qwen2.5-1.5B-Instruct`
Architecture	`Qwen2ForCausalLM` (28 layers, 12 attn heads, 2 KV heads)
Hidden / intermediate size	1536 / 8960
Vocab size	151,936
Max position embeddings	32,768 (trained on 2,048)
Total parameters	1,562,179,072
Trainable (LoRA)	18,464,768 (1.18%)
LoRA config	`r=16, alpha=16, dropout=0, bias=none, targets=[q,k,v,o,gate,up,down]_proj`
Precision	Trained 4-bit (NF4), released 16-bit (bfloat16), ~2.9 GB
Tokenizer	Qwen2 tokenizer + `chat_template.jinja` (qwen-2.5)
License	Apache 2.0 (Qwen license applies)

3. Dataset

qwen_egyptian_80k_terms.jsonl — 80,000 examples.

Source & curation: synthetic generation from an Egyptian lexicon + manual vulgarity filtering.
Format: {"messages": [{"role":"user","content":"..."},{"role":"assistant","content":"مصري..."}]}
Preprocessing: get_chat_template(tokenizer, chat_template="qwen-2.5") + tokenizer.apply_chat_template(..., tokenize=False) → packed text field.

Distribution:

Category	Share
Greetings & daily chat	35%
Customer service & instructions	20%
Humor & jokes	15%
Expressions & idioms	15%
Culture & food	15%

Example:

{"messages": [{"role": "user", "content": "يعني ايه عيش وملح؟"},
              {"role": "assistant", "content": "عيش وملح يعني العشرة والعهد بين الصحاب، اللي كل معاك عيش وملح ميخونكش"}]}

4. Training Pipeline & Deep Loss Analysis

Pipeline: HF Hub → FastLanguageModel (4-bit) → get_peft_model (QLoRA) → SFTTrainer (TRL) → save_pretrained_merged (16-bit) → HF Hub

Hyperparameters (final fast config, Kaggle T4×2):

Param	Value	Rationale
`max_seq_length`	2048	Captures longer Egyptian rants without OOM
`per_device_train_batch_size`	32	Maximized for T4 15 GB
`gradient_accumulation_steps`	1	Effective batch 32 → 80k/32 = 2,500 steps
`learning_rate`	2e-4	QLoRA standard for 1.5B, cosine decay
`optim`	`adamw_8bit`	GPU-resident (vs paged → RAM)
`weight_decay`	0.01	Regularization against overfitting on 80k
`warmup_steps`	20	0.8% of steps
`lr_scheduler`	cosine	Best for 1 epoch
`packing`	True	10× sample/sec on short sentences
`num_train_epochs`	1	2nd epoch risks memorization

Loss curve (2,500 steps, 2h 13m):

Step 10:    0.091025
Step 100:   0.098729  (initial spike — warmup)
Step 500:   0.082923
Step 1000:  0.081538
Step 1500:  0.077019
Step 2000:  0.074811
Step 2500:  0.074394
Avg: 0.07995   Perplexity exp(0.079) ≈ 1.083

Convergence: smooth 18.7% drop, no divergence. Loss plateaus at 0.074–0.077 after step 1,500 → dataset saturated. More epochs would overfit.
Throughput: 10.002 samples/sec, 0.313 steps/sec (Unsloth 2× vs vanilla).
GPU: 2.4 GB / 15 GB (GPU0, 93%) + 4.1 GB / 15 GB (GPU1, 51%). Low memory is a feature of 4-bit + Unsloth, not a bug. CPU at 100% was the tokenization bottleneck.
FLOPs: 1.05e17 total ≈ 13.1 TFLOPs/s on T4.

Why not 10 GB VRAM? A 1.5B 4-bit model weighs ~0.8 GB; even batch 32 adds ~1.5 GB activations. To reach 10 GB you must disable 4-bit (full 16-bit = 3 GB base + 6 GB optimizer) — 3× slower and unnecessary. The current 2–4 GB is optimal.

5. Evaluation & Benchmarks

Status note: these figures are an internal pilot, not an independently reproduced benchmark. See EVALUATION.md in the repo for protocol and limitations.

Quantitative:

Benchmark	MasryGPT Chat	Qwen2.5-1.5B Base	Gain
Train loss	0.074	0.115 (zero-shot)	−35%
Perplexity (5% test split)	1.08	1.22	−11%
Egyptian fluency (human, 100 prompts, 1–5)	4.6	3.1	+48%
MSA fluency	4.2	4.7	−10% (tradeoff)
Idiom correctness ("عيش وملح")	40%	20%	+100%, still weak

ARC-Challenge (via lm_eval, vLLM on H100, greedy, temp 0): 47.7% acc_norm (43.69% acc) on 1,172 tasks.

Qualitative:

✅ User: ازيك؟ → ازيك يا باشا عامل ايه النهاردة؟ (natural)
❌ User: يعني ايه عيش وملح؟ → عيش وملس = خبز (FAIL — needs more idiom data)
✅ User: احكيلي نكتة → short joke (needs max_new_tokens=256, temperature=0.8)

Base Qwen answers in MSA ("كيف حالك؟"); MasryGPT answers "ازيك يا باشا؟" — judged 48% more Egyptian by 3 native speakers.

6. Usage

Unsloth (recommended, fast):

from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="ISLAM-PO/MasryGPT-Chat-1.5B",
    max_seq_length=2048, dtype=None, load_in_4bit=True,
)
FastLanguageModel.for_inference(model)
messages = [{"role": "user", "content": "ازيك يا باشا؟"}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_tensors="pt", return_dict=True).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.8,
                        top_p=0.95, do_sample=True, repetition_penalty=1.1)
print(tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:],
      skip_special_tokens=True)[0])

Transformers (standard):

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
    "ISLAM-PO/MasryGPT-Chat-1.5B", device_map="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("ISLAM-PO/MasryGPT-Chat-1.5B")

Generation tips: temperature 0.7–0.8 for creativity, 0.3 for facts; always return_dict=True to fix the attention_mask warning; add fix_mistral_regex=True if you see that warning.

7. Limitations & Error Analysis

Idioms still weak (40%) — dataset only 15% idioms; needs ~5k real idiom pairs.
Hallucination — may invent Egyptian facts; no RAG.
Short responses — trained on packed short text; raise max_new_tokens and use repetition_penalty.
MSA degradation (−10%) — catastrophic forgetting from dialect fine-tuning.
No safety filter beyond base Qwen.

Failure-mode analysis: loss plateau at step 1,500 → dataset saturated. Fix: add 40k diverse real conversations + 1 more epoch at LR 1e-4.

8. Roadmap

[ ] v1.1: +5k idioms + 10k real chat logs → target 90% idiom accuracy
[ ] v2.0: DPO/RLHF for Egyptian humor alignment
[ ] v2.5: 3B version for better reasoning, keep 1.5B "lite"
[ ] Deployment: GGUF + Ollama + ONNX

9. Citation

@misc{masrygpt_chat2026,
  author = {ISLAM-PO},
  title  = {MasryGPT Chat: Egyptian Dialect Qwen2.5-1.5B-Instruct via QLoRA & Unsloth},
  year   = {2026},
  publisher = {Hugging Face},
  url    = {https://huggingface.co/ISLAM-PO/MasryGPT-Chat-1.5B},
  note   = {80k terms, 2500 steps, loss 0.079}
}

Contribute: found a wrong Masry word? Open an issue with prompt + expected answer. Have anonymized Egyptian chat data (with documented permission)? Propose a separate dataset repo after privacy/provenance review.

Author: Islam Al-Nashar — Full-Stack Developer & AI builder. GitHub · Hugging Face · Portfolio