DEV Community: Muhammad Zubair

How I Fine-Tuned a 7B LLM with LoRA and Unsloth

Muhammad Zubair — Mon, 14 Sep 2026 09:30:55 +0000

How I Fine-Tuned a 7B LLM with LoRA and Unsloth

When I was working on Khatta-ka-LLM, two technologies became especially important:

LoRA + Unsloth

If you've heard these names but aren't sure what they actually do, here's a simple explanation.

What is LoRA?

LoRA stands for Low-Rank Adaptation.

Instead of updating all the parameters of a large language model during fine-tuning, LoRA keeps the original model weights frozen and trains a much smaller set of additional parameters.

Think of it like this:

Full fine-tuning

Entire model
↓
Update billions of parameters

LoRA

Original model
↓
Keep the original weights
↓
Train small adapter matrices

The result is a much more parameter-efficient fine-tuning process.

This is especially useful when you don't have access to expensive GPU clusters.

Where does Unsloth come in?

Unsloth is a framework designed to make LLM fine-tuning more efficient.

It focuses on reducing memory usage and improving training speed, making techniques such as LoRA and QLoRA much more practical.

For my Khattak Pashto experiment, my workflow was essentially:

Qwen2.5-7B
↓
LoRA fine-tuning
↓
Unsloth optimization
↓
Custom Khattak Pashto dataset
↓
Fine-tuned model

The goal wasn't to teach the model language from scratch.

The base model already had broad language knowledge.

Instead, I wanted to adapt that existing knowledge toward a specific regional dialect by exposing it to carefully prepared examples.

What happened during training?

For my experiment, the training loss went from:

3.44 → 0.22

That doesn't mean the model became perfect.

Training loss alone isn't enough to prove that a model performs well in real-world usage. Proper evaluation and testing are still important.

But it was a great demonstration of what parameter-efficient fine-tuning could achieve for this experiment.

Why LoRA is interesting

The important idea behind LoRA is that you don't necessarily need to modify every parameter of a large model to adapt it to a new task or domain.

You can keep the original model and learn a relatively small set of trainable parameters.

That makes experimentation significantly more accessible.

For example, instead of thinking:

«"I need a massive GPU cluster to fine-tune an LLM."»

You can start thinking:

«"What is the smallest efficient fine-tuning approach that can solve my problem?"»

That's a very different way of approaching LLM engineering.

The bigger lesson

You don't always need massive infrastructure to experiment with LLMs.

Understanding concepts such as:

Parameter-Efficient Fine-Tuning (PEFT)
LoRA
QLoRA
Quantization
Efficient training frameworks

can make LLM experimentation much more accessible.

That's one of the things I enjoyed most about building Khatta-ka-LLM.

The process was simple:

Build
↓
Experiment
↓
Measure
↓
Improve

If you're learning LLM fine-tuning, I'd recommend starting by understanding what LoRA is actually changing inside the model.

Once that clicks, the rest of the workflow becomes much easier to understand.
Want to see the full project?

I documented the complete Khatta-ka-LLM experiment, including the dataset, training approach, model architecture, and results, in a detailed case study on my website.

Read the full case study:
https://www.mzubair.online/blog/khattak-llm-case-study

What's next?

I can break down either:

LoRA vs QLoRA

How LoRA works under the hood mathematically.

Which one would you like to see next?

How I Built an Edge-to-Cloud AI Assistant for the Blind (Kotlin + Vertex AI)

Muhammad Zubair — Thu, 10 Sep 2026 11:50:35 +0000

Navigating complex physical environments is a daily challenge for the visually impaired. While standard GPS apps help with macro-navigation, they fail to detect immediate physical obstacles, read environmental text, or recognize faces in real-time.

To bridge this gap, I engineered binAI—a cutting-edge, voice-controlled Android application that acts as a real-time mobility instructor and visual assistant for the blind.

By combining lightning-fast on-device sensors with powerful Google Vertex AI cloud models, binAI keeps users safe and aware of their surroundings with zero latency.

(Note: You can read the original architectural breakdown on my Software Engineering Portfolio).

🛠️ The Tech Stack: Android, FastAPI, and Vertex AI

Building a real-time mobility radar requires a highly optimized, concurrent architecture. I designed binAI using a hybrid Edge-to-Cloud approach to ensure maximum speed and reliability.


| Layer               | Technology Used               | Purpose                                               |
| :------------------ | :---------------------------- | :---------------------------------------------------- |
| **Frontend (Edge)** | Kotlin, Jetpack Compose       | Native Android UI and hardware integration            |
| **On-Device AI**    | Google ML Kit, CameraX        | Zero-latency object, face, and text detection         |
| **Backend (Cloud)** | Python 3, FastAPI             | High-performance API routing and data processing      |
| **Cloud AI**        | Google Vertex AI (Gemini 2.5) | Spatial reasoning and complex scene analysis          |
| **Infrastructure**  | Docker, Google Cloud Run      | Auto-scaling (scales to 0) with CI/CD via Cloud Build |

⚡ The Hybrid Edge-to-Cloud Pipeline

To provide the fastest and safest experience, an AI assistant for the blind cannot rely solely on the cloud (which introduces latency) or solely on the phone (which lacks deep reasoning). Here is how the hybrid architecture solves this:

1. The Edge (Phone): The Android app uses Kotlin Coroutines to process camera frames locally at high speeds, detecting bounding boxes, calculating obstacle distances in feet, and finding faces.

2. The Cloud (Server): The app injects this mathematical sensor data into a prompt alongside a compressed image and sends it to the FastAPI backend.

3. The Brain (Vertex AI): The Gemini model reads the image and the exact sensor data to generate highly accurate, spatial, and urgent instructions (e.g., "STOP! Stairs going down").

💻 Code Spotlight: Solving Edge-Case Complexities

1. Fixing Mobile Camera Rotation for Face Recognition

A major issue with Android cameras is that they often send frames rotated by 90 degrees, causing standard face recognition libraries to fail on the backend. I engineered a fallback mechanism using NumPy to rotate the image matrix and fix the underlying C++ memory contiguity before re-scanning.


# 1. Look for faces in the standard orientation
face_locations = face_recognition.face_locations(img)

# 2. THE ROTATION FIX (With C++ Memory Fix)
if not face_locations:
    # Rotate -90 degrees and fix memory layout for the C++ backend
    img_rotated = np.ascontiguousarray(np.rot90(img, k=-1))
    face_locations = face_recognition.face_locations(img_rotated)

    if face_locations:
        img = img_rotated
    else:
        # Fallback: Rotate +90 degrees
        img_rotated = np.ascontiguousarray(np.rot90(img, k=1))
        face_locations = face_recognition.face_locations(img_rotated)
        if face_locations:
            img = img_rotated

2. Sensor Data Injection into Vertex AI

To prevent the LLM from hallucinating distances, the backend dynamically injects the exact mathematical data calculated by the phone's edge sensors directly into the system prompt.


@app.post("/navigate")
async def navigate(image: UploadFile = File(...), on_device_data: Optional[str] = Form(None)):

    # Injecting Edge ML data into the Cloud LLM prompt
    sensor_injection = f"ON-DEVICE SENSOR DATA: {on_device_data}. Use these exact distances for your evasion commands." if on_device_data else ""

    prompt = f"""
    You are a real-time mobility radar for a blind person walking forward.
    {sensor_injection}

    CRITICAL CONSTRAINTS:
    - Do NOT use full sentences.
    - Do NOT be polite.
    - Prioritize distance (feet/steps) and directional commands (left/right).
    """

    result = call_vision_model(prompt, await image.read())
    return {"status": "success", "script": result}

💡 The Impact of AI for the Visually Impaired

binAI demonstrates how modern AI infrastructure—when combined with thoughtful, accessibility-first mobile engineering—can directly improve human lives. By leveraging Google Cloud Run and Vertex AI, the application remains highly scalable, cost-effective, and incredibly fast.

I'd love to hear your thoughts on Edge-to-Cloud architectures! Have you worked with CameraX or Vertex AI recently? Let me know in the comments. 👇

👨‍💻 About the Developer

I am Muhammad Zubair, a Software, DevOps, and Platform Engineer from Pakistan. I specialize in architecting scalable AI infrastructure, cloud platforms (AWS/Kubernetes), and full-stack systems.

My passion lies in solving real-world problems through code, whether that's building real-time mobility radars for the visually impaired or preserving regional languages through AI.

🔗 Let's Connect:

Portfolio & Case Studies: mzubair.online
GitHub (76+ Repos): Muhammad-Zubair796
LinkedIn: Muhammad Zubair

How I Engineered the World's First Khattak Pashto LLM (Qwen2 + LoRA)

Muhammad Zubair — Thu, 10 Sep 2026 11:12:58 +0000

Standard Large Language Models (LLMs) are incredible, but they have a massive blind spot: rural and regional dialects.

While standard Pashto is supported by some models, they completely fail to capture the rich, localized grammar of the Khattak dialect spoken in Karak, Nowshera, and Kohat (Pakistan).

To solve this, I built Khatta-ka-LLM—the world's first AI language model fine-tuned specifically for the Khattak (Khatak) dialect.

(Note: You can read the original architectural breakdown on my Software Engineering Portfolio or test the model live below).

Watch the Live Testing Video:

🛠️ Technical Architecture

As a Software Engineer specializing in AI Infrastructure, my focus was on creating a clean, scalable, and reproducible pipeline for low-resource languages. Here is how I built it:

1. The Custom Dataset

Because no dataset existed for this dialect, I curated a custom dataset of 2,000+ English-to-Khattak pairs. This represents the first digitized collection of Khattak linguistic markers, focusing heavily on the unique grammar used in the Karak and Kohat regions.

2. The Fine-Tuning Pipeline

To achieve high performance on consumer-grade hardware, I utilized the following stack:

Base Model: Qwen2 Architecture
Framework: Unsloth (for highly optimized, faster training)
Technique: LoRA (Low-Rank Adaptation)
Performance: Through careful hyperparameter tuning, the training loss dropped significantly from 3.44 to 0.22.

📘 The Linguistic Engineering (Rulebook)

To ensure the AI didn't just speak "standard Pashto with a Khattak accent," I had to engineer specific linguistic rules into the model's training weights.

Here are a few of the core rules the model learned:

1. Pronouns & Possession

My / Mine: The model uses مو والا (Mo wala) instead of the standard Zama.
To me: Uses موته (Mota) instead of standard Mala.
We: Uses موخ (Moxh) instead of standard Mung.

2. The "Noon Ghunna" (ں) Rule

In the Khattak dialect, first-person verbs end in a nasal Noon Ghunna (ں) rather than the standard "M".

Standard: Za Kar Kawom (I do work)
Khattak AI: زه چار کاوں (Za char kaon)

3. Phonetic "O" Shifts

The model successfully converts standard "A" sounds to the deep Khattak "O":

Asmaan ➡️ Asmon (اسمون)
Baraan ➡️ Baron (بارون)
Bazaar ➡️ Bazor (بازور)

💡 Why This Matters

By focusing on high-quality, localized data, Khatta-ka-LLM successfully translates English into pure Khattak Pashto. As a Software Engineer from Pakistan, my goal is to bridge the gap between modern AI infrastructure and regional linguistic heritage. We shouldn't have to lose our dialects to use modern technology.

I would love to hear your thoughts on fine-tuning models for low-resource languages! Have you worked with Unsloth or LoRA recently? Let me know in the comments. 👇

👨‍💻 About the Developer

My passion lies in solving real-world problems through code, whether that's building real-time mobility radars for the visually impaired or preserving regional languages through AI.

🔗 Let's Connect:

Portfolio & Case Studies: mzubair.online
GitHub (76+ Repos): Muhammad-Zubair796
LinkedIn: Muhammad Zubair