DEV Community: ARSHIYA Sohrevardi

How I Fine-Tuned a 1.5B LLM for Lightning-Fast Offline Q&A on 1GB VRAM

ARSHIYA Sohrevardi — Fri, 14 Aug 2026 15:02:46 +0000

Running large language models locally often demands expensive hardware with high VRAM. However, for specialized tasks like offline Q&A and knowledge retrieval, a lightweight, highly optimized small language model (SLM) can deliver incredible speed and efficiency without breaking the bank.

In this post, I will share how I fine-tuned moeinGTS 1.5B, quantized it to GGUF, and deployed it for fast offline inference.

🚀 Model Overview: moeinGTS 1.5B
moeinGTS is a compact 1.5B parameter language model fine-tuned specifically for fast, accurate question-and-answering based on Wikipedia and structured datasets.

Key Highlights:
Parameter Size: 1.5 Billion
Quantization: Q4_K_M & F16 GGUF
Download Size: ~1 GB
VRAM Requirements: Only ~1.5 GB
Primary Use Case: Offline local Q&A, low-latency edge AI applications
🛠️ The Pipeline: Fine-Tuning & Deployment
Dataset & Training: Fine-tuned on high-quality Q&A and knowledge retrieval datasets to improve concise factual responses.
LoRA Fine-Tuning: Applied Low-Rank Adaptation (LoRA) via PEFT to optimize training efficiency and keep resource usage minimal.
Quantization: Converted the model weights to GGUF format for seamless cross-platform execution on low-end hardware, CPUs, and edge devices.

How I Fine-Tuned a 1.5B LLM for Lightning-Fast Offline Q&A on 1GB VRAM

ARSHIYA Sohrevardi — Sat, 08 Aug 2026 11:49:55 +0000

In this post, I will share how I fine-tuned moeinGTS 1.5B, quantized it to GGUF, and deployed it for fast offline inference.

🚀 Model Overview: moeinGTS 1.5B

moeinGTS is a compact 1.5B parameter language model fine-tuned specifically for fast, accurate question-and-answering based on Wikipedia and structured datasets.

Key Highlights:

Parameter Size: 1.5 Billion
Quantization: Q4_K_M & F16 GGUF
Download Size: ~1 GB
VRAM Requirements: Only ~1.5 GB
Primary Use Case: Offline local Q&A, low-latency edge AI applications

🛠️ The Pipeline: Fine-Tuning & Deployment

Dataset & Training: Fine-tuned on high-quality Q&A and knowledge retrieval datasets to improve concise factual responses.
LoRA Fine-Tuning: Applied Low-Rank Adaptation (LoRA) via PEFT to optimize training efficiency and keep resource usage minimal.
Quantization: Converted the model weights to GGUF format for seamless cross-platform execution on low-end hardware, CPUs, and edge devices.

💻 Quickstart: Run It Locally

You can run moeinGTS instantly in your terminal using Ollama directly from Hugging Face:


bash
# Run GGUF version directly via Ollama
ollama run hf.co/arshiysohrevardi/moeinGTS1.5-1.5b-F16-GGUF

moeingts

ARSHIYA Sohrevardi — Fri, 07 Aug 2026 12:49:22 +0000

well last week i start to have a llm model but with a big different!! i make a chatbot that it be just for me!! you know i made a llm model with fine tunning on important question related to wikipedia and sites that answer to them, the model latest named moeinGTS1,5:1,5b in ollama!!
the link : https://ollama.com/arshiyasohrevardimoein/moeinGTS

and good think about size and ram! this model look alike qwen and llama model but it size is 1 GIG not 3 or 2 or 4 GIG and your RAM feel better😎😊🤖