The latest articles on DEV Community by Mudasir Habib (@mudasirhabib123). https://dev.to/mudasirhabib123 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1446387%2F5c2834aa-8c5c-4be1-a134-c3c9ada65ded.jpeg https://dev.to/mudasirhabib123 en Mudasir Habib Thu, 23 Apr 2026 14:30:13 +0000 https://dev.to/mudasirhabib123/a-minimal-9m-parameter-transformer-llm-trained-from-scratch-1p21 https://dev.to/mudasirhabib123/a-minimal-9m-parameter-transformer-llm-trained-from-scratch-1p21 <h2> SelfLM — Building a Tiny LLM from Scratch (End-to-End) </h2> <p>LLMs are complex, but not magical — once you break them into components, everything becomes understandable.</p> <p>It started with a simple question:<br> <strong>“How do models like GPT actually work?”</strong></p> <p>So I decided to build a smaller version myself — step by step — from dataset generation to tokenization, training, and deployment. Everything is fully open-source.</p> <h2> What This Project Covers </h2> <p>Instead of treating models as black boxes, this project focuses on the <strong>entire pipeline</strong>:</p> <ul> <li>Synthetic dataset generation (~60K samples)</li> <li>Tokenization &amp; preprocessing</li> <li>Transformer architecture (from scratch)</li> <li>Training pipeline</li> <li>Inference &amp; deployment</li> </ul> <h2> Highlights </h2> <ul> <li>Trained in ~5 minutes (Colab T4 GPU)</li> <li>Fully custom LLM (~9M parameters)</li> <li>Hugging Face model + dataset + live Space</li> <li>Serverless deployment using ONNX on Vercel (free tier)</li> <li>Lightweight, browser-friendly inference</li> </ul> <h2> Live Demo </h2> <p><a href="https://selflm.vercel.app/docs" rel="noopener noreferrer">https://selflm.vercel.app/docs</a></p> <h2> Hugging Face Space </h2> <p><a href="https://huggingface.co/spaces/Mudasir-Habib/selflm-demo" rel="noopener noreferrer">https://huggingface.co/spaces/Mudasir-Habib/selflm-demo</a></p> <h2> Colab Notebook </h2> <p><a href="https://colab.research.google.com/drive/1EyR5mFuHupJWdnJWazvdjU1Bre2rF2RD?usp=sharing" rel="noopener noreferrer">https://colab.research.google.com/drive/1EyR5mFuHupJWdnJWazvdjU1Bre2rF2RD?usp=sharing</a></p> <h2> GitHub Repository </h2> <p><a href="https://github.com/Mudasirhabib123/selflm" rel="noopener noreferrer">https://github.com/Mudasirhabib123/selflm</a></p> <h2> Customization Feature </h2> <p>One of the most interesting parts:</p> <p>You can <strong>customize the model with your own data</strong> by simply:</p> <ul> <li>Editing the first cell in the Colab notebook</li> <li>OR modifying <code>src/dataset/data.py</code> </li> </ul> <p>Add your own context, retrain, and instantly get a personalized LLM.</p> <h2> Goal </h2> <p>This project is built for:</p> <ul> <li>Learning how LLMs actually work</li> <li>Experimentation with small-scale models</li> <li>Understanding the full pipeline end-to-end</li> </ul> <h2> Open Source </h2> <p>Fully open-source and designed to make LLMs <strong>accessible, transparent, and understandable</strong>.</p> <p>If you find it useful, consider giving it a star on GitHub.</p> ai opensource llm challenge