<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Darshan Pl</title>
    <description>The latest articles on DEV Community by Darshan Pl (@darshan_pl_edf45ed62d6d42).</description>
    <link>https://dev.to/darshan_pl_edf45ed62d6d42</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4250360%2F5d712ff8-7266-4e62-a011-4fea2e98f00b.png</url>
      <title>DEV Community: Darshan Pl</title>
      <link>https://dev.to/darshan_pl_edf45ed62d6d42</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/darshan_pl_edf45ed62d6d42"/>
    <language>en</language>
    <item>
      <title>[Boost]</title>
      <dc:creator>Darshan Pl</dc:creator>
      <pubDate>Tue, 17 Nov 2026 18:29:15 +0000</pubDate>
      <link>https://dev.to/darshan_pl_edf45ed62d6d42/-1o2d</link>
      <guid>https://dev.to/darshan_pl_edf45ed62d6d42/-1o2d</guid>
      <description>&lt;div class="ltag__link--embedded"&gt;
  &lt;div class="crayons-story "&gt;
  &lt;a href="https://dev.to/darshan_pl_edf45ed62d6d42/labha-ai-building-a-small-language-model-from-scratch-my-journey-into-llm-development-mpi" class="crayons-story__hidden-navigation-link"&gt;Labha AI: Building a Small Language Model from Scratch — My Journey into LLM Development&lt;/a&gt;


  &lt;div class="crayons-story__body crayons-story__body-full_post"&gt;
    &lt;div class="crayons-story__top"&gt;
      &lt;div class="crayons-story__meta"&gt;
        &lt;div class="crayons-story__author-pic"&gt;

          &lt;a href="/darshan_pl_edf45ed62d6d42" class="crayons-avatar  crayons-avatar--l  "&gt;
            &lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4250360%2F5d712ff8-7266-4e62-a011-4fea2e98f00b.png" alt="darshan_pl_edf45ed62d6d42 profile" class="crayons-avatar__image" width="96" height="96"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        &lt;div&gt;
          &lt;div&gt;
            &lt;a href="/darshan_pl_edf45ed62d6d42" class="crayons-story__secondary fw-medium m:hidden"&gt;
              Darshan Pl
            &lt;/a&gt;
            &lt;div class="profile-preview-card relative mb-4 s:mb-0 fw-medium hidden m:inline-block"&gt;
              
                Darshan Pl
                
                
              
              &lt;div id="story-author-preview-content-5058950" class="profile-preview-card__content crayons-dropdown branded-7 p-4 pt-0"&gt;
                &lt;div class="gap-4 grid"&gt;
                  &lt;div class="-mt-4"&gt;
                    &lt;a href="/darshan_pl_edf45ed62d6d42" class="flex"&gt;
                      &lt;span class="crayons-avatar crayons-avatar--xl mr-2 shrink-0"&gt;
                        &lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4250360%2F5d712ff8-7266-4e62-a011-4fea2e98f00b.png" class="crayons-avatar__image" alt="" width="96" height="96"&gt;
                      &lt;/span&gt;
                      &lt;span class="crayons-link crayons-subtitle-2 mt-5"&gt;Darshan Pl&lt;/span&gt;
                    &lt;/a&gt;
                  &lt;/div&gt;
                  &lt;div class="print-hidden"&gt;
                    
                      Follow
                    
                  &lt;/div&gt;
                  &lt;div class="author-preview-metadata-container"&gt;&lt;/div&gt;
                &lt;/div&gt;
              &lt;/div&gt;
            &lt;/div&gt;

          &lt;/div&gt;
          &lt;a href="https://dev.to/darshan_pl_edf45ed62d6d42/labha-ai-building-a-small-language-model-from-scratch-my-journey-into-llm-development-mpi" class="crayons-story__tertiary fs-xs"&gt;&lt;time&gt;Nov 17&lt;/time&gt;&lt;span class="time-ago-indicator-initial-placeholder"&gt;&lt;/span&gt;&lt;/a&gt;
        &lt;/div&gt;
      &lt;/div&gt;

    &lt;/div&gt;

    &lt;div class="crayons-story__indention"&gt;
      &lt;h2 class="crayons-story__title crayons-story__title-full_post"&gt;
        &lt;a href="https://dev.to/darshan_pl_edf45ed62d6d42/labha-ai-building-a-small-language-model-from-scratch-my-journey-into-llm-development-mpi" id="article-link-5058950"&gt;
          Labha AI: Building a Small Language Model from Scratch — My Journey into LLM Development
        &lt;/a&gt;
      &lt;/h2&gt;
        &lt;div class="crayons-story__tags"&gt;
            &lt;a class="crayons-tag  crayons-tag--monochrome " href="/t/ai"&gt;&lt;span class="crayons-tag__prefix"&gt;#&lt;/span&gt;ai&lt;/a&gt;
            &lt;a class="crayons-tag  crayons-tag--monochrome " href="/t/llm"&gt;&lt;span class="crayons-tag__prefix"&gt;#&lt;/span&gt;llm&lt;/a&gt;
            &lt;a class="crayons-tag  crayons-tag--monochrome " href="/t/machinelearning"&gt;&lt;span class="crayons-tag__prefix"&gt;#&lt;/span&gt;machinelearning&lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="crayons-story__bottom"&gt;
        &lt;div class="crayons-story__details"&gt;
          &lt;a href="https://dev.to/darshan_pl_edf45ed62d6d42/labha-ai-building-a-small-language-model-from-scratch-my-journey-into-llm-development-mpi" class="crayons-btn crayons-btn--s crayons-btn--ghost crayons-btn--icon-left"&gt;
            &lt;div class="multiple_reactions_aggregate"&gt;
              &lt;span class="multiple_reactions_icons_container"&gt;
                  &lt;span class="crayons_icon_container"&gt;
                    &lt;img src="https://assets.dev.to/assets/sparkle-heart-5f9bee3767e18deb1bb725290cb151c25234768a0e9a2bd39370c382d02920cf.svg" width="24" height="24"&gt;
                  &lt;/span&gt;
              &lt;/span&gt;
              &lt;span class="aggregate_reactions_counter"&gt;1&lt;span class="hidden s:inline"&gt;&amp;nbsp;reaction&lt;/span&gt;&lt;/span&gt;
            &lt;/div&gt;
          &lt;/a&gt;
            &lt;a href="https://dev.to/darshan_pl_edf45ed62d6d42/labha-ai-building-a-small-language-model-from-scratch-my-journey-into-llm-development-mpi#comments" class="crayons-btn crayons-btn--s crayons-btn--ghost crayons-btn--icon-left flex items-center"&gt;
              

              &lt;span class="hidden s:inline"&gt;Add&amp;nbsp;Comment&lt;/span&gt;
            &lt;/a&gt;
        &lt;/div&gt;
        &lt;div class="crayons-story__save"&gt;
          &lt;small class="crayons-story__tertiary fs-xs mr-2"&gt;
            3 min read
          &lt;/small&gt;
          
          &lt;div class="only-sidebar-menu-item trusted-visible-block print-hidden align-center"&gt;
                
                    
                
          &lt;/div&gt;

        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;


&lt;/div&gt;


</description>
    </item>
    <item>
      <title>Labha AI: Building a Small Language Model from Scratch — My Journey into LLM Development</title>
      <dc:creator>Darshan Pl</dc:creator>
      <pubDate>Tue, 17 Nov 2026 17:37:05 +0000</pubDate>
      <link>https://dev.to/darshan_pl_edf45ed62d6d42/labha-ai-building-a-small-language-model-from-scratch-my-journey-into-llm-development-mpi</link>
      <guid>https://dev.to/darshan_pl_edf45ed62d6d42/labha-ai-building-a-small-language-model-from-scratch-my-journey-into-llm-development-mpi</guid>
      <description>&lt;h1&gt;
  
  
  Labha AI: Building a Small Language Model from Scratch — My Journey into LLM Development
&lt;/h1&gt;

&lt;p&gt;Artificial Intelligence is evolving rapidly, and Large Language Models (LLMs) are transforming how we interact with technology. While using existing models is valuable, I wanted to understand what happens behind the scenes: How are language models designed? How do tokenizers work? How does a model learn from text? Can I build my own language model?&lt;/p&gt;

&lt;p&gt;These questions inspired me to start &lt;strong&gt;Labha AI&lt;/strong&gt;, my experimental Small Language Model (SLM) project.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. What is Labha AI?
&lt;/h2&gt;

&lt;p&gt;Labha AI is an experimental language model that I am developing to understand the fundamentals of Large Language Model architecture, tokenization, training, and text generation.&lt;/p&gt;

&lt;p&gt;Rather than simply integrating an existing LLM API, my objective is to explore the core components of a language model and develop a model with my own architecture and training pipeline.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Model Architecture
&lt;/h2&gt;

&lt;p&gt;The current Labha AI base model has the following configuration:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;Configuration&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Architecture&lt;/td&gt;
&lt;td&gt;Transformer-based&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Total parameters&lt;/td&gt;
&lt;td&gt;1.54 million&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Vocabulary size&lt;/td&gt;
&lt;td&gt;1,821&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Embedding dimension&lt;/td&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Transformer layers&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Attention heads&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Context length&lt;/td&gt;
&lt;td&gt;128 tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;This compact architecture is designed for experimentation and learning rather than competing with large production-scale LLMs.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. My Training Environment
&lt;/h2&gt;

&lt;p&gt;For model development and experimentation, I have used:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Framework:&lt;/strong&gt; PyTorch&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Programming language:&lt;/strong&gt; Python&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Training environment:&lt;/strong&gt; Google Colab&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GPU:&lt;/strong&gt; NVIDIA Tesla T4&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Training approach:&lt;/strong&gt; Custom language-model training experiments&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Working with a GPU helped me experiment with model training and understand how computational resources affect the development process.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Dataset Preparation
&lt;/h2&gt;

&lt;p&gt;One of the important parts of language-model development is preparing good-quality training data.&lt;/p&gt;

&lt;p&gt;During my experiments, I worked on a Wikipedia text dataset.&lt;/p&gt;

&lt;p&gt;My data-cleaning report showed:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Total articles: 549,976&lt;/li&gt;
&lt;li&gt;Clean articles: 541,831&lt;/li&gt;
&lt;li&gt;Duplicate articles: 0&lt;/li&gt;
&lt;li&gt;Short articles: 8,145&lt;/li&gt;
&lt;li&gt;Processed dataset size: Approximately 1.983 GB&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This process helped me understand the importance of data cleaning, preprocessing, duplicate detection, and dataset quality.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Challenges I Faced
&lt;/h2&gt;

&lt;p&gt;Developing a language model is not just about writing a neural network. I encountered several challenges during experimentation:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Tokenizer vocabulary and token-generation issues.&lt;/li&gt;
&lt;li&gt;Repeated tokens during text generation.&lt;/li&gt;
&lt;li&gt;Training stability and loss optimization.&lt;/li&gt;
&lt;li&gt;Understanding attention mechanisms and context length.&lt;/li&gt;
&lt;li&gt;Model checkpoint management and inference.&lt;/li&gt;
&lt;li&gt;Balancing model size, training data, and available GPU resources.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These challenges became valuable learning opportunities.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. What I Learned
&lt;/h2&gt;

&lt;p&gt;Through Labha AI, I have been exploring:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Transformer architecture&lt;/li&gt;
&lt;li&gt;Tokenization and vocabulary construction&lt;/li&gt;
&lt;li&gt;Embeddings and positional information&lt;/li&gt;
&lt;li&gt;Multi-head self-attention&lt;/li&gt;
&lt;li&gt;Language-model training&lt;/li&gt;
&lt;li&gt;Loss functions and optimization&lt;/li&gt;
&lt;li&gt;Text generation and inference&lt;/li&gt;
&lt;li&gt;Dataset preprocessing&lt;/li&gt;
&lt;li&gt;GPU-based model experimentation&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The project has helped me move beyond simply using AI models toward understanding how they are built.&lt;/p&gt;

&lt;h2&gt;
  
  
  7. Future Roadmap
&lt;/h2&gt;

&lt;p&gt;My next development goals include:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Improving tokenizer quality and vocabulary coverage.&lt;/li&gt;
&lt;li&gt;Experimenting with larger context windows.&lt;/li&gt;
&lt;li&gt;Improving generated-text quality and reducing repetition.&lt;/li&gt;
&lt;li&gt;Building a more reliable training and evaluation pipeline.&lt;/li&gt;
&lt;li&gt;Exploring instruction fine-tuning.&lt;/li&gt;
&lt;li&gt;Developing a simple interface for interacting with Labha AI.&lt;/li&gt;
&lt;li&gt;Evaluating the model on structured language-generation tasks.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;These are development goals, not features I am claiming are already complete.&lt;/p&gt;

&lt;h2&gt;
  
  
  8. Why I Started This Project
&lt;/h2&gt;

&lt;p&gt;My long-term goal is to become an AI Engineer with a strong understanding of machine learning, deep learning, and language-model architectures.&lt;/p&gt;

&lt;p&gt;Labha AI represents an important step in that journey. It is an opportunity to learn through implementation, experimentation, debugging, and continuous improvement.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;I believe the best way to understand AI is not only to use existing models, but also to build, experiment, and learn how they work.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;This is just the beginning of Labha AI. More experiments and improvements are ahead!&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Let's connect and discuss AI, Small Language Models, Transformers, and LLM development.&lt;/strong&gt;&lt;/p&gt;

&lt;h1&gt;
  
  
  LabhaAI #SmallLanguageModels #LLM #ArtificialIntelligence #MachineLearning #DeepLearning #PyTorch #TransformerArchitecture #GenerativeAI #BuildInPublic
&lt;/h1&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>machinelearning</category>
    </item>
  </channel>
</rss>
