<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Mbonimpa Kelia</title>
    <description>The latest articles on DEV Community by Mbonimpa Kelia (@mbonimpa_kelia_9).</description>
    <link>https://dev.to/mbonimpa_kelia_9</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4151839%2F7316dfc3-fddd-4a0a-b62a-79a3ad7aa784.png</url>
      <title>DEV Community: Mbonimpa Kelia</title>
      <link>https://dev.to/mbonimpa_kelia_9</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/mbonimpa_kelia_9"/>
    <language>en</language>
    <item>
      <title>How Open-Source LLMs Can Improve My Tiny GPT-2 Next-Word Predictor</title>
      <dc:creator>Mbonimpa Kelia</dc:creator>
      <pubDate>Mon, 26 Oct 2026 14:17:55 +0000</pubDate>
      <link>https://dev.to/mbonimpa_kelia_9/how-open-source-llms-can-improve-my-tiny-gpt-2-next-word-predictor-4peg</link>
      <guid>https://dev.to/mbonimpa_kelia_9/how-open-source-llms-can-improve-my-tiny-gpt-2-next-word-predictor-4peg</guid>
      <description>&lt;h2&gt;
  
  
  1. Project Background: From a Tiny Model to a Better Predictor
&lt;/h2&gt;

&lt;p&gt;For our Natural Language Processing (NLP) group project, we built a Tiny GPT-2 from scratch: a decoder-only Transformer with a causal attention mask, trained to predict the next token in English text. The model runs, and given a prompt such as “She went to the”, it can suggest possible continuations such as “market”, “school” or “park”.&lt;/p&gt;

&lt;p&gt;However, getting a model to generate text does not necessarily mean it generates good text. Our small model has limited capacity, learns from a relatively small training corpus and sometimes produces repetitive or grammatically awkward sentences.&lt;/p&gt;

&lt;p&gt;This raised an important question: &lt;strong&gt;How can open-weight Large Language Models (LLMs) help improve our Tiny GPT-2 without replacing the model we built?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;After researching several models, I concluded that they can serve different purposes: generating supplementary training data, providing a reference for knowledge distillation, helping evaluate generated text and offering insights into training techniques. My main choice for data generation and evaluation is Qwen3-4B-Instruct-2507, while GPT-2 Small is a potential teacher for distillation.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. The Open-Weight Models I Researched
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Qwen3-4B-Instruct-2507 — My Primary Choice
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Developer:&lt;/strong&gt; Qwen team, Alibaba Cloud&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Parameters:&lt;/strong&gt; 4 billion, including approximately 3.6 billion non-embedding parameters&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Context window:&lt;/strong&gt; 262,144 tokens&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Licence:&lt;/strong&gt; Apache 2.0&lt;/p&gt;

&lt;p&gt;Qwen3-4B-Instruct-2507 is an instruction-tuned language model designed for tasks including text generation, instruction following, logical reasoning and multilingual understanding. Unlike a basic next-token predictor, it can respond to instructions and generate text in a requested format.&lt;/p&gt;

&lt;p&gt;The model supports non-thinking mode and does not generate separate &lt;code&gt;&amp;lt;think&amp;gt;&lt;/code&gt; blocks. This can simplify its use in automated data-generation pipelines, although the requested output format still needs to be checked.&lt;/p&gt;

&lt;p&gt;I selected Qwen3 for three potential uses:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Data augmentation:&lt;/strong&gt; Generate additional English sentences and continuations to supplement our original training corpus.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Semantic evaluation:&lt;/strong&gt; Assess whether generated continuations appear relevant and plausible in context, alongside objective metrics such as perplexity.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Training examples:&lt;/strong&gt; Produce examples for experiments involving specific continuation styles.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Qwen3's public model weights can be downloaded for local use without a paid inference API. Its Apache 2.0 licence permits broad use subject to its terms. At 16-bit precision, its weights require approximately 8 GB of memory. This makes it a plausible candidate for a Google Colab T4 with 16 GB of GPU memory, but the complete model may require additional memory for inference, temporary computations and the attention cache. Shorter contexts or quantisation may be necessary.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Reference:&lt;/strong&gt; &lt;a href="https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507" rel="noopener noreferrer"&gt;Qwen3-4B-Instruct-2507 model card&lt;/a&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  GPT-2 Small — A Potential Distillation Teacher
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Developer:&lt;/strong&gt; OpenAI&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Parameters:&lt;/strong&gt; Approximately 124 million&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Context window:&lt;/strong&gt; 1,024 tokens&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Licence:&lt;/strong&gt; MIT&lt;/p&gt;

&lt;p&gt;GPT-2 Small is a pretrained, decoder-only Transformer that predicts the next token using causal language modelling. Its architecture and training objective make it a useful reference for our Tiny GPT-2.&lt;/p&gt;

&lt;p&gt;Knowledge distillation is a technique in which a smaller student model learns from the outputs of a larger or more capable teacher. Instead of training only against the correct next token, the student can learn from the teacher's probability distribution over possible next tokens.&lt;/p&gt;

&lt;p&gt;However, successful distillation requires compatible tokenisation and output vocabularies, or an appropriate method for aligning them. Using the same GPT-2 tokenizer and vocabulary would make the experiment more straightforward. The training loop would also need to support a distillation loss; this is not achieved automatically without code changes.&lt;/p&gt;

&lt;p&gt;GPT-2 Small is therefore a promising teacher to investigate, particularly if our Tiny GPT-2 uses the same tokenizer. Its smaller size also makes it practical for experiments on limited hardware.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Reference:&lt;/strong&gt; &lt;a href="https://huggingface.co/openai-community/gpt2" rel="noopener noreferrer"&gt;GPT-2 model card&lt;/a&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  SmolLM3-3B — A Training and Reproducibility Reference
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Developer:&lt;/strong&gt; Hugging Face&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Parameters:&lt;/strong&gt; Approximately 3 billion&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Context window:&lt;/strong&gt; 65,536 tokens by default; extension to 131,072 tokens is possible with YaRN configuration&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Licence:&lt;/strong&gt; Apache 2.0&lt;/p&gt;

&lt;p&gt;SmolLM3-3B is an open-weight language model whose documentation provides useful information about its architecture, training and supported configurations.&lt;/p&gt;

&lt;p&gt;For our project, its main value is not necessarily to become another component in the final system. Instead, we can study its published training approach and compare its techniques with our own implementation.&lt;/p&gt;

&lt;p&gt;For example, we can investigate how data preparation, tokenisation, training duration and model configuration affect language modelling performance. Its approximately 6 GB of weights at 16-bit precision make it a plausible candidate for a T4, although additional runtime memory is still required.&lt;/p&gt;

&lt;p&gt;SmolLM3 is particularly relevant to a learning project because its documented training recipe offers a reference for understanding how modern open models are developed.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Reference:&lt;/strong&gt; &lt;a href="https://huggingface.co/HuggingFaceTB/SmolLM3-3B" rel="noopener noreferrer"&gt;SmolLM3-3B model card&lt;/a&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Other Models Worth Considering
&lt;/h3&gt;

&lt;p&gt;Several additional models provide useful points of comparison:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Llama 3.2-3B-Instruct — Meta:&lt;/strong&gt; A 3.21-billion-parameter instruction-tuned model with a 128K context window. It is useful for text generation and multilingual dialogue, but its custom community licence and access conditions require review.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Gemma 3 4B IT — Google:&lt;/strong&gt; A 4-billion-parameter instruction-tuned model with a 128K context window. It supports text and image input, making it interesting for multimodal research, although image processing is not necessary for our current text-only project. It uses Google's Gemma terms of use.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Phi-4-mini-instruct — Microsoft:&lt;/strong&gt; A 3.8-billion-parameter model with a 128K context window, designed for instruction following and strong reasoning performance. It is another candidate for generating and assessing text.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mistral 7B Instruct v0.3 — Mistral AI:&lt;/strong&gt; A 7-billion-parameter instruction-tuned model with a 32K context window and Apache 2.0 licence. Its 16-bit weights alone require approximately 14 GB, making it a less comfortable choice for a 16 GB T4.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These models demonstrate that parameter count, context length, licensing and intended use all matter when selecting a model. A larger model is not automatically the best choice for a student project.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Comparison Table
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Developer&lt;/th&gt;
&lt;th&gt;Parameters&lt;/th&gt;
&lt;th&gt;Context window&lt;/th&gt;
&lt;th&gt;Licence&lt;/th&gt;
&lt;th&gt;T4 suitability at 16-bit precision&lt;/th&gt;
&lt;th&gt;Main value for my project&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Qwen3-4B-Instruct-2507&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Alibaba Cloud's Qwen team&lt;/td&gt;
&lt;td&gt;4B&lt;/td&gt;
&lt;td&gt;262K&lt;/td&gt;
&lt;td&gt;Apache 2.0&lt;/td&gt;
&lt;td&gt;Plausible; ~8 GB weights&lt;/td&gt;
&lt;td&gt;Data augmentation and semantic evaluation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;GPT-2 Small&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;124M&lt;/td&gt;
&lt;td&gt;1,024&lt;/td&gt;
&lt;td&gt;MIT&lt;/td&gt;
&lt;td&gt;Very comfortable; ~0.25 GB weights&lt;/td&gt;
&lt;td&gt;Potential distillation teacher&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;SmolLM3-3B&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Hugging Face&lt;/td&gt;
&lt;td&gt;~3B&lt;/td&gt;
&lt;td&gt;65,536 native&lt;/td&gt;
&lt;td&gt;Apache 2.0&lt;/td&gt;
&lt;td&gt;Plausible; ~6 GB weights&lt;/td&gt;
&lt;td&gt;Training reference and reproducibility&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Llama 3.2-3B-Instruct&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Meta&lt;/td&gt;
&lt;td&gt;3.21B&lt;/td&gt;
&lt;td&gt;128K&lt;/td&gt;
&lt;td&gt;Llama 3.2 Community Licence&lt;/td&gt;
&lt;td&gt;Plausible; ~6.4 GB weights&lt;/td&gt;
&lt;td&gt;Alternative data generator&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Gemma 3 4B IT&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Google&lt;/td&gt;
&lt;td&gt;4B&lt;/td&gt;
&lt;td&gt;128K&lt;/td&gt;
&lt;td&gt;Gemma terms of use&lt;/td&gt;
&lt;td&gt;Plausible; ~8 GB weights, with additional multimodal overhead where applicable&lt;/td&gt;
&lt;td&gt;Alternative instruction-following model&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Phi-4-mini-instruct&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Microsoft&lt;/td&gt;
&lt;td&gt;3.8B&lt;/td&gt;
&lt;td&gt;128K&lt;/td&gt;
&lt;td&gt;MIT&lt;/td&gt;
&lt;td&gt;Plausible; ~7.6 GB weights&lt;/td&gt;
&lt;td&gt;Alternative for text generation and reasoning&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Mistral 7B Instruct v0.3&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Mistral AI&lt;/td&gt;
&lt;td&gt;7B&lt;/td&gt;
&lt;td&gt;32K&lt;/td&gt;
&lt;td&gt;Apache 2.0&lt;/td&gt;
&lt;td&gt;Tight; ~14 GB weights&lt;/td&gt;
&lt;td&gt;Larger instruction-following comparison&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;em&gt;Memory figures are approximate weight sizes, not measured peak GPU memory. Actual requirements depend on precision, context length, batch size and implementation. Public weights can generally be used locally without a paid API, but account requirements and licence acceptance vary by model repository.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Why I Chose Qwen3-4B-Instruct-2507
&lt;/h2&gt;

&lt;p&gt;My choice is based on three main reasons.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;First, it can help expand our training data.&lt;/strong&gt; A small training corpus limits the range of language patterns our model encounters. Qwen3 can generate varied English sentences and continuations that supplement the original corpus. However, synthetic examples must be reviewed for quality and diversity. Adding more generated text does not automatically improve performance and may introduce errors or repetitive patterns.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Second, it offers an additional evaluation method.&lt;/strong&gt; Perplexity measures how well a language model predicts a reference sequence, but it does not directly measure every aspect of grammatical quality or semantic usefulness. Qwen3 can act as an auxiliary evaluator by scoring continuations against a consistent rubric. Because an LLM judge can also make mistakes or show preferences for particular writing styles, its scores should be compared with human assessments and objective metrics.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Third, it is practical for experimentation.&lt;/strong&gt; The model has a permissive Apache 2.0 licence and can be used locally through tools such as Hugging Face Transformers. Its weight size makes a T4 experiment plausible, although actual memory usage and generation speed must be measured rather than assumed.&lt;/p&gt;

&lt;p&gt;Why not use Qwen3 for every task, including distillation? GPT-2 Small may be a more convenient teacher when our student uses the original GPT-2 tokenizer and vocabulary. A teacher with a compatible vocabulary makes it easier to compare next-token distributions. Qwen3 has a different tokenizer, so direct distribution matching requires additional alignment or a different distillation strategy.&lt;/p&gt;

&lt;p&gt;My proposed division of work is therefore: &lt;strong&gt;Qwen3 for supplementary data and evaluation, and GPT-2 Small for a carefully designed distillation experiment.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  5. What I Plan to Build and Measure
&lt;/h2&gt;

&lt;p&gt;To determine whether these techniques genuinely improve Tiny GPT-2, I would conduct the following experiments:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Baseline:&lt;/strong&gt; Train Tiny GPT-2 on the original corpus and record validation loss, perplexity and generated samples.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Distillation:&lt;/strong&gt; Train a separate student using GPT-2 Small's probability distributions, provided the tokenizer and vocabulary are compatible. Compare it with the baseline.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Data augmentation:&lt;/strong&gt; Generate supplementary English text with Qwen3, retrain the model under comparable conditions and measure changes in validation performance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Semantic evaluation:&lt;/strong&gt; Ask Qwen3 to assess a fixed sample of continuations using a consistent rubric. Compare its ratings with human judgements and perplexity.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Error analysis:&lt;/strong&gt; Examine repetition, grammatical errors, irrelevant continuations and differences between training and validation performance.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;All experiments should use a held-out validation set that is not included in the training data. This helps reveal whether an improvement generalises beyond the examples used to train the model.&lt;/p&gt;

&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Building Tiny GPT-2 helped us understand the practical foundations of language modelling, from tokenisation and causal attention to next-token prediction. Researching open-weight LLMs showed me that we can improve a small model without simply replacing it with a larger one.&lt;/p&gt;

&lt;p&gt;Qwen3-4B-Instruct-2507 is my preferred model for generating supplementary training examples and providing an additional perspective on output quality. GPT-2 Small offers a promising teacher for distillation, while SmolLM3 provides a useful reference for studying modern training methods.&lt;/p&gt;

&lt;p&gt;The key is to test these ideas systematically. If our experiments show lower validation perplexity, more useful continuations and fewer repetitive errors, we will have evidence that the improvements worked. If not, the results will still teach us something important about the limitations of synthetic data, distillation and automated evaluation.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;My main takeaway is that open-weight LLMs can be more than ready-made applications: they can also be tools for learning, experimentation and improving the models we build ourselves.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Qwen Team. &lt;a href="https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507" rel="noopener noreferrer"&gt;Qwen3-4B-Instruct-2507 model card&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;OpenAI Community. &lt;a href="https://huggingface.co/openai-community/gpt2" rel="noopener noreferrer"&gt;GPT-2 model card&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Hugging Face. &lt;a href="https://huggingface.co/HuggingFaceTB/SmolLM3-3B" rel="noopener noreferrer"&gt;SmolLM3-3B model card&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Meta. &lt;a href="https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md" rel="noopener noreferrer"&gt;Llama 3.2 model documentation&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Google DeepMind. &lt;a href="https://ai.google.dev/gemma/docs/core/model_card_3" rel="noopener noreferrer"&gt;Gemma 3 model card&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Microsoft. &lt;a href="https://ai.azure.com/catalog/models/Phi-4-mini-instruct" rel="noopener noreferrer"&gt;Phi-4-mini-instruct model information&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Mistral AI. &lt;a href="https://docs.mistral.ai/models/mistral-7b-0-3" rel="noopener noreferrer"&gt;Mistral 7B v0.3 documentation&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Hinton, G., Vinyals, O., and Dean, J. &lt;a href="https://arxiv.org/abs/1503.02531" rel="noopener noreferrer"&gt;Distilling the Knowledge in a Neural Network&lt;/a&gt;.&lt;/li&gt;
&lt;/ol&gt;

</description>
      <category>nlp</category>
      <category>machinelearning</category>
      <category>llm</category>
      <category>python</category>
    </item>
    <item>
      <title>Vision-Language Models (VLMs): How AI Is Learning to Understand Both Images and Text</title>
      <dc:creator>Mbonimpa Kelia</dc:creator>
      <pubDate>Mon, 26 Oct 2026 12:08:12 +0000</pubDate>
      <link>https://dev.to/mbonimpa_kelia_9/vision-language-models-vlms-how-ai-is-learning-to-understand-both-images-and-text-5j4</link>
      <guid>https://dev.to/mbonimpa_kelia_9/vision-language-models-vlms-how-ai-is-learning-to-understand-both-images-and-text-5j4</guid>
      <description>&lt;h2&gt;
  
  
  Introduction: What If AI Could Both See and Read?
&lt;/h2&gt;

&lt;p&gt;Imagine uploading a photograph of a busy road to an AI system and asking, “What problems can you identify in this picture?” Instead of simply detecting objects, the system might describe the vehicles, identify traffic congestion, read visible road signs and explain what it observes in natural language.&lt;/p&gt;

&lt;p&gt;This is one of the exciting possibilities of &lt;strong&gt;Vision-Language Models (VLMs)&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;As I explored this topic, I discovered that artificial intelligence is no longer limited to processing text or recognising objects in images separately. VLMs are designed to connect visual information with language, allowing machines to answer questions about images, describe scenes and perform tasks that require both visual and textual understanding.&lt;/p&gt;

&lt;p&gt;In this blog, I will explain what VLMs are, how they work, where they are used, and what challenges researchers are still trying to solve.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. What Is a Vision-Language Model?
&lt;/h2&gt;

&lt;p&gt;A Vision-Language Model is an artificial intelligence model designed to process and connect visual information, such as images, with natural language, such as English or Kinyarwanda.&lt;/p&gt;

&lt;p&gt;Traditional computer vision systems often focus on specific tasks, such as detecting cars, identifying faces or classifying objects. Traditional language models, on the other hand, primarily work with text.&lt;/p&gt;

&lt;p&gt;VLMs bring these two areas together.&lt;/p&gt;

&lt;p&gt;For example, if we provide a VLM with a photograph of a fruit market, we could ask:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;What fruits are visible in the picture?&lt;/li&gt;
&lt;li&gt;What is the seller doing?&lt;/li&gt;
&lt;li&gt;Can you read the price written on the sign?&lt;/li&gt;
&lt;li&gt;Describe the scene in one paragraph.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The model uses visual information from the image and language processing to generate an answer.&lt;/p&gt;

&lt;p&gt;This combination makes VLMs useful for tasks such as image captioning, visual question answering and matching images with text.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. How Do Vision-Language Models Work?
&lt;/h2&gt;

&lt;p&gt;During my research, I found that many modern VLMs combine a visual encoder with a language model and a component that connects the two.&lt;/p&gt;

&lt;p&gt;Here is a simplified explanation of the process.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 1: The model receives an image and a question.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;For example, a user uploads a picture of a damaged road and asks, “What can you observe?”&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 2: The visual encoder processes the image.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The visual encoder converts image information into numerical representations that the AI system can process. Depending on the model, it may use a vision transformer or another image-processing architecture.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 3: The visual and language representations are connected.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A connector or projection layer can translate visual features into a representation that the language model can use. The exact architecture differs between models.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 4: The language model generates an answer.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The system combines the visual information with the user's question to produce a text response.&lt;/p&gt;

&lt;p&gt;The result might be: “The image shows a damaged road surface with several visible cracks.”&lt;/p&gt;

&lt;p&gt;This is a simplified example, not a guarantee that every model will interpret the image correctly.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Important Vision-Language Models and Research
&lt;/h2&gt;

&lt;p&gt;My research highlighted several influential models that have helped advance this field.&lt;/p&gt;

&lt;h3&gt;
  
  
  CLIP
&lt;/h3&gt;

&lt;p&gt;CLIP, developed by OpenAI, learns relationships between images and text. It can compare a picture with text descriptions and estimate which description matches best.&lt;/p&gt;

&lt;p&gt;For example, it could compare an image against the descriptions “a dog playing outside” and “a cat sleeping on a sofa.”&lt;/p&gt;

&lt;p&gt;CLIP helped demonstrate how learning from large collections of image-text pairs can support flexible image classification and retrieval.&lt;/p&gt;

&lt;h3&gt;
  
  
  BLIP and BLIP-2
&lt;/h3&gt;

&lt;p&gt;BLIP and BLIP-2 are research models developed to improve vision-language understanding and generation. They support tasks such as image captioning and answering questions about images.&lt;/p&gt;

&lt;p&gt;BLIP-2 is particularly notable for connecting a pretrained vision encoder to a language model through a lightweight intermediary, reducing the need to train every component from scratch.&lt;/p&gt;

&lt;h3&gt;
  
  
  LLaVA
&lt;/h3&gt;

&lt;p&gt;LLaVA is an important open research project in large vision-language models. It combines visual features with a language model and uses visual instruction tuning to improve its ability to follow questions and instructions about images.&lt;/p&gt;

&lt;p&gt;This approach helped make conversational image understanding more accessible to researchers and developers.&lt;/p&gt;

&lt;h3&gt;
  
  
  Modern multimodal assistants
&lt;/h3&gt;

&lt;p&gt;Today's multimodal AI assistants can accept image inputs and respond to questions about their contents. Some systems can also work with multiple images or video, depending on their capabilities.&lt;/p&gt;

&lt;p&gt;Research published by Hugging Face in May 2025 described advances in smaller yet capable models, visual reasoning, video understanding and multimodal AI agents. These developments show that the field is expanding beyond simple image descriptions.&lt;/p&gt;

&lt;p&gt;Sources: &lt;a href="https://huggingface.co/blog/vlms" rel="noopener noreferrer"&gt;Hugging Face — Vision-Language Models Explained&lt;/a&gt; and &lt;a href="https://huggingface.co/blog/vlms-2025" rel="noopener noreferrer"&gt;Vision-Language Models: Better, Faster, Stronger&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Real-World Applications of VLMs
&lt;/h2&gt;

&lt;p&gt;One of the most interesting things I discovered is that VLMs can be useful in many different industries.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Healthcare:&lt;/strong&gt; VLMs can help researchers and qualified medical professionals explore relationships between medical images and written descriptions. However, their outputs require careful validation and should not replace professional medical judgement.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Education:&lt;/strong&gt; A student can upload a diagram, photograph or chart and ask an AI system to explain what it shows. This can make complex learning materials easier to explore.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Agriculture:&lt;/strong&gt; Farmers and agricultural researchers could use image-based systems to investigate visible crop symptoms, identify possible plant problems and obtain explanations. Results should be verified because an image alone may not provide enough information for a reliable diagnosis.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Transportation and infrastructure:&lt;/strong&gt; VLMs could help interpret photographs of roads, bridges and construction sites, describe visible damage and organise inspection reports. For infrastructure planning, this could support human experts by helping them review visual evidence more efficiently.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Accessibility:&lt;/strong&gt; Image-description systems can explain visual scenes to people who have difficulty seeing them. Their usefulness depends on the accuracy and detail of the descriptions.&lt;/p&gt;

&lt;p&gt;These applications illustrate why connecting computer vision with natural language processing is an important direction in AI research.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. VLMs and Traditional Computer Vision: What Is the Difference?
&lt;/h2&gt;

&lt;p&gt;Although both technologies work with images, they are not exactly the same.&lt;/p&gt;

&lt;p&gt;Traditional computer vision systems are often trained to perform a specific task, such as classifying images into predefined categories or detecting objects.&lt;/p&gt;

&lt;p&gt;VLMs connect visual information with language, allowing users to ask different questions about an image and receive natural-language responses. Depending on the architecture, a VLM may also perform object recognition or other computer vision tasks.&lt;/p&gt;

&lt;p&gt;For example, a traditional classifier might label a photograph as “road damage.” A VLM might describe the visible cracks and answer a follow-up question about what is shown.&lt;/p&gt;

&lt;p&gt;However, a specialised computer vision system can still outperform a general-purpose VLM on a particular task. The best choice depends on the application, available data and required level of accuracy.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. The Challenges Facing VLMs
&lt;/h2&gt;

&lt;p&gt;Despite their impressive capabilities, VLMs still have important limitations.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Hallucinations:&lt;/strong&gt; A model may describe an object that is not present or provide an incorrect explanation of an image.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Difficulty with small details:&lt;/strong&gt; Tiny text, crowded scenes, complex diagrams and subtle visual differences can be difficult to interpret.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Bias and fairness:&lt;/strong&gt; If training data does not represent different people, environments and contexts adequately, model performance may vary across those settings.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Computational costs:&lt;/strong&gt; Large models can require substantial memory and processing power, making deployment expensive or difficult on low-resource devices.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Privacy:&lt;/strong&gt; Images may contain faces, personal information, documents or sensitive locations. Developers need appropriate safeguards when collecting and processing visual data.&lt;/p&gt;

&lt;p&gt;A 2026 systematic review of VLM research also identified robustness, interpretability, hallucination control and generalisation to unfamiliar situations as continuing challenges. This suggests that future progress must focus not only on building more capable models, but also on making them more reliable.&lt;/p&gt;

&lt;p&gt;Research reference: &lt;a href="https://www.sciencedirect.com/science/article/pii/S2590005626000627" rel="noopener noreferrer"&gt;A Systematic Review of Vision-Language Models&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  7. What Does the Future Hold for VLMs?
&lt;/h2&gt;

&lt;p&gt;Based on the research I reviewed, several developments are worth watching.&lt;/p&gt;

&lt;p&gt;First, smaller and more efficient models may make visual AI more accessible to developers and organisations with limited computing resources.&lt;/p&gt;

&lt;p&gt;Second, improvements in video understanding could allow AI systems to analyse events across multiple frames rather than interpreting isolated images.&lt;/p&gt;

&lt;p&gt;Third, multimodal AI agents may combine visual understanding with tools and other sources of information to assist with more complex workflows.&lt;/p&gt;

&lt;p&gt;Finally, research into evaluation, safety and multilingual understanding will be important if VLMs are to work reliably across different communities and real-world environments.&lt;/p&gt;

&lt;p&gt;For countries such as Rwanda, potential applications in agriculture, education, road infrastructure and public services make this field especially interesting. These are opportunities to investigate, rather than proof that a particular VLM is already suitable for every local application.&lt;/p&gt;

&lt;h2&gt;
  
  
  Conclusion: AI Is Becoming More Multimodal
&lt;/h2&gt;

&lt;p&gt;My main takeaway from researching Vision-Language Models is that the future of artificial intelligence is increasingly multimodal. Instead of treating images and language as completely separate forms of information, VLMs connect them to support more flexible interactions between humans and machines.&lt;/p&gt;

&lt;p&gt;However, being able to describe an image does not mean that a model understands everything about the situation or always provides correct answers. Reliable applications still require testing, human oversight and attention to privacy and fairness.&lt;/p&gt;

&lt;p&gt;As someone interested in software engineering and AI, I find VLMs particularly exciting because they create opportunities to build applications that interact with the visual world in more natural ways.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The question I am left with is this:&lt;/strong&gt; As vision-language models continue to improve, what new problems could we solve by allowing computers to connect what they see with what we say?&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Hugging Face. &lt;em&gt;Vision Language Models Explained.&lt;/em&gt; &lt;a href="https://huggingface.co/blog/vlms" rel="noopener noreferrer"&gt;https://huggingface.co/blog/vlms&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Hugging Face. &lt;em&gt;Vision Language Models (Better, Faster, Stronger).&lt;/em&gt; &lt;a href="https://huggingface.co/blog/vlms-2025" rel="noopener noreferrer"&gt;https://huggingface.co/blog/vlms-2025&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;OpenAI. &lt;em&gt;Images and Vision — API Documentation.&lt;/em&gt; &lt;a href="https://developers.openai.com/api/docs/guides/images-vision" rel="noopener noreferrer"&gt;https://developers.openai.com/api/docs/guides/images-vision&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;em&gt;A Systematic Review of Vision-Language Models: Comprehensive Analysis of Architectures, Applications, Datasets and Challenges Towards Robust Multimodal Intelligence.&lt;/em&gt; Array, 2026. &lt;a href="https://www.sciencedirect.com/science/article/pii/S2590005626000627" rel="noopener noreferrer"&gt;https://www.sciencedirect.com/science/article/pii/S2590005626000627&lt;/a&gt;
&lt;/li&gt;
&lt;/ol&gt;

</description>
      <category>ai</category>
      <category>machinelearning</category>
      <category>computervision</category>
      <category>multimodalai</category>
    </item>
    <item>
      <title># What I Learned About Computer Vision as a Beginner: My kLab TekHer AI Journey</title>
      <dc:creator>Mbonimpa Kelia</dc:creator>
      <pubDate>Tue, 13 Oct 2026 14:03:25 +0000</pubDate>
      <link>https://dev.to/mbonimpa_kelia_9/-what-i-learned-about-computer-vision-as-a-beginner-my-klab-tekher-ai-journey-155f</link>
      <guid>https://dev.to/mbonimpa_kelia_9/-what-i-learned-about-computer-vision-as-a-beginner-my-klab-tekher-ai-journey-155f</guid>
      <description>&lt;p&gt;A beginner's reflection on the kLab TekHer AI Program — Cohort 2&lt;/p&gt;

&lt;p&gt;When I started the "Introduction to Computer Vision" notebook, I thought I was going to learn how to make computers "see." I was half right. What I actually learned is that computers don't see anything at all — they just do math on grids of numbers. And once that clicked, everything else made sense.&lt;/p&gt;

&lt;p&gt;Here's what I learned, the challenges I ran into, and how I got past them.&lt;/p&gt;

&lt;p&gt;What I Learned&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;An image is just numbers — lots of them
The first lesson was the most surprising. A photo isn't a picture to a computer. It's a giant grid of numbers.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;My photo of the bus was 1080 rows × 810 columns × 3 channels (R, G, B) = 2,624,400 numbers. That's it. No "bus," no "people," no "sunlight" — just over 2.6 million numbers between 0 and 255.&lt;/p&gt;

&lt;p&gt;Each pixel has three values:&lt;/p&gt;

&lt;p&gt;Red, Green, Blue (in that order)&lt;/p&gt;

&lt;p&gt;0 = none of that color, 255 = full of that color&lt;/p&gt;

&lt;p&gt;White = (255, 255, 255). Black = (0, 0, 0).&lt;/p&gt;

&lt;p&gt;That was my first "aha" moment. When I removed the red channel from an image, all the white areas turned cyan. Why? Because white − red = green + blue = cyan. Simple arithmetic, but it changed how I think about photos.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Filters: how a computer finds edges
Next, I learned about filters (also called kernels). A filter is a tiny grid of numbers — usually 3×3 or 5×5 — that slides across the image. At each position, it multiplies the neighboring pixels by the filter's weights and adds the results. That becomes the new value of the center pixel.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Two classic filters I built by hand:&lt;/p&gt;

&lt;p&gt;Filter  What it does&lt;br&gt;
Box blur (all weights = 1/25)   Averages each pixel with its 25 neighbors → softens the image&lt;br&gt;
Sobel x ([-1,0,1], [-2,0,2], [-1,0,1])  Subtracts left side from right side → finds vertical edges&lt;br&gt;
Sobel y (Sobel x transposed)    Finds horizontal edges&lt;br&gt;
When I ran Sobel x on the bus photo, I could see exactly where the vertical edges were — the sides of the bus, the traffic sign, the people's arms. But the horizontal edges (like the roof of the bus) were invisible until I ran Sobel y. Combining both with sqrt(x² + y²) gave me every edge.&lt;/p&gt;

&lt;p&gt;This is what a CNN does. The difference is that a CNN doesn't use hand-picked weights — it learns them from thousands of labeled photos.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Classification: one label for a whole image
I loaded a pretrained YOLO classifier (trained on ImageNet — 1.2 million photos, 1,000 classes) and ran it on the bus photo. It said:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;minibus — 57.1%&lt;/p&gt;

&lt;p&gt;police_van — 33.7%&lt;/p&gt;

&lt;p&gt;trolleybus — 4.2%&lt;/p&gt;

&lt;p&gt;...&lt;/p&gt;

&lt;p&gt;Top guess was correct! But the notebook asked me to test it on something I know well: a Rwandan agaseke (traditional woven basket).&lt;/p&gt;

&lt;p&gt;The result? "vase" at 74.4%.&lt;/p&gt;

&lt;p&gt;The model had never seen an agaseke. It looked at the shape, decided it most resembled a vase, and gave me a confident but wrong answer.&lt;/p&gt;

&lt;p&gt;This was the single most important lesson in the whole notebook.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Object Detection: what and where
I moved on to object detection — a YOLO model (yolo11n.pt) trained on COCO (80 everyday Western classes: person, car, bus, dog, chair, laptop...).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;On the bus photo, it worked beautifully:&lt;/p&gt;

&lt;p&gt;bus — 94%&lt;/p&gt;

&lt;p&gt;person — 89%, 88%, 86%, 62%&lt;/p&gt;

&lt;p&gt;5 boxes, correctly drawn around each object.&lt;/p&gt;

&lt;p&gt;On my agaseke, it drew 2 boxes:&lt;/p&gt;

&lt;p&gt;vase — 45% (covering the whole basket)&lt;/p&gt;

&lt;p&gt;bird — 58% (around the pointed tip of the lid)&lt;/p&gt;

&lt;p&gt;Both wrong. But the boxes themselves were reasonably placed. The model could see where the object was — it just couldn't name it.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Segmentation: pixel-by-pixel
Segmentation is the most detailed level. Instead of a rectangle, the model predicts a mask — a grid of 0s and 1s where 1 means "this pixel belongs to the object."&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;On the bus photo, the segmenter produced clean outlines of the bus and each person. On my agaseke, the mask traced the shape of the basket beautifully — but again, the label was "vase."&lt;/p&gt;

&lt;p&gt;Key insight: the AI could see the shape perfectly. It just had never learned the word "agaseke."&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;When models fail
The last section was the most eye-opening. I took the same photo and made it harder in 5 ways:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Variant Result&lt;br&gt;
Original    5 objects, 4 people — perfect&lt;br&gt;
Very dark   6 objects (with a false "skateboard")&lt;br&gt;
Upside down 1 object, mislabeled as "screen"&lt;br&gt;
Low resolution  0 objects — total failure&lt;br&gt;
Blurry  A few objects, some misses&lt;br&gt;
The upside-down image crushed the model. Why? Because the model's internal filters were trained on photos where "up" is sky and "down" is ground. Flip it and everything breaks.&lt;/p&gt;

&lt;p&gt;The low-resolution image also crushed it. Fine details = the entire basis of detection.&lt;/p&gt;

&lt;p&gt;Lesson: AI is not magic. It's pattern matching. If the pattern is unfamiliar, it fails.&lt;/p&gt;

&lt;p&gt;The Challenges I Faced&lt;br&gt;
Challenge 1: "NameError: name 'np' is not defined"&lt;br&gt;
What happened: I ran a cell that used np (NumPy) and Image (PIL), but the notebook didn't know what those were. The cell crashed.&lt;/p&gt;

&lt;p&gt;Why: Google Colab doesn't remember variables between sessions. When I restarted the runtime (or closed and reopened the notebook), all the import statements were forgotten. The setup cell had to be re-run.&lt;/p&gt;

&lt;p&gt;How I fixed it: I learned the golden rule — always run cells from the top down, in order, one at a time. If you see any NameError, scroll up and look for the cell that defines the missing name.&lt;/p&gt;

&lt;p&gt;Challenge 2: "NameError: name 'convolve' is not defined"&lt;br&gt;
What happened: I added a new cell to test new kernels (sharpen, big blur, Sobel y), but the convolve function — defined in the previous cell — wasn't in memory anymore.&lt;/p&gt;

&lt;p&gt;Why: Same issue as Challenge 1. I had skipped running the earlier cell.&lt;/p&gt;

&lt;p&gt;How I fixed it: Re-ran the cell that starts with def convolve(gray, kernel):. Immediately, my new test cell worked.&lt;/p&gt;

&lt;p&gt;Challenge 3: Colab couldn't find my image&lt;br&gt;
What happened: I wrote IMAGE_PATH = r"C:\Users\PC\Documents\klab\computer science.jpg" and got a FileNotFoundError.&lt;/p&gt;

&lt;p&gt;Why: Google Colab runs on a cloud server. It has no access to your C: drive. Your files are on your laptop; Colab is in Google's data center.&lt;/p&gt;

&lt;p&gt;How I fixed it: I used the correct upload workflow:&lt;/p&gt;

&lt;p&gt;python&lt;br&gt;
from google.colab import files&lt;br&gt;
uploaded = files.upload()&lt;br&gt;
Then clicked "Choose Files," selected my image, and let Colab save it locally. I could then access it by filename (e.g., "Agaseke (1).jpg").&lt;/p&gt;

&lt;p&gt;Bonus lesson: Colab sometimes renames your file — for example, Agaseke.jpg → Agaseke (1).jpg. To avoid hard-coding a wrong name, I used IMAGE_PATH = list(uploaded.keys())[0] to grab whatever filename Colab assigned.&lt;/p&gt;

&lt;p&gt;Challenge 4: The upload widget disappeared&lt;br&gt;
What happened: A message said: "Upload widget is only available when the cell has been executed in the current browser session."&lt;/p&gt;

&lt;p&gt;Why: The runtime had disconnected in the background, so the "Choose Files" button became inactive.&lt;/p&gt;

&lt;p&gt;How I fixed it: Re-ran the upload cell (Shift+Enter). The button came back. If it still didn't work, I could use IMAGE_PATH = "Agaseke (1).jpg" to load the file directly.&lt;/p&gt;

&lt;p&gt;Challenge 5: Not sure whether to edit the existing cell or add a new one&lt;br&gt;
What happened: I wasn't sure whether to change code inside an existing cell or add a new one.&lt;/p&gt;

&lt;p&gt;How I fixed it: I asked, and the answer was:&lt;/p&gt;

&lt;p&gt;If you're learning or experimenting → add a new cell. Safe, reversible.&lt;/p&gt;

&lt;p&gt;If you're fixing a bug → edit the existing cell.&lt;/p&gt;

&lt;p&gt;For the "Try it yourself" tasks → a new cell is usually cleanest.&lt;/p&gt;

&lt;p&gt;Challenge 6: The domain gap — a lesson disguised as a bug&lt;br&gt;
What happened: When I ran the classifier on my agaseke, it said "vase, 74.4%". I thought I had done something wrong.&lt;/p&gt;

&lt;p&gt;Why: I hadn't. The model simply had never been trained on agaseke. It was doing exactly what it was supposed to — finding the closest thing it knew.&lt;/p&gt;

&lt;p&gt;How I "fixed" it: I didn't. I learned from it. This is one of the most important unsolved problems in AI: a model trained on American photos will fail on Rwandan objects, even when it's technically "working."&lt;/p&gt;

&lt;p&gt;What I Would Do Differently Next Time&lt;br&gt;
Always run cells from the top before starting work.&lt;/p&gt;

&lt;p&gt;Keep a "session check" cell at the top of my notebook that prints which image is loaded:&lt;/p&gt;

&lt;p&gt;python&lt;br&gt;
print("IMAGE_PATH =", IMAGE_PATH)&lt;br&gt;
print("img shape =", img.shape)&lt;br&gt;
Save intermediate copies of variables (like my_img = img.copy()) so I don't lose my original image when experimenting.&lt;/p&gt;

&lt;p&gt;Predict before running. Every "Try it yourself" became more interesting when I stopped and thought about what would happen first.&lt;/p&gt;

&lt;p&gt;Final Reflection&lt;br&gt;
This notebook took me from "an image is a picture" to "an image is 2,624,400 numbers, filtered and classified by learned patterns." Along the way, I:&lt;/p&gt;

&lt;p&gt;Manipulated RGB channels&lt;/p&gt;

&lt;p&gt;Built blur and edge-detection filters&lt;/p&gt;

&lt;p&gt;Ran a real ImageNet classifier&lt;/p&gt;

&lt;p&gt;Ran a real COCO object detector&lt;/p&gt;

&lt;p&gt;Ran a real segmentation model&lt;/p&gt;

&lt;p&gt;Discovered the domain gap in person, using an agaseke.&lt;/p&gt;

&lt;p&gt;The hardest part wasn't the code. It was realizing that the AI is doing exactly what I tell it to — I just have to give it training data that represents my world.&lt;/p&gt;

&lt;p&gt;My biggest takeaway: If we want AI that works in Rwanda, we need to build it with Rwandan data. And that's the next thing I want to learn.&lt;/p&gt;

&lt;p&gt;— Written after completing the "Introduction to Computer Vision" notebook, kLab TekHer AI Program, Cohort 2.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>beginners</category>
      <category>learning</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>What I Learned About Word Embeddings While Building a FAQ Chatbot</title>
      <dc:creator>Mbonimpa Kelia</dc:creator>
      <pubDate>Wed, 30 Sep 2026 13:26:50 +0000</pubDate>
      <link>https://dev.to/mbonimpa_kelia_9/what-i-learned-about-word-embeddings-while-building-a-faq-chatbot-350c</link>
      <guid>https://dev.to/mbonimpa_kelia_9/what-i-learned-about-word-embeddings-while-building-a-faq-chatbot-350c</guid>
      <description>&lt;p&gt;When I set out to build a simple FAQ chatbot for Python programming questions, I thought I'd be done in a weekend. I was wrong — but in the best way. Along the way, I ran into a concept that changed how I think about text: word embeddings.&lt;/p&gt;

&lt;p&gt;This post is a lessons-learned write-up of what I discovered. If you're a developer getting into NLP, this might save you a few hours of confusion.&lt;/p&gt;

&lt;p&gt;What Are Word Embeddings, Anyway?&lt;br&gt;
Computers don't understand words. They understand numbers. So any NLP system has to convert text into numbers before it can do anything useful.&lt;/p&gt;

&lt;p&gt;The naive way to do this is one-hot encoding or bag-of-words: assign each word a number, and represent a sentence as a list of word counts. This works, but it has a big problem — it treats every word as completely independent. To a bag-of-words model, "car" and "automobile" are as unrelated as "car" and "banana."&lt;/p&gt;

&lt;p&gt;Word embeddings solve this. An embedding represents each word as a dense vector of real numbers — usually 100 to 300 dimensions — where the position of the vector in space reflects the word's meaning. Words that appear in similar contexts end up close together in this vector space. "Dog" and "puppy" end up near each other. "King" and "queen" end up near each other. "Python" and "programming" end up near each other.&lt;/p&gt;

&lt;p&gt;Why does this matter? Because once words are vectors, you can do math on them. You can measure similarity, find clusters, and even solve analogies.&lt;/p&gt;

&lt;p&gt;The Method That Clicked for Me: Word2Vec&lt;br&gt;
The assignment notes I was working from mentioned Word2Vec, and it's the one that finally made embeddings click.&lt;/p&gt;

&lt;p&gt;Word2Vec comes in two flavors:&lt;/p&gt;

&lt;p&gt;CBOW (Continuous Bag of Words): predicts a word from its surrounding context&lt;/p&gt;

&lt;p&gt;Skip-gram: predicts the surrounding context from a word&lt;/p&gt;

&lt;p&gt;The key insight is the distributional hypothesis: words that appear in similar contexts have similar meanings. If "cat" and "dog" both frequently appear near "pet," "feed," and "vet," Word2Vec learns that they're semantically related.&lt;/p&gt;

&lt;p&gt;Training happens by sliding a window over a large corpus. For each word, the model tries to predict its neighbors. Over millions of examples, the weights of a small neural network adjust until each word has a useful vector. What surprised me is that nobody labels this data — the text itself is the supervision. You just throw a lot of text at it and it figures out meaning on its own.&lt;/p&gt;

&lt;p&gt;The Analogy That Blew My Mind&lt;br&gt;
The classic Word2Vec demo is this:&lt;/p&gt;

&lt;p&gt;text&lt;br&gt;
king - man + woman ≈ queen&lt;br&gt;
Translated: take the vector for "king," subtract "man," add "woman" — and the closest vector to the result is "queen." The model wasn't told that kings and queens are related, or that man and woman are a gender pair. It learned both facts from raw text alone.&lt;/p&gt;

&lt;p&gt;I tried to reproduce this with a Python example using gensim:&lt;/p&gt;

&lt;p&gt;python&lt;br&gt;
from gensim.models import Word2Vec&lt;/p&gt;

&lt;h1&gt;
  
  
  Assume &lt;code&gt;sentences&lt;/code&gt; is a list of tokenized sentences
&lt;/h1&gt;

&lt;p&gt;model = Word2Vec(sentences, vector_size=100, window=5, min_count=5)&lt;/p&gt;

&lt;h1&gt;
  
  
  Find words most similar to "python"
&lt;/h1&gt;

&lt;p&gt;similar = model.wv.most_similar("python", topn=5)&lt;br&gt;
print(similar)&lt;/p&gt;

&lt;h1&gt;
  
  
  [('programming', 0.71), ('java', 0.68), ('code', 0.65), ...]
&lt;/h1&gt;

&lt;p&gt;The fact that a model trained on raw Wikipedia text knows "python" is close to "programming" still amazes me.&lt;/p&gt;

&lt;p&gt;What I Struggled With&lt;br&gt;
Here's the honest part: I did not use word embeddings in my final FAQ chatbot.&lt;/p&gt;

&lt;p&gt;My project used TF-IDF, which is a sparse, frequency-based vectorization method — not an embedding. TF-IDF is fast, interpretable, and works great for small FAQ datasets. But it has a real limitation: it can't handle paraphrases.&lt;/p&gt;

&lt;p&gt;For example, if a user asks "How do I set up Python?" and my FAQ has "How do I install Python?", TF-IDF sees "set up" and "install" as completely different words. It might miss the match. A word embedding model would recognise that "set up" and "install" mean the same thing and would match them correctly.&lt;/p&gt;

&lt;p&gt;So I learned embeddings as theory but implemented TF-IDF in practice. If I were to expand this project to hundreds or thousands of FAQs, the very first upgrade I'd make is swapping TF-IDF for sentence embeddings like Sentence-BERT, which can encode entire questions into a single dense vector and handle paraphrasing gracefully.&lt;/p&gt;

&lt;p&gt;Other Embedding Methods Worth Knowing&lt;br&gt;
While learning, I also came across:&lt;/p&gt;

&lt;p&gt;GloVe (Global Vectors): similar to Word2Vec but trained on global word co-occurrence statistics rather than a sliding window.&lt;/p&gt;

&lt;p&gt;FastText: extends Word2Vec by treating words as bags of character n-grams. This means it can generate vectors for words it has never seen — very handy for typos or rare words.&lt;/p&gt;

&lt;p&gt;Contextual embeddings (BERT, ELMo): the same word gets different vectors in different sentences. "Bank" in "river bank" vs. "money bank" gets two different vectors. This is the foundation of modern LLMs.&lt;/p&gt;

&lt;p&gt;What I'd Tell Another Developer&lt;br&gt;
If you're just starting with NLP:&lt;/p&gt;

&lt;p&gt;Learn TF-IDF first. It's simpler, faster, and good enough for many tasks.&lt;/p&gt;

&lt;p&gt;Learn Word2Vec next. It builds the intuition for why embeddings work.&lt;/p&gt;

&lt;p&gt;Then try sentence embeddings (SBERT). This is the practical modern approach for semantic search.&lt;/p&gt;

&lt;p&gt;Don't skip the theory. Understanding why embeddings work — the distributional hypothesis — makes it much easier to know when to use them.&lt;/p&gt;

&lt;p&gt;Embeddings turned text into geometry. That's the mental shift that changed how I look at NLP problems.&lt;/p&gt;

&lt;p&gt;Sources:&lt;/p&gt;

&lt;p&gt;Mikolov et al., "Efficient Estimation of Word Representations in Vector Space" (2013)&lt;/p&gt;

&lt;p&gt;Pennington et al., "GloVe: Global Vectors for Word Representation" (2014)&lt;/p&gt;

&lt;p&gt;Bojanowski et al., "Enriching Word Vectors with Subword Information" (FastText, 2017)&lt;/p&gt;

&lt;p&gt;Reimers &amp;amp; Gurevych, "Sentence-BERT" (2019)&lt;/p&gt;

&lt;p&gt;Dev.to community posts on TF-IDF vs. embeddings&lt;/p&gt;

</description>
      <category>nlp</category>
      <category>machinelearning</category>
      <category>python</category>
    </item>
  </channel>
</rss>
