DEV Community: Marie Claire Uwiringiyimana

Improving InkMind: Selecting an Open Large Language Model for AI-Powered Poetry Generation

Marie Claire Uwiringiyimana — Mon, 26 Oct 2026 14:30:11 +0000

Author: Marie Claire Uwiringiyimana
Project: InkMind — AI-Powered Poetry Generation
Field: Natural Language Processing (NLP)

1. Introduction

Natural Language Processing (NLP) is a branch of Artificial Intelligence that enables computers to process, understand, and generate human language. Recent developments in Large Language Models (LLMs) have significantly improved the ability of computers to produce meaningful text, answer questions, and support creative writing.

As part of my NLP learning journey, I worked on InkMind, a poetry generation project that transforms a simple word or idea into a complete poem. For example, a user can enter the word "Love," and the system generates a poem inspired by that theme.

Our current implementation uses GPT-2, a pretrained language model developed by OpenAI. Although GPT-2 provides a foundation for generating poems, we want to explore whether another modern language model can produce more coherent, creative, and relevant poetry.

To investigate this possibility, I reviewed the Botpress article The 10 Best Large Language Models (LLMs) in 2026 and examined the official documentation of several open-weight models. Based on this research, I selected Qwen3-4B-Instruct-2507 as the proposed alternative to GPT-2.

This blog explains the models considered, why Qwen3 is suitable for InkMind, and how we will compare the two models using the same prompts.

2. Understanding Large Language Models

Large Language Models are neural networks trained on large collections of text to learn patterns in human language. Many modern LLMs use the Transformer architecture and generate text by predicting the next token based on the preceding context.

A token may represent a complete word, part of a word, or another text unit. By repeatedly predicting and generating tokens, a language model can produce sentences, paragraphs, stories, and poems.

For example, InkMind receives the following input:

User prompt: Love

The model then generates a sequence of text inspired by that word, potentially producing several poetic lines and stanzas.

Different models may generate different poems from the same prompt because they differ in their training, architecture, parameter count, instruction-following capabilities, and generation settings.

According to Botpress, LLMs have different strengths, limitations, and deployment requirements. Therefore, selecting a model should depend on the intended application rather than simply choosing the largest or most popular model (Botpress, n.d.).

For InkMind, the most important requirements are creative text generation, relevance to the user's prompt, coherent poetry, accessible model weights, and manageable computing requirements.

3. Existing Model: GPT-2

GPT-2 is the model currently used in InkMind. Developed by OpenAI, it is a decoder-only Transformer language model pretrained on English text to predict the next token in a sequence.

The smallest GPT-2 model has approximately 124 million parameters. It can generate text from a prompt and can be further adapted to specific tasks through fine-tuning (OpenAI Community, n.d.).

Advantages and Limitations of GPT-2

✅ Advantages	⚠️ Limitations
It is relatively lightweight and suitable for experimentation on modest computing resources.	It may generate repetitive or incoherent text.
Its pretrained weights are available for download.	It does not consistently follow detailed instructions because the standard model is not instruction-tuned.
It can generate text without requiring a paid API for each generation.	Its output may lose the original theme as a poem becomes longer.
It can be integrated into Python applications using libraries such as Hugging Face Transformers.	Its original training does not guarantee good poetic structure, rhythm, or emotional depth.
It provides a useful baseline for evaluating alternative language models.

These limitations do not mean GPT-2 cannot generate good poems. Instead, they provide reasons to investigate whether a newer model can improve InkMind's output.

Official model documentation: GPT-2 on Hugging Face

4. Researching Alternative Large Language Models

Before selecting a replacement, I considered several models that offer different capabilities and deployment options.

4.1 GPT-4o

GPT-4o is a general-purpose model designed to support tasks such as writing, reasoning, and instruction following. It could generate poems from short prompts and follow additional instructions concerning tone, style, and structure.

For example, a user could enter "Love" and request a romantic poem containing four stanzas and vivid imagery.

However, using a hosted model through an API may involve usage charges and requires dependence on an external service. Its model weights are not available for ordinary local deployment.

Although GPT-4o could be useful for a future version of InkMind, I did not select it because our experiment prioritizes an open-weight model that we can download and run ourselves.

4.2 Gemma 3 (4B)

Gemma 3 is a family of language models developed by Google. Its four-billion-parameter version is a potential candidate for creative text generation and local experimentation.

It offers an alternative to GPT-2, but its resource requirements are higher. In addition, the applicable Gemma model terms must be reviewed before choosing it for deployment.

Gemma 3 remains a relevant candidate, but I selected Qwen3 because its specific model card documents open-ended text-generation improvements and its Apache 2.0 licence provides a clear basis for experimentation and adaptation.

4.3 Mistral Small 3.1

Mistral Small 3.1 is another model discussed in the Botpress comparison. It offers general-purpose language capabilities and open-weight deployment options.

It could be evaluated for creative writing, but its computing requirements may be more demanding than those of smaller models.

Since InkMind currently performs a focused task—generating poems from short prompts—we do not need to begin with a larger model unless our experiments demonstrate that its capabilities justify the additional resources.

4.4 Qwen3-4B-Instruct-2507

Qwen3-4B-Instruct-2507 is an instruction-tuned, four-billion-parameter language model in the Qwen3 family.

Its official model card describes improvements in instruction following, text comprehension, multilingual capabilities, and open-ended tasks. It is distributed under the Apache 2.0 licence, which permits use and modification subject to the licence's conditions (Qwen, n.d.).

These characteristics make Qwen3 a promising alternative for InkMind.

Unlike the standard GPT-2 model, which primarily continues a sequence of text, Qwen3-4B-Instruct has been adapted to respond to instructions. This may help it interpret a creative request and produce a poem that follows the requested theme and structure.

For example, the model can receive the following prompt:

Write a complete poem about love. Use four stanzas, emotional language, vivid imagery, and a hopeful ending.

The model can then attempt to generate a poem that follows those requirements.

However, instruction tuning does not automatically guarantee superior poetry. We must evaluate the actual outputs produced by Qwen3 and GPT-2 using the same prompts and assessment criteria.

Official model card: Qwen3-4B-Instruct-2507 on Hugging Face

5. Comparing the Models for InkMind

The following table summarizes the models considered during my research.

Model	Main strength	Main limitation	Suitability for InkMind
GPT-2	Lightweight text generation	May struggle with coherence and detailed instructions	Existing baseline
GPT-4o	General-purpose generation and instruction following	Hosted API dependency and potential usage costs	Alternative requiring an external service
Gemma 3 (4B)	Modern, relatively compact model	Higher resource requirements than GPT-2	Potential local alternative
Mistral Small 3.1	General-purpose language generation	More demanding deployment requirements	Candidate for later evaluation
Qwen3-4B-Instruct-2507	Instruction following and open-ended generation	Requires more computing resources than GPT-2	Selected alternative

The comparison is based on the models' documented characteristics, not on a completed benchmark of InkMind. Therefore, the table identifies promising candidates rather than claiming that one model has already won the poetry-generation experiment.

6. My Selected Model: Qwen3-4B-Instruct-2507

After considering the available alternatives, I selected Qwen3-4B-Instruct-2507 as the proposed replacement model for InkMind's GPT-2 implementation.

My choice is based on the following reasons.

6.1 Open-Weight Availability

Qwen3-4B-Instruct-2507 provides downloadable model weights. This allows us to experiment with the model in our own environment instead of depending entirely on a commercial text-generation API.

Its Apache 2.0 licence also permits broad use and adaptation, subject to the licence terms.

This is important because we want to understand how the model works, integrate it into our application, and evaluate its output directly.

6.2 Better Instruction Following

InkMind currently accepts a simple word as a prompt. In the future, we may want users to specify a poem's mood, length, style, or structure.

Qwen3-4B-Instruct is designed to follow instructions, making it a promising candidate for supporting both simple prompts and more detailed creative requests.

For example, users could generate a poem about love and then request a sad, romantic, inspirational, or humorous version.

6.3 Potential for Improved Creative Writing

The Qwen3 model card reports improvements in open-ended text generation and alignment with user preferences. These capabilities suggest that it is worth evaluating for poetry generation.

However, the model has not been specifically proven superior for InkMind's poetry task. We will determine whether it produces better results through our own experiment.

6.4 Flexibility for Future Development

Because Qwen3 can be run using supported local inference tools and adapted where appropriate, it gives us flexibility as InkMind develops.

We could later explore more advanced prompting, poetry-specific fine-tuning, or additional generation controls without necessarily changing the overall purpose of the application.

6.5 A Suitable Research Comparison

Comparing GPT-2 with Qwen3 will help us investigate how a lightweight pretrained language model differs from a newer instruction-tuned model.

The experiment will also help us understand the relationship between model size, instruction following, output quality, and computing requirements.

Although Qwen3 has substantially more parameters than GPT-2 small, a larger parameter count alone does not prove that it will generate better poems. The quality of the output and the cost of generating it must both be considered.

7. How We Will Compare GPT-2 and Qwen3

To determine whether Qwen3 improves InkMind, we will conduct a controlled comparison.

7.1 Experimental Procedure

Step 1: Prepare the prompts.

We will use the same five prompts for both models:

Love
Hope
Nature
Friendship
Dreams

Step 2: Generate poems using GPT-2.

We will enter each prompt into the existing InkMind implementation and save the generated poems.

Step 3: Generate poems using Qwen3.

We will provide the same prompts to Qwen3 and save its generated poems.

Step 4: Keep the comparison fair.

We will use consistent output-length targets where possible, record the generation settings, and distinguish the effects of model choice from the effects of prompting.

Because GPT-2 is a base model and Qwen3 is instruction-tuned, the prompt formats may differ. We will document any formatting differences while keeping the requested poem topic and evaluation requirements equivalent.

Step 5: Evaluate the outputs.

We will compare the poems according to relevance, coherence, creativity, repetition, poetic structure, and generation time.

7.2 Evaluation Criteria

Criterion	What we will evaluate
Prompt relevance	Does the poem reflect the original word?
Coherence	Do the lines communicate a consistent idea?
Creativity	Does the poem use interesting language and imagery?
Repetition	Does it repeat lines, phrases, or ideas unnecessarily?
Poetic structure	Are the line breaks and stanzas appropriate?
Generation time	How long does each model take to produce the poem?

We will use human evaluation to assess creative qualities that cannot be fully captured by automated metrics.

7.3 Results Table

The following table will be completed after running the experiment.

Prompt	GPT-2 output	Qwen3 output	Preferred output
Love	To be recorded	To be recorded	To be evaluated
Hope	To be recorded	To be recorded	To be evaluated
Nature	To be recorded	To be recorded	To be evaluated
Friendship	To be recorded	To be recorded	To be evaluated
Dreams	To be recorded	To be recorded	To be evaluated

This approach will allow us to make a decision based on observed results rather than assumptions.

8. Challenges and Limitations

Although Qwen3 is a promising choice, several limitations should be considered.

First, its four billion parameters make it considerably larger than GPT-2 small. Running the model locally requires more memory and computing resources, and actual performance will depend on the hardware and model precision used.

Second, instruction following does not guarantee that every generated poem will be coherent, original, or artistically meaningful. Human review remains necessary.

Third, both models can produce repetitive language or outputs that do not fully match the user's intended meaning.

Finally, the experiment will evaluate only a small set of prompts. A larger dataset and repeated evaluations would provide stronger evidence about overall performance.

These limitations will be documented so that the final model selection reflects both output quality and practical deployment considerations.

9. Conclusion

This research examined several Large Language Models that could potentially improve InkMind, an NLP project designed to generate complete poems from simple word prompts.

After considering GPT-2, GPT-4o, Gemma 3, Mistral Small 3.1, and Qwen3-4B-Instruct-2507, I selected Qwen3-4B-Instruct-2507 as the proposed alternative to our existing GPT-2 model.

The main reasons for this selection are its downloadable weights, Apache 2.0 licence, instruction-following capabilities, and documented improvements in open-ended text generation.

Our next step is to integrate Qwen3 into the experimental version of InkMind and compare its output directly with GPT-2. By using the same prompts and evaluation criteria, we can investigate whether the new model generates more relevant, coherent, creative, and varied poems.

The experiment will help us make an evidence-based decision about the best model for InkMind while strengthening our practical understanding of NLP and modern language models.

References

Botpress. (n.d.). The 10 best large language models (LLMs) in 2026.
OpenAI Community. (n.d.). GPT-2 model card. Hugging Face.
Qwen. (n.d.). Qwen3-4B-Instruct-2507. Hugging Face.
Qwen. (n.d.). Qwen3. Hugging Face.

From Pixels to Segmentation: My Journey into Computer Vision

Marie Claire Uwiringiyimana — Tue, 13 Oct 2026 14:40:53 +0000

A research-based learning journey exploring how computers process and interpret images, from pixels and RGB colour channels to mathematical kernels, image filters, object classification, detection, and segmentation. In this blog, I share the concepts I researched, the mathematical foundations I explored, and the key insights I gained while learning how computer vision works.

1. Introduction
Computer vision is a field of artificial intelligence that enables computers to extract useful information from images and videos. A computer does not see an image as a person does; instead, it receives numerical values arranged in a grid.
Image processing applies mathematical operations to these values to improve an image or reveal features such as edges and textures. Deep-learning models can then use these features for tasks such as image classification, object detection, and image segmentation (Gonzalez & Woods, 2018; Szeliski, 2022).
2. Understanding Pixels and RGB Images
2.1 What is a pixel?
A pixel, meaning picture element, is the smallest individual sample in a digital image. An image consists of many pixels arranged in rows and columns.
In an 8-bit grayscale image, each pixel usually stores an intensity value between 0 and 255:
0: Black
255: White
Values between 0 and 255: Different shades of gray
A grayscale image can be represented mathematically as: I(x,y)
Where:
(I) represents the image.
(x) represents the row position.
(y) represents the column position.
(I(x,y)) represents the intensity of the pixel at that position.
2.2 Understanding RGB images
RGB stands for Red, Green, and Blue. It is a colour model that represents colours by combining three colour channels.
Each pixel in a standard 8-bit RGB image contains three values:
I(x,y)= [R(x,y), G(x,y), B(x,y)]

Where:
R(x,y) represents the red intensity.
G(x,y) represents the green intensity.
B(x,y) represents the blue intensity.
Each channel usually ranges from 0 to 255.
Examples:
RGB value Colour
(255, 0, 0) Red
(0, 255, 0) Green
(0, 0, 255) Blue
(255, 255, 255) White
(0, 0, 0) Black
(255, 255, 0) Yellow

An RGB image can therefore be represented as a three-dimensional array with dimensions: H×W×3
Where H is the image height, W is its width, and 3 represents the RGB channels.
2.3 Converting RGB to grayscale
Sometimes, image processing algorithms operate on grayscale images because they need intensity information rather than colour information.
A common luminance-weighted conversion formula is:
(Gray=0.299R+0.587G+0.114B)

Where (R), (G), and (B) are the red, green, and blue channel values.
Example:
Suppose a pixel has RGB values:
R=100, G=150, B=200
Substituting these values:
Gray = 0.299(100) + 0.587(150) + 0.114(200)
Gray = 29.9 + 88.05 + 22.8
Gray = 140.75 ~ 141

The resulting grayscale pixel has an intensity of approximately 141.
This weighted formula is associated with standard video luminance calculations, including ITU-R BT.601 (International Telecommunication Union, 2011).

3. Understanding Kernels and Filters
3.1 What is a kernel?
A kernel, also called a filter, is a small matrix of numerical values applied to an image to modify its appearance or extract useful features.
For example, a 3×3 kernel contains nine values:

           [ 1  1  1 ]
      K =  [ 1  1  1 ]
           [ 1  1  1 ]

The kernel moves across the image. At each position, its values are multiplied by the corresponding neighbouring pixel values, and the results are added together.
This operation is used for blurring, sharpening, edge detection, and other image-processing tasks.

3.2 General mathematical formula for convolution
A two-dimensional convolution can be expressed as:
O(x, y) = Σᵢ Σⱼ K(i, j) I(x − i, y − j)
Where:
O(x,y) is the output pixel.
I(x,y) is the input image.
K(i,j) represents the kernel values.
i,j represent positions within the kernel.
∑ indicates summation over the kernel positions.
Strict mathematical convolution flips the kernel before applying it. Some image-processing libraries instead perform cross-correlation, which uses the kernel without flipping it.

3.3 Common image kernels and filters
A. Average blur filter
Kernel

                [ 1  1  1 ]
      K = 1/9 × [ 1  1  1 ]
                [ 1  1  1 ]

This filter calculates the average of a pixel and its eight neighbours. It smooths an image and reduces small details and noise.

The output pixel is:
O(x, y) = (1/9) Σᵢ₌₋₁¹ Σⱼ₌₋₁¹ I(x + i, y + j)

B. Large blur filter (15 × 15)
Kernel

Large Blur Kernel (15 × 15)

               [ 1  1  ...  1 ]
         1
K =  ─────── ×
       225     [ 1  1  ...  1 ]
               [ ⋮  ⋮   ⋱  ⋮ ]
               [ 1  1  ...  1 ]  ₁₅ₓ₁₅

This filter averages 225 neighbouring pixels. It produces stronger smoothing than a 3×3 blur and removes more fine details.
The output pixel is:
O(x, y) = (1/225) Σᵢ₌₋₇⁷ Σⱼ₌₋₇⁷ I(x + i, y + j)

C. Sharpening filter
Kernel


      [  0  -1   0 ]
 K =  [ -1   5  -1 ]
      [  0  -1   0 ]

The sharpening filter strengthens the central pixel and subtracts the four directly adjacent pixels. This emphasizes local changes and makes details appear sharper.

The output is:
O(x, y) = 5I(x, y)
− I(x − 1, y) − I(x + 1, y)
− I(x, y − 1) − I(x, y + 1)

D. Sobel X filter
Kernel

     [ -1   0   1 ]
Gₓ = [ -2   0   2 ]
     [ -1   0   1 ]

The Sobel X filter estimates intensity changes along the horizontal direction. It commonly highlights vertical edges in an image.

The response is:
Gₓ = Kₓ * I
Here, * denotes convolution.

E. Sobel Y filter
Kernel

     [ -1  -2  -1 ]
Gᵧ = [  0   0   0 ]
     [  1   2   1 ]

The Sobel Y filter estimates intensity changes along the vertical direction. It commonly highlights horizontal edges.
The response is:
Gᵧ = Kᵧ * I

F. Gradient magnitude filter
After calculating the Sobel X and Sobel Y responses, they can be combined to estimate the strength of an edge.
G = √(Gₓ² + Gᵧ²)
Where:
Gₓ is the horizontal gradient response.
Gᵧ is the vertical gradient response.
G is the gradient magnitude.
Larger gradient magnitudes indicate stronger intensity changes, which often correspond to image edges.

3.4 Summary of common filters

Filter	Main purpose
Average blur	Smooths an image and reduces small details.
Large blur	Produces stronger smoothing.
Sharpening	Emphasizes details and local changes.
Sobel X	Highlights predominantly vertical edges.
Sobel Y	Highlights predominantly horizontal edges.
Gradient magnitude	Combines edge responses to estimate edge strength.

4. From Image Processing to Computer Vision
Traditional image-processing filters operate directly on pixel values. Modern computer-vision systems often use convolutional neural networks (CNNs), which learn useful filters from training data.
Early network layers may learn edges and textures, while deeper layers combine these features into more complex patterns.
Three important computer-vision tasks are image classification, object detection, and image segmentation.

4.1 Image classification
Question: What is in the image?
Image classification assigns one or more labels to an entire image.
For example, a model might classify an image as:
A dog
A car
A building
A landscape
Classification generally does not identify the precise location of each object.

4.2 Object detection
Question: What objects are present, and where are they located?
Object detection identifies objects and estimates their locations, commonly using bounding boxes.
For example, a traffic image might contain:
Three cars
Two pedestrians
One bicycle
The model returns the object classes and their corresponding locations.
4.3 Image segmentation
Question: Which pixels belong to each object or region?
Image segmentation assigns labels to individual pixels, allowing a computer to identify the precise regions occupied by objects.
Two common forms are:
Semantic segmentation: Assigns a class to every pixel, such as road, building, person, or vehicle.
Instance segmentation: Separates individual objects, including different objects belonging to the same class.
For example, in a street photograph, segmentation can label the pixels belonging to roads, buildings, vehicles, and pedestrians.

4.4 Comparing the three tasks

Task	Output	Example
Classification	Image-level label	"This is a street scene."
Object detection	Labels and bounding boxes	"There is a car at this location."
Segmentation	Pixel-level labels or masks	"These pixels belong to the car."

The main difference is the level of spatial detail: classification identifies the overall content, detection locates objects, and segmentation identifies their regions at pixel level.

5. What I Learned
All of these things I learned about computer vision helped me understand how computers process and interpret images, from analyzing individual pixels and RGB colour channels to applying mathematical kernels and filters to extract useful visual information.
Through these concepts and practical experiments, I learned how image filters can smooth images, sharpen details, and detect edges. I also understood the differences between image classification, object detection, and image segmentation, and how each technique helps computers understand visual information at a different level.
Working with pretrained models also helped me recognize their limitations. I learned that models can produce incorrect predictions when an image contains information that differs from the patterns or categories learned during training. For this reason, confidence scores should not automatically be considered proof that a prediction is correct.
Most importantly, this learning experience helped me connect mathematical concepts with practical applications. I realized that understanding computer vision requires not only learning the theory but also experimenting with real images, analyzing model outputs, and critically evaluating the results.

6. Conclusion
Pixels and colour channels form the numerical foundation of digital images. RGB represents colour using red, green, and blue intensity values, while grayscale represents image intensity using a single channel.
Kernels and filters apply mathematical operations to local groups of pixels. Blur filters smooth images, sharpening filters emphasize details, and Sobel filters estimate edge information.
These fundamental operations help explain how computers process visual information. Building on this foundation, image classification identifies image content, object detection locates objects, and image segmentation assigns labels to individual pixels.
Understanding these concepts provides a strong foundation for studying image processing, convolutional neural networks, and modern computer-vision applications.

7. References
Gonzalez, R. C., & Woods, R. E. (2018). Digital image processing (4th ed.). Pearson.

International Telecommunication Union. (2011). Studio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratios (ITU-R Recommendation BT.601). https://www.itu.int/rec/R-REC-BT.601

Sobel, I., & Feldman, G. (1968). A 3×3 isotropic gradient operator for image processing. Stanford Artificial Intelligence Project.

Szeliski, R. (2022). Computer vision: Algorithms and applications (2nd ed.). Springer. https://szeliski.org/Book/

From Words to Vectors: Understanding Word Embeddings with FastText

Marie Claire Uwiringiyimana — Wed, 30 Sep 2026 13:04:30 +0000

When I started learning Natural Language Processing (NLP), one question I had was, "How can a computer work with words when computers understand numbers better than human language?"

This is where word embeddings come in. Word embeddings are a way of representing words as numerical vectors. Instead of giving a machine only the word "cat," we represent it using a list of numbers. These numbers allow machine-learning models to work with words mathematically and learn relationships between them [1].

For example, a word could have a representation like

cat → [0.21, 0.45, -0.13, 0.72, ...]

A real embedding can contain hundreds of numbers. The important thing is not what one individual number means, but the position of the whole vector in the vector space. Words that have similar meanings or are used in similar contexts can have similar vector representations [1].

Why Do Word Embeddings Matter?

Before learning about embeddings, I looked at one-hot encoding. With one-hot encoding, every word is represented using a vector containing mostly zeros and one value of 1.

For example:

cat → [1, 0, 0]
dog → [0, 1, 0]
banana → [0, 0, 1]

This tells the computer that the words are different, but it does not show that "cat" and "dog" are more related than "cat" and "banana."

Word embeddings provide a better representation because they can learn relationships between words from how those words are used in text. They also use dense vectors with fewer dimensions than a vocabulary-sized one-hot representation, making them more useful for machine-learning applications [1].

This is important for NLP tasks such as text classification, sentiment analysis, search, machine translation, and other applications that need numerical representations of language [1].

A famous example of relationships captured by word embeddings is:

king - man + woman ≈ queen

This does not mean that every embedding model will produce this exact result, but it shows how vector representations can capture useful relationships between words [2].

The Embedding Method I Chose: FastText

FastText is based on the skip-gram approach and extends the idea of word embeddings by using subword information. Instead of looking at a word only as a complete unit, FastText also represents a word using character n-grams [3].

A character n-gram is simply a group of consecutive characters.

For example, parts of:

playing

can include:

pla
lay
ayi
yin
ing

The exact n-grams depend on the model settings.

This is useful because related words often share parts of their spelling:

play
played
playing
player
plays

FastText can learn information from these shared character patterns. This is especially useful for rare words and words that were not directly seen during training [3].

My Python Experiment

I used the gensim Python library to train a small FastText model on a few example sentences. The model was configured with 100-dimensional word vectors, a context window of 3, and the Skip-gram approach.

After training, I used most_similar() to find the words closest to “playing”. In my experiment, “plays” had the highest similarity score, followed by “played”, “love”, “scored”, and “player”. The similarity scores show how close their learned vector representations are.

I also inspected the vector generated for “playing”. The model represents the word as a list of numerical values rather than as text. Since I used vector_size=100, the complete representation contains 100 values.

What I Found Interesting

I then tested “playingly”, a word that was not included in my original training sentences. FastText was still able to generate a vector for it. This helped me understand one of the main advantages of FastText: it uses character n-grams (subword information) to build word representations. This means the model can make use of parts of a word, even when the complete word was not seen during training [3][4].

The screenshots below show the actual code and outputs from my Jupyter Notebook.

            **Figure 1**

What I Learned

Before working with embeddings, I thought a vector was simply a list of numbers and did not understand how those numbers could represent language.

One thing I learned is that we should not try to give every individual number a specific human meaning. The whole vector is what matters.

I also initially found character n-grams confusing. I was thinking about words as complete units, but FastText showed me that words can also be understood through smaller pieces. For example, words such as "play," "played," "playing," and "player" share many character patterns.

This made me understand why FastText can be useful for languages and datasets containing rare words, different word forms, or words that may not appear frequently in the training data.

Conclusion

Word embeddings provide a way to connect human language with machine learning by representing words as numerical vectors. Instead of treating every word as completely independent, embeddings can capture relationships based on how words are used.

FastText extends this idea by using character n-grams, allowing it to learn from smaller parts of words. My small Python experiment helped me see this concept in practice rather than only reading about it.

My main takeaway is simple:

Words
↓
Context + subword information
↓
Numerical vectors
↓
Compare relationships.
↓
NLP applications

For me, the biggest lesson was that word embeddings are not just about converting words into numbers. They are about creating numerical representations that allow machines to work with relationships in language.

References

[1] IBM. What Are Word Embeddings?

[2] Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space.

[3] Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching Word Vectors with Subword Information.

[4] Gensim Documentation. FastText Model and Similarity Operations.

word embedding project

Marie Claire Uwiringiyimana — Wed, 30 Sep 2026 12:33:06 +0000

word embedding project

Marie Claire Uwiringiyimana — Wed, 30 Sep 2026 09:45:03 +0000