Large Language Model- Models worth considering to improve our project.

Celia Kaligirwa — Mon, 26 Oct 2026 14:16:23 +0000

The previous TinyGPT that my team worked on was a small character or word-level Transformer trained from scratch on a small text collection dataset.

Where TinyGPT hit its limits

A TinyGPT trained from scratch on a small dataset can learn the shape of language, but it hits predictable walls:

It has seen very little text, so it repeats itself and loses the thread after a few words.
Its vocabulary is small or character-based, so rare words and spelling get messy.
Its context window is short, so it forgets the start of a sentence by the end.

Models to check if they are suitable for improving our project;

LSTM / GRU (the classic baseline). These recurrent networks were the standard for next-word prediction before Transformers. They are small, fast to train, and make a good baseline, but they struggle with long-range dependencies. They’d be a step sideways rather than forward.
DistilGPT-2 / GPT-2 Small. These are the same architecture as TinyGPT, but pretrained on a huge amount of web text. DistilGPT-2 is a compressed version of GPT-2 (about 82M parameters) that runs on a laptop or free Colab GPU. Because it has already learned grammar, facts and style, fine-tuning on my own data gives a big jump in quality.
Modern small language models (SmolLM, Qwen small variants, Phi-style models). Newer models in the roughly 100M to 3B range are trained on carefully curated data and are far stronger than GPT-2 at the same size. The trade-off is that they need more memory, and I’d need to check which versions are currently available and what their licenses allow.
Mamba / RWKV (state-space and linear-attention models). These handle long sequences more cheaply than standard attention. They are exciting and educational, but the tooling is less mature and less beginner-friendly.
Retrieval-augmented approaches. Here the model looks up relevant text before predicting. That helps with factual accuracy, but it’s more than a next-word predictor needs.

My Choice: DistilGPT2

After comparing these models, I would choose DistilGPT2 to improve my TinyGPT project.

My main reason is that both models are designed for autoregressive text generation, but DistilGPT2 already benefits from pretraining on a large text dataset. My original TinyGPT was built to predict the next word using a small Transformer architecture, with token embeddings, positional information, attention layers, and residual connections. Although this approach helped me understand how language models work internally, a model trained from scratch may not learn enough language patterns from a limited dataset.

DistilGPT2 would give me an opportunity to benefit from previously learned language patterns while keeping the experiment more manageable than using a much larger model. I could compare its predictions with those of my original TinyGPT and evaluate both models using the same test sentences.

Although DistilGPT2 would be the best option to use to improve our project there are also some models that can also be worth trying. They include;

Qwen2.5-0.5B

This model has approximately 500 million parameters and supports a much larger context window than your original TinyGPT configuration.
Why choose it: It is an interesting option if you want to explore a more capable small language model, particularly for longer text prompts and multilingual experiments.

_ SmolLM2_

SmolLM2 comes in different sizes, including 135M and 360M parameters. It is designed for relatively lightweight language-model applications and offers pretrained text-generation capabilities.
Why choose it: You can compare a pretrained model against your smaller, custom-built TinyGPT without needing to train a large model from scratch.

Computer vision

Celia Kaligirwa — Tue, 13 Oct 2026 14:12:08 +0000

INTRODUCTION

Humans use their eyes and brain to recognize objects and notice details whereby computers do not see in the same way. They receive images as numbers and use mathematical operations and trained models to identify objects and make predictions.

Computer vision is simply a way computers use to process and understand visual information.

IMAGE PROCESSING

This is a way the image is changed, improved, analyzed by a computer to get useful information from it.
It comprises processes like;

Sharpening : making image more clear and identifying details
Grayscale : removing colors from an image and replaing it with shades of gray, black and white.
Blurring : making an image smoother by removing sharp details
Edge detection : help a computer to analyse the boundaries and shape of an image

COMPUTER VISION TASKS

Image classification : identifies what an image contains or what is in an image
Object detection : identifies an object in an image and shows where they are located using boxes and confidence
Image segmentation : identifies the number of pixels of each object

COMPUTER VISION MODELS FAILURES AND IMPERFECTION

Things such as poor lighting, blurry images, and unusual angles can affect their results. This is because the results depend on how much the model was trained. So the model needs to be tested with a variety of images so that it gets familiar.

This image shows how some details where not mentioned maybe because the model was not trained enough to know about those predictions. Some of the predictions are right others are wrong and some of them are not even in the image.

This image shows different predictions of the same image just because it was it was slightly changed and the model was not trained enough to predict same output.

CHALLENGES I FACED WITH THIS UNIT

Some challenges i faced where that the tasks are all relating to the image yet they perform differently. One notices an object another where its located and the other one the number of pixels belonging to it.

COMPUTER VISION APPLICATIONS IN REAL LIFE

It can help identify objects in security systems, support medical image analysis, and allow self-driving cars to recognize their surroundings without causing accidents. Also, the way our phones recognise our faces with the FaceID feature.

CONCLUSION

This unit helped me understand more of how computers see and make sense of what they see by analysing and making predictions from what they are seeing yet they've not seen it before. It's just a matter of training the model.

DEV Community: Celia Kaligirwa

Large Language Model- Models worth considering to improve our project.

Computer vision