- About our project
Our project is an assistant that tells people which bin an item belongs in, and why. Some items are easy, like a plastic bottle. Others are tricky, like a greasy pizza box or a broken light bulb. We want the assistant to answer in a clear format, for example “Category: Recyclable. Reason: …”.
In my NLP class I built a next-word predictor using GPT, but I only used the basic version and never reached GPT-2. Researching this blog helped me see that next-word prediction is exactly what every modern LLM does, just at a much bigger scale. An LLM gives a probability to every possible next token and picks one, over and over, until it has a full answer.
Our team has no budget and uses the free Google Colab T4 GPU, which has 16 GB of memory. So I searched for open models of about 4 billion parameters or fewer.
- The four models I looked at
Qwen3-4B-Instruct-2507 (Alibaba, Qwen team). It has 4 billion parameters and a very large context window of 262,144 tokens. The licence is Apache-2.0. It is built to follow instructions well, and it has no “thinking mode,” so its answers don’t start with long reasoning text.
Llama-3.2-3B-Instruct (Meta). It has about 3.21 billion parameters and a 128K context window. It was released in September 2024 under the Llama 3.2 Community License. It is good at conversation and summarising. To download it, you need a Hugging Face account and must accept Meta’s terms.
Phi-4-mini-instruct (Microsoft). It has 3.8 billion parameters and a 128K context window, under the MIT licence. It came out in February 2025 and is especially good at maths and logic for its size.
SmolLM3-3B (Hugging Face). It has 3 billion parameters. It was trained with a 64K context and can reach 128K using a method called YaRN. The licence is Apache-2.0. It supports six languages and has a reasoning mode you can switch on or off. Its training recipe is fully public.
- Comparison table
es/71wb73shg6qyxk5gvfbq.jpeg)
The memory column is my estimate: parameters × 2 bytes. All four fit inside the T4’s 16 GB.
- The model I choose: Qwen3-4B-Instruct-2507
I chose Qwen3-4B-Instruct-2507 for four reasons.
- It fits on the free Colab T4. I downloaded and loaded it on a free Colab T4. The model files are 8.04 GB, which matches my estimate of 4 billion × 2 bytes. I loaded it in float16, because the T4 does not support bfloat16. It loaded successfully, so it fits within the 16 GB limit. I did not measure the exact speed.
It is free and easy to start. I didn’t need an API key, a paid plan, or to accept any terms. It is Apache-2.0, so I could download and use it right away. Llama needs one extra step, an account and Meta’s agreement, which is more work for a team.
It follows instructions and gives clean answers. Our assistant must answer in the same format every time. Qwen3-4B-Instruct-2507 was improved for instruction following, and because it has no thinking mode, its answer comes straight, without extra text. That makes it easy to read and easy to use in code.
It can help our project in three ways.
Sorting accuracy: it sticks to our categories and our format.
Explaining disposal rules: the large context window lets us paste our local recycling rules into the prompt, so it explains using the rules instead of guessing.
Tricky items: we can show it a few examples of hard items and ask for a one-sentence reason.
Why I did not choose the others. Phi-4-mini is best at maths and logic, which matters less for sorting waste. Llama needs the account and terms step. SmolLM3 is very open and good, but it is built around six languages and a shorter trained context, which helps us less.
Limits. Qwen3-4B is the biggest of the four, so it is a little slower and uses the most memory. If it ever runs out of memory, our backup is Qwen3-1.7B. Like all LLMs, it can make things up (hallucinate), so for rules that matter, we should put the real rules in the prompt and not trust the model’s memory.
- Conclusion
I started this task knowing only next-word prediction from my class project. Now I understand how bigger models work and how to choose one with real facts: size, licence, memory, and how well it follows instructions. For our project, I choose Qwen3-4B-Instruct-2507.
References
Qwen team. Qwen3-4B-Instruct-2507 model card. https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507
Meta. Llama-3.2-3B-Instruct model card. https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
Microsoft. Phi-4-mini-instruct model card. https://huggingface.co/microsoft/Phi-4-mini-instruct
Hugging Face. SmolLM3-3B model card. https://huggingface.co/HuggingFaceTB/SmolLM3-3B
Qwen Team. Qwen3 Technical Report. arXiv (confirm the number before publishing).
Course lecture notes: LLM_01 Introduction to LLMs.
Top comments (0)