As someone learning software engineering and Natural Language Processing (NLP), I have been exploring how machines understand and generate human language. One of the most interesting topics I have learned about is Large Language Models (LLMs).
In this article, I will share what I learned about LLMs, how they work, how they generate text, and how we can run open-weight models using Python and Hugging Face.
1. What Is a Large Language Model?
A Large Language Model is a neural network trained on large amounts of text to learn patterns in language and predict the next token.
The main idea is simple: predict the next token based on the context that comes before it.
For example:
"The student is learning to write..."
A language model calculates the probabilities of possible next tokens, such as "code," "stories," or "essays." It then selects a token and repeats the process to generate a longer response.
This process allows LLMs to perform tasks such as:
- Answering questions
- Generating text
- Writing and explaining code
- Summarizing documents
- Translating languages
I learned that even though these models can produce human-like responses, their core training objective is based on learning patterns and predicting tokens.
2. From Traditional Language Models to Transformers
Language models have evolved over time.
N-gram models
Earlier language models used word sequences to estimate which word was likely to come next. However, they struggled with unseen combinations and had limited context.
RNNs and LSTMs
Recurrent Neural Networks (RNNs) processed sequences one step at a time. LSTMs improved their ability to retain information, but processing long sequences remained challenging.
Transformers
The Transformer architecture introduced a more effective way to process relationships between tokens using attention mechanisms.
Transformers made it possible to train much larger language models efficiently.
One important distinction I learned is that a Transformer is an architecture, while an LLM is a trained language model. Not every Transformer is an LLM; for example, Vision Transformers process image patches instead of text.
3. Tokenization: How Text Becomes Numbers
Computers process numerical representations rather than ordinary text. Before an LLM can work with a sentence, the text must be converted into tokens.
There are three common tokenization approaches:
- Word tokenization: Splits text into words.
- Character tokenization: Splits text into individual characters.
- Subword tokenization: Splits text into smaller units that may be complete words or parts of words.
Modern LLMs commonly use subword tokenization because it balances vocabulary size and the ability to represent unfamiliar words.
For example, the word "unhappiness" might be split into:
un | happi | ness
The exact split depends on the tokenizer.
Each token is then mapped to a numerical ID. These IDs are the inputs the model uses to process text.
I also learned about Byte Pair Encoding (BPE), a tokenization method that repeatedly merges frequently occurring adjacent token pairs to build a vocabulary.
4. Context Windows and Embeddings
A token ID is only the beginning. The model converts token IDs into embeddings, which are numerical vectors representing tokens.
Embeddings help the model learn relationships between words and their surrounding context.
For example, the word "bank" can refer to a financial institution or the side of a river. Its contextual representation changes depending on the surrounding words.
Another important concept is the context window. It determines how many tokens the model can process together, including the prompt and the generated response.
If the input exceeds the model's context limit, we may need to truncate the text, divide it into chunks, or retrieve only the most relevant information.
This helped me understand why tokenization, embeddings, and context management are important when building NLP applications.
5. Inside a Decoder-Only Transformer
Many modern chat-oriented LLMs use a decoder-only Transformer architecture.
The general process looks like this:
- Text is converted into token IDs.
- Token IDs are converted into embeddings.
- Positional information is incorporated.
- Transformer blocks process the representations using masked self-attention and feed-forward layers.
- The model produces scores for possible next tokens.
- Softmax converts those scores into probabilities.
The model then uses these probabilities to generate the next token.
Understanding the causal mask
One concept I found especially important is the causal mask.
During next-token prediction, the model must not look at future tokens. When predicting a token, it can only attend to the current position and the tokens before it.
This prevents the model from seeing the correct answer during training and simply copying it.
Understanding this concept helped me see how Transformers can learn to generate text one token at a time.
6. How LLMs Learn: Self-Supervised Pretraining
LLMs learn from large text datasets through pretraining.
One major approach is autoregressive language modeling, where the model predicts the next token using the preceding context.
For example:
"The glass jar is..."
The model learns to assign probabilities to possible next tokens based on the context.
Another approach is masked language modeling, where some tokens are hidden and the model learns to predict them from the surrounding text. BERT is a well-known example.
I also learned about three important model families:
- BERT: An encoder-only model commonly used for classification, search, and named-entity recognition.
- GPT: A decoder-only family designed for next-token prediction and text generation.
- T5: An encoder-decoder model that treats tasks as text-to-text problems, such as translation and summarization.
These models have different architectures and training objectives, so choosing one depends on the task we want to solve.
7. Model Size, Parameters, and Memory
Another important lesson was that model size affects computational requirements.
Parameters are the learned numerical weights that help a model produce its outputs.
For example, a model with four billion parameters stored in 16-bit floating-point format requires approximately 8 GB of memory just for its weights.
Larger models may offer stronger capabilities, but they generally require more memory, computing power, and time.
I also learned about quantization, which reduces the precision used to store model weights. Four-bit quantization can significantly reduce memory requirements and make larger models more practical to run on limited hardware.
This is particularly useful when experimenting with open models in Google Colab.
8. How LLMs Generate Text
Text generation is a repeated process:
- The model receives the current context.
- It calculates probabilities for possible next tokens.
- A token is selected.
- The selected token is added to the context.
- The process repeats until the model reaches an end condition or the generation limit.
Several parameters influence the result.
Temperature: Controls how concentrated or spread out the token probabilities are. Lower values generally produce more predictable outputs, while higher values allow more variation.
Top-k sampling: Restricts token selection to the k most likely candidates.
Top-p sampling: Selects from a set of likely tokens whose cumulative probability reaches a chosen threshold.
Max new tokens: Limits how many tokens the model can generate.
Repetition penalty: Discourages the model from repeatedly using tokens it has already generated.
I learned that these settings matter because the same model can produce different outputs depending on how text generation is configured.
9. Running an Open-Weight LLM with Python
One of the most practical parts of this topic was learning that we can run open-weight models without relying entirely on paid APIs.
For example, the course introduced Qwen/Qwen3-4B-Instruct-2507, which can be loaded using Hugging Face Transformers.
Here is a simplified example:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "Qwen/Qwen3-4B-Instruct-2507"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
dtype=torch.float16,
device_map="auto"
)
messages = [
{
"role": "system",
"content": "You are a helpful and concise tutor."
},
{
"role": "user",
"content": "Explain what a token is in an LLM."
}
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer(
prompt,
return_tensors="pt"
).to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=100,
do_sample=True,
temperature=0.7,
top_p=0.8
)
response = tokenizer.decode(
outputs[0][inputs.input_ids.shape[1]:],
skip_special_tokens=True
)
print(response)
This example demonstrates the basic workflow: load a tokenizer and model, format the conversation, tokenize the prompt, generate new tokens, and decode the response.
The model's first download can be large, and successful execution depends on having compatible software and enough available memory. A smaller model or quantized version may be a better starting point on limited hardware.
10. Limitations of Large Language Models
Although LLMs are powerful, they have important limitations.
Hallucinations
An LLM can generate a convincing answer that contains incorrect information. A fluent response is not necessarily a factual one.
Knowledge limitations
A model may not know about events or information that were not included in its training data.
Bias
Models can reproduce biases and stereotypes present in their training data.
Computational cost
Training and running large models can require significant computing resources and energy.
These limitations remind me that building a useful AI application involves more than selecting a powerful model. We also need to evaluate outputs, verify important information, and consider the risks associated with our applications.
Retrieval-Augmented Generation (RAG), which supplies relevant external information to a model, is one approach to addressing some knowledge limitations and reducing hallucinations.
11. Choosing the Right Model for an NLP Project
When selecting an LLM for a project, I would consider the following factors:
- Task: Does the project require text generation, classification, summarization, translation, or question answering?
- Performance: Does the model produce useful and reliable results for the intended task?
- Memory requirements: Can it run with the available hardware?
- License: Are its usage and distribution terms compatible with the project?
- Cost: Can the project operate within its budget?
- Privacy: Can the model be run locally if the application handles sensitive information?
Models introduced in the course include Qwen, Llama, SmolLM, and Phi. Each has different sizes, licensing conditions, and capabilities.
For a practical project, I would begin by testing a suitable smaller model, evaluating its outputs against real examples, and comparing it with alternatives before making a final decision.
Conclusion
Learning about Large Language Models has helped me understand what happens behind the AI tools we use every day.
I learned that LLMs do not simply understand text in the same way humans do. They process token IDs, use learned embeddings, apply Transformer architectures, and generate responses through next-token prediction.
I also learned about self-supervised pretraining, causal masking, context windows, decoding parameters, model memory requirements, and running open-weight models with Python.
Most importantly, I now understand that choosing an AI model requires balancing its capabilities, computational requirements, limitations, and suitability for the problem.
My next step is to apply these concepts to an NLP project, experiment with available models, and evaluate which one best fits the project's requirements.
Learning AI is not only about using models; it is also about understanding how they work and knowing when to trust their outputs.
References
- GeeksforGeeks: Large Language Models
- Hugging Face Transformers Documentation
- Qwen3-4B-Instruct-2507 Model Card
Top comments (0)