LLM
LLM is a model, which means an equation.
Example:
y = mx + c
y = m1x^3 + m2x^2 + m3x + m4
A model is actually made up of weights. In any model, e.g., ChatGPT model or Gemini model, they would have used a large amount of input to train the model.
Input means a large amount of text/image data that is available on the internet. The input would have been fed into the Transformer architecture to get the output, which is the model.
Weights are floating-point numbers that represent the model's learned parameters. A 10B or 100B parameter model means how many parameters (weights) are present inside the model.
We cannot store a large-parameter model on our computer due to inadequate storage and computational power. Storage and CPU/GPU power decide what size of model can be run on a computer.
To run a model locally, we can use one of the following tools:
- Llama.cpp
- Ollama
- LM Studio
Open Weight Model vs Open Source Model
An open-weight model shares its model weights. So, we can run them, fine-tune them, and host them on a local system.
Here, the training code, data, and full methodology are not shared.
Whereas, in an open-source model, the weights, training code, data, and sometimes the dataset are shared.
Why Do We Need to Use LLMs?
LLM is a next-word predictor.
Suppose we ask:
"Hi, how..."
The answer can be:
- How are you?
- How do you do?
- How is your life?
- etc.
These are possibilities.
Here, most of the time, the answer will be "How are you?" because if a word has more presence, it has a higher possibility of occurring.
Each possibility will have a score between 0 and 1.
We have 3 controlling parameters to control the output generated by the LLM.
1. Temperature
Usually set from 0–1. It controls the randomness of the model.
If the value is 0–0.3, which is low, it means generating the most likely words, i.e., facts or commonly occurring words.
If the value is high, the model will choose less likely words.
We use this high value in storytelling and creative writing.
2. Top K
Controls the number of possibilities.
For example, K = 3 means choosing only the 3 most likely possibilities.
Top K will be used along with Temperature.
For example, if we set Top K = 5 and Temperature = 0.5, the LLM will take possibilities from the selected Top K values based on the temperature.
Top K is used to limit the number of possibilities.
3. Top P
It is also called a sampling method.
The possibility scores are added up until they reach the defined value in Top P.
Top P is also used along with Temperature.
Tokens
Tokens are the process of splitting words into small pieces. A small piece may not be complete.
For example, a token can be split into "to" and "kens".
These are called tokens, and the process is called tokenization.
A tokenizer, such as those used by OpenAI and GPT, can be used.
We can specify the size of the token that we want to split.
Each token will be represented by a number, which in turn is converted into an embedding.
Context Window
It is a short-term memory.
The context window is the maximum number of tokens that a model can see at a given time.
Short context window and long context window are the types.
Top comments (0)