We use chatGPT everyday writing emails,to get some ideas or write a document.It feels like talking to human who. But have you ever wondered what actucally happens behind the scenes?
What is AI?
Artifical Intelligence is ability of a computer to perform tasks that normally require human intelligence such as understanding language,recognizing images, making decisions or solving problems.
What is GPT?
GPT stands of Generative Pre-trained transformer
- Generative: It generates new text instead of simply searching for existing answers
- Pre-trained:It had already learned from enormous amount of books,articles,websites
- Transformer: It uses a neural network architecture called the Transformer, introduced in the famous research paper "Attention Is All You Need"
What is ChatGPT?
GPT is the brain and ChatGPT is the chatbot application that is built on top of GPT
You can think it as GPT is a chef and ChatGPT is the restaurant where customers interact with that chef
What is LLM?
LLM stands for Large Language Model
Large means trained on massive amount of data
Language understands natural language
Model predict the next token
What is a Token?
Models can understand numbers.GPT converts all the text in number which is called Token
Embeddings
Numbers alone have no meaning.The model converts every token into a mathematical representation called embedding
Imagine every word being placed on gaint map and words similar meaning are placed closed together
Next Token Prediction
GPT main Job is predicts the next token
eg> "The sun rises in the ----
east 97%
west 2%
north 1%
Positional Encoding
Suppose you have 2 sentences Man bites dog and dog bites man
They contain same words but completely different meaning
To understand this difference GPT adds positional encoding which tells the model where each token appears in the sentence.
Self-Attention
Tokens looks at the other token in the sentence to decide which ones are important
Consider the sentence:
The trophy didn't fit into the suitcase because it was too small
self attention helps GPT determins that "it" refers to the suitcase because of the surrounding context.
Instead of reading one word at time ,the model understands relationships between all words simultaneously
Probability Distrubution and softmax
After understanding the sentence,GPR predicts several possible next tokens.This probabilities are calcualted using mathematical function called softmax which converts raw scores into probabilities that add up to 100%
Model selects next token based on probabilites
What is temperature
Temperature controls how creative the model should be with low temperature produces produces consistent answers and with high temperature produces more creative and sometimes unpredictable responses.
Transformer has 2 phases
- Training Phase This phase is performed by AI companies using enoromous datasets and powerful GPUs. During training model learns to predict next tokens
- Inference Phase This phase is when user uses it eg. when you write prompt to chatgpt Model is not learning new knowledge instead it uses everything it learned during training to predit the next token.
Top comments (0)