DEV Community: Mohammad Mahabub Alam

Introduction -> RAG

Mohammad Mahabub Alam — Fri, 10 Jul 2026 12:44:24 +0000

What is RAG?

Formal Defination: Retrieval Augmented Generator it is an AI framework that improves the accuracy of LLM's by fetching facts from an external authoritative knowledge base before generating a response. It prevents the AI from "hallucinating"(making the facts by it's own doesn't matter the fact is false) and provides up to date, domain specific answers.

Why RAG shows up in AI eco system?

Okay, let's talk about without RAG. So we use AI for our day to day life, in different domains. LLM's are trained on the real world data. But as you know in our day to day life every second new data produce and the size of those data is humongous. Now based on the that humongous data if the companies try to update their LLM models than their will be chaos because to trained LLM models we need a lot of computing power and other things. So it is not possible now to trained LLM's based on real time data. Their is a term for this situation which is "Knowledge cutoff".

Now in your mind a question might arise, that so if they can not trained models with real time data than how can it gives response to question related to present time?

Here enters the RAG (Retrieval Augmented Generator). So let's talk about what RAG actually do. So at first when you give AI a prompt related to present time, where the LLM's are not trained yet with present time data. So instead of giving the response to the user, LLM calls tools to get information from the internet. For example you prompt this "What is the weather today in Dhaka? ". So here present time comes in so LLM's are smart enough and it detects that user wants to know about present day weather information, so it has some tools, it uses them to get the data from the internet in current time and in this way it is able to get the real time data. This all happens by using RAG framework.

Now you have the idea about RAG, we can go to the simple architecture of RAG:

So basically in RAG there are two pipelines present:

Indexing phrase
Query phrase

Indexing phrase:

In indexing phrase these steps are followed:

Text to Chunk
Chunk to vector embedding
Store chunk and vector embedding into a vector db.

Now let's explore each steps:

Text to chunk
First text to chunk it means that text is basically the prompt, docs, files etc that user provides. Suppose user give some pdf's and later based on these pdf's user will asked questions. So now on the pdf all the texts will divided into small small pieces and each piece is basically called a chunk.
Chunk to vector embedding
We have chunks but these chunks does not mean anything, so to give meaning to each chunk we use vector embedding. After doing the vector embedding we have the real world meaning about each chunk.
Store chunk and vector embedding in vector db
After getting the meaning of each chunk now we need to store those chunks with vector embedding so that in the query phrase where user pass any query it can go through all the meaning and select the closest one and based on that chunk it can perform next things. So there are so many vector databases present to store the vector embeddings with chunk. So we store the chunks and meaning to a db.

Query phrase:

In query phrase these steps are followed:

User text to Chunk
Chunk to vector embedding
Similarity query search in the vectorDB.
Chunk retrieved from vectorDB.
Send user query and retrieved chunks to the LLM as system prompt.

Now let's explore each steps:

User text to Chunk:

Just like indexing phrase here also user query or say prompt or text convert into chunks.
Chunk to vector embedding:
Same as indexing phrase. To give mathematical meaning to the chunks we go through vector embedding.
Similarity query search in the vectorDB:
We have the user query chunk and it's mathematical meaning we can search the vectorDB to find that, is there any close to user query vector meaning find in the db. Only this thing is done here.
Chunk retrieved from VectorDB:
Here the similar chunks retrieved from the db.
Send user query and retrieved chunks to the LLM as system prompt:
Here the user query and the chunks that are retrieved from the DB send to the LLM models as a system prompt. So the LLM have the context and based on the user query give response to it.

Image credit: Piyush Garg

Common scenarios where RAG works well

For students or any learning professionals if they wants to ask questions or find topics from big pdf's than RAG can be used, where user can upload files and based on those files student can asked the agent related questions. Now the agent will have a good reference and it will also look to it's trained data to give a solid answer, which will help the student to understand the topic more better.
Employees waste their most of the time searching something in docs, pdf's, files, chats etc.So a RAG system can be introduced which will decrease the time needed for searching stuffs and employee can focus on more to solve the real problems.
Privacy policies search: Most of the time users are not interested to read the privacy policies of a website or company because of the long listed paragraphs and the number of pages there in the policy section. So there can be RAG introduced so that it can read the policies and than user can asked questions about the policies.
Financial Analysis: In financial sectors there are lots of documents and reports. Until now all the documents need to check at a time which takes a lot of time and also man power also money. So if we use RAG here user can uploads documents and then the agents will go through the documents and answer the user questions. Also it can remember that where which data is place like page number etc which makes easier for user to find out correct things in correct place.
Education area: In all online courses each course contains many videos. So if someone wants to learn something or look at some specific topic he have to go through so many videos. Sometimes he get will found the topics he is looking for but most of the time didn't get the desired topic which makes learning more difficult,if student wants to go through some topics he can not. So if we use RAG here we can help students to find their desired topic and also able to give them exact time stamp and video number which will make the study more reliable and more enjoyable.

RAG issues

If the agent have incorrect data than it will give incorrect answers. Because it takes context or idea based on incorrect data. We human have the capability to understand corrupt or say unstructured data which is not possible for the LLM models. Also there is a limitation of memory it can have. If you give a huge data set into the context than it started hallucinating as a result it gives wrong answers with full confident.

Why issues arise?

Poor retrieval: Poor retrieval simply means that retriever found documents, but they are the wrong ones. For example: You provide documents about "how to reset password in facebook" but you give this prompt "how can i reset my linkedin password?" now the LLM will give you response based on that provided documents which will be related to facebook password reset but you want for linkedin.

Why poor retrieval happens:
->Bad embedding : Embedding is basically converts text into vectors. If two related sentences end up far a part in that 3D vector space than similarity search fails. For example: You give a document which contains "How to reset your fb password." but the user query is "I forgot my fb password." Now if bad embedding happens than the "reset password" and "forgot password" the distance between them will be so far that the model can not make a proper similarity match between them, as a result the model will not understand the user query.

Poor Chunking: In the RAG their is a phrase where the text converted into small chunks. Now if these chunks divided in a way were in each chunk there is no connection between other chunks than it will be a reason for bad response. For example: Suppose your document is this:

Step 1:
Go to settings
Click password
Enter new password

now when after you provide this document and the chunking is like this:

Chunk A: Step 1:
Go to settings

Chunk B: Click password

Chunk C: Enter new password

Now if the chunks are in this way than if the retriever retrieve chunk C than the LLM will get "Enter new password" but it has no idea where to go as a result it can not provide a good response to the user because the LLM itself don't know where to go.

Missing context: It means the retriever actually found the correct document but the information provided is incomplete. For example: User provide a document which contains all the ingredients name and the portion numbers, but it doesn't contain anything related to "how to make biryani?" and if the user query is like "tell me how to cook biryani?" that time for missing the context the agent response will be incomplete or wrong.

Generative AI: What Happens When You Type a Prompt?

Mohammad Mahabub Alam — Wed, 01 Jul 2026 18:28:30 +0000

If you’ve used ChatGPT, Claude, or Gemini recently, you’ve witnessed a minor miracle. You type a messy, half-formed thought in plain English, and a few seconds later, you get a coherent, context-aware response. It feels like you're talking to a incredibly well-read human who lives inside your screen.
But beneath the sleek user interface lies a fascinating machinery of math, statistics, and linguistic translation. Because here is the ultimate plot twist of the AI revolution: computers still don’t understand a single word of human language.
If you are a JavaScript developer looking to build with Generative AI, or just someone curious about how these models actually tick, you need to understand the pipeline. Let’s pull back the curtain on what happens between hitting "Enter" and seeing that flashing cursor generate your answer.

1. Demystifying the LLM

Before looking at the mechanics, let's define the core engine: the LLM, or Large Language Model.

Large: Refers to the scale. These models are trained on massive datasets (essentially the public internet, books, and articles) and contain billions of internal settings called "parameters."
Language: Their domain of expertise. They are built to process, predict, and generate human-like text.
Model: A mathematical representation of patterns learned from data. ### The Problems LLMs Solve Traditionally, coding an application to understand human intent meant writing endless chains of if/else statements, regex patterns, or complex sentiment analysis rules. Even then, a typo could break the system. LLMs solve the problem of unstructured data. They can parse chaotic, unstructured human text and extract meaning, summarize it, or transform it into structured formats like JSON. ### LLMs in the Wild You encounter these models daily through popular examples like:
OpenAI's GPT-4o (powering ChatGPT)
Google's Gemini
Anthropic's Claude
Meta's Llama (an open-source model you can run locally) Beyond chatbots, they power features you might take for granted: email autocomplete, real-time language translation, IDE code suggestions (like GitHub Copilot), and automated customer support lines that actually solve your problem. ## 2. Why Computers Don't Speak Human To understand the workflow, we have to address a fundamental limitation of computers. Computers are, at their core, glorified calculators. They operate entirely on binary, logic gates, and math. They excel at processing numbers but have absolutely no concept of what the letter "A," the word "apple," or the concept of "existential dread" means. To bridge this gap, everything we type must be converted into numerical values before an AI can touch it. If you feed raw English text into a machine learning model, it’s the equivalent of pouring sand into a car's engine. To turn text into math, the AI uses a process called Tokenization. ## 3. Tokenization: Breaking Down the Words Before an LLM reads your prompt, a specialized piece of software called a tokenizer chops your sentence into bite-sized pieces called tokens. > What is a Token? > A token is the basic unit of text that an AI reads. It isn’t always a whole word. It can be a single character, a syllable, part of a word, or even punctuation marks. > ### Why not just use whole words? If an AI mapped every single unique word in human history to a number, the vocabulary list would be impossibly massive and rigid. It would struggle with new slang, typos, or compounded words. By breaking text into sub-words, the AI can understand entirely new words just by looking at their pieces. As a general rule of thumb, 1 token is roughly equal to 4 characters, or about 0.75 words in English. ### A Visualizing Example Take the sentence: "Coding in JS is fantastic." A tokenizer might break it down like this: ["Cod", "ing", " in", " JS", " is", " fan", "tastic", "."] Each of these distinct fragments is assigned a specific ID number from a massive pre-defined dictionary.
"Cod" might become 412
"ing" might become 102
" JS" might become 8453 By the time the AI receives your input, it isn’t reading your words; it’s reading an array of integers: [412, 102, 8453, ...]. ## 4. The Brain Inside: The Transformer Once your prompt is a sequence of numbers, it enters the core architecture of modern AI: the Transformer. Introduced by Google researchers in a groundbreaking 2017 paper titled "Attention Is All You Need," the Transformer architecture completely revolutionized how machines process language. Before Transformers, AI read sentences sequentially—word by word. If a sentence was too long, it forgot how it started by the time it reached the end. Transformers changed everything using a mechanism called Self-Attention. ### How Transformers Understand Context Self-attention allows the model to look at a word and instantly look at every other word in the sentence to figure out its exact meaning. Consider these two sentences:
"The bank of the river was muddy."
"The money is safely deposited in the bank." As a human, you easily deduce the meaning of "bank" from the surrounding words ("river" vs "money"). A Transformer does exactly this mathematically. It calculates relationships between all the tokens in your prompt simultaneously, building a rich map of context. This architectural breakthrough is why almost every major LLM today—from GPT to Claude—has a "T" in its name or uses Transformers under the hood. It allows the AI to grasp nuance, sarcasm, and complex instructions over incredibly long stretches of text. ## 5. What Happens When You Send a Message? (The High-Level Workflow) Let’s piece the whole assembly line together. What actually happens when you type a prompt into ChatGPT and hit send? ### Step 1: The Input (Typing the Prompt) You submit your text. Along with your visible message, the system often injects hidden developer instructions (called a System Prompt) behind the scenes, such as "You are a helpful programming assistant that writes clean JavaScript." ### Step 2: Tokenization & Mapping Your text is converted into token IDs. These IDs are then translated into vectors (long lists of numbers) that represent where that token sits in a multi-dimensional "meaning space." ### Step 3: Processing inside the Transformer The numerical data passes through dozens of layers of the Transformer network. The model weighs the tokens against each other using self-attention, figuring out the intent of your question and what information matters most. ### Step 4: Generation (Predicting the Next Token) Here is the big secret: LLMs do not copy and paste answers from a database or search engine. They don’t "know" facts the way a human encyclopedian does. Instead, an LLM is an incredibly sophisticated next-token predictor. Based on your prompt, it calculates a probability distribution for what token should come next. If your prompt is "The sky is...", the model evaluates its training data and decides there is a 95% chance the next token should be " blue", a 2% chance it's " cloudy", and a 0.1% chance it's " soup". It selects the best token, appends it to your prompt, and feeds the entire thing back into itself to predict the token after that. It repeats this loop at lightning speed until it hits a designated "stop" token. This autocomplete loop is why you see the text stream onto your screen letter-by-letter. ## 6. Controlling the Output: Temperature If the AI simply picked the #1 most probable token every single time, its responses would be completely rigid, repetitive, and robotic. To give the AI creativity and variance, developers use a setting called Temperature. Temperature controls how much risk the model takes when picking its next word. | Low Temperature (e.g., 0.2) | High Temperature (e.g., 0.9) | |---|---| | Predictable & Focused: The AI sticks strictly to the highest-probability words. | Creative & Random: The AI is allowed to pick less probable, more "unexpected" words. | | Ideal for: Writing code, debugging text, math, or factual summaries. | Ideal for: Brainstorming, writing fiction, naming a startup, or creative copy. | ## Wrapping Up: The Context Window The final piece of the puzzle to keep in mind is the Context Window. Because the model has to process your entire conversation history every time it generates a new token to remember what you said five minutes ago, there is a limit to how much data it can hold in its active memory at once. Think of the context window as the AI's short-term working memory. If a model has a 128k token context window, it can remember roughly 90,000 words of conversation before it starts "forgetting" the oldest messages. As a developer entering the GenAI space in 2026, understanding this pipeline—from prompt to token, transformer to probability calculation—is your superpower. You aren't just sending strings into a black box anymore; you are orchestrating a highly tuned, mathematical prediction engine.

SnapShot 📸🔫

Mohammad Mahabub Alam — Fri, 26 Jun 2026 16:11:28 +0000

A snapshot is a frozen view of a database at a specific moment in time. It is like taking a digital photocopy of a document. Here document is the database and the photocopy is the snapshot. It is a read only static view of data means that you can look at the data but you can not change it within that snapshot itself.

The way it works:

Generally we think that snapshot is a massive bulky copy of the entire database but that would be incredibly slow ans expensive. Instead modern databases use a clever trick called COW (Copy on write). So when you first take a snapshot, the database doesn't actually copy any data. It simply creates a pointer map, at the exact millisecond the snapshot and the live database are looking at the same physical block of the data on the disk. The magic happens only when someone tries to change the live data.
For example: You have a row of data in block "A". If a user update that row the system identifies that block "A" is about to be changed. That time it copies that original old data to a specific snapshot storage area. after that it writes the newly data to the live database.

Remember that the snapshot is taken only when the original data is going to update.

More explanation : The snapshot pointer and the live database pointer both points to the same block "A" memory location. After that when the user wants to update block "A" with the new data, the system reads the original data from block "A" and writes it into the new physical space on the disk also can be refer as the snapshot storage. After that the snapshot pointer is updated to point the newly moved original data. The live database pointer stays where it is or moved to the newly created block to write the new data.

The snapshot pointer is used to find the data that are already snapshot.

If the system only update a pointer without copying the data than live update would overwrite the original data and your snapshot would be ruined. For that reason first it copies the old data, stores it into the snapshot storage and than the snapshot pointer update and points this memory storage.

Snapshot eventually slow down a database if they are kept for too long. Every time the data updates it copies the old one and than writes into the live data and update. It works double in this way. So to make it efficient modern databases used a method called ROW (Redirect on write).
Copy on Write (COW) : First copy the old data and than write the new one.
Redirect on write (ROW): As in COw it moved the old data to save it but in ROW we leave the old data as it is and write the new data somewhere else. Same as before the snapshot and live database both point to block "A". When user is trying to update the data it keeps the old data as it is and write the new data into another new memory location. The snapshot pointer stays the same place but the live database pointer is redirected to the new block where the new data is stored.

Copy on Write	Redirect on Write
It require 1 read 2 write (Read old→ write old elsewhere→ write new )	It require only 1 write (new data stores in a new location.)
Instant to create but slowdown overtime.	Instant to create and stays fast during database updates.
Read the data faster than ROW	Reading data is slow as new data took place in different locations.