𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐚 𝐃𝐲𝐧𝐚𝐦𝐢𝐜 𝐑𝐀𝐆 𝐏𝐢𝐩𝐞𝐥𝐢𝐧𝐞 𝐰𝐢𝐭𝐡 𝐋𝐚𝐧𝐠𝐂𝐡𝐚𝐢𝐧 (𝐓𝐡𝐚𝐭 𝐒𝐭𝐚𝐲𝐬 𝐅𝐫𝐞𝐬𝐡)

Most RAG (Retrieval-Augmented Generation) systems work fine for static knowledge bases—but the moment your documents start changing (new policies, updated financials, revised product specs), they quickly go stale.

We solved that with a dynamic RAG pipeline that keeps embeddings and context fresh without doing heavy full rebuilds. Here’s how it works:

🧩 High-Level Flow

1️⃣ 𝐖𝐚𝐭𝐜𝐡𝐞𝐫 (𝐅𝐢𝐥𝐞/𝐒3 𝐜𝐡𝐚𝐧𝐠𝐞𝐬)
▪ Continuously listens for file changes (local folder or S3 bucket).
▪ Detects when a document is new, updated, or deleted.
2️⃣𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠 (𝐨𝐧𝐥𝐲 𝐮𝐩𝐝𝐚𝐭𝐞𝐬)
▪ Instead of re-embedding everything, it re-embeds only the changed chunks.
▪ Saves time and compute costs while keeping the knowledge base fresh.
3️⃣ 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐁 (𝐂𝐡𝐫𝐨𝐦𝐚)
▪ Stores embeddings with metadata like updated_at.
▪ When conflicts arise (e.g., same document with old + new facts), retrieval logic can guide the LLM to trust the freshest snippet.
4️⃣ 𝐋𝐋𝐌 (𝐎𝐥𝐥𝐚𝐦𝐚/𝐎𝐩𝐞𝐧𝐀𝐈)
▪ Takes the top-k retrieved chunks and augments the query.
▪ Produces a contextualized answer with citations.
5️⃣ 𝐒𝐭𝐫𝐞𝐚𝐦𝐥𝐢𝐭 𝐔𝐈
▪ Users simply ask questions.
▪ The UI calls the FastAPI backend, retrieves from Chroma, and passes to the LLM.
▪Responses include answers + sources, so users know why the model said what it did.

🚧 The Challenge (Simple Example)
One file said:
➡️ “All banks must maintain capital reserves of 10%.”
Later, an update stated:
➡️ “All banks must maintain capital reserves of 12%.”
When I asked: “What is the required capital reserve?”

Static RAG: “I don’t know.” (confused by conflicting facts)
Dynamic RAG: “12%” (trusts the most recent doc)

𝐓𝐡𝐞 𝐒𝐨𝐥𝐮𝐭𝐢𝐨𝐧 — 𝐃𝐲𝐧𝐚𝐦𝐢𝐜 𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠𝐬
🔄 Watches for new/updated docs in real time
⚡ Re-embeds only what changes (no full rebuilds)
🏷️ Tracks updated_at so the LLM knows the freshest fact
🧠 Guides the model to resolve conflicts by trusting the most recent snippet
Now, when a file is updated, the system re-embeds instantly and gives the right answer.

For the full working codebase, check my GitHub repo
https://github.com/rajeevchandra/dynamic_embeddings

At the end of the day, AI systems are only as useful as the freshness of the knowledge they rely on. Building dynamic pipelines isn’t just about better tech — it’s about building assistants that can actually keep up with how fast the world changes.