I Built a RAG System, But I Didn't Want It to Be Another "Chat With Your PDF" App

Rayulu Mukku — Sun, 30 Aug 2026 20:05:06 +0000

# I Built a RAG Workstation Instead of Another "Chat With Your PDF" App

I've been building RAG Studio for a while, and I wanted to do something a little different from the usual RAG projects.

Most RAG demos are basically:

Upload a document → ask a question → get an answer.

That's fine for demonstrating the concept.

But when I started building my own, I kept thinking about everything happening between those three steps.

What happens to the document before it's embedded?

What if the document contains an API key or someone's email address?

Which chunks actually get retrieved?

Does a high similarity score mean the retrieved content is actually useful?

And what happens when the answer isn't in the local documents at all?

I didn't want all of that to be hidden behind a single "Ask AI" button.

So I built RAG Studio — an interactive RAG workstation where I can actually see and inspect what's happening throughout the pipeline.

The project ended up combining three areas I wanted to explore:

ETL + DAG orchestration + Corrective RAG (CRAG).

And one other thing was important to me:

Generate embeddings locally in the browser whenever possible.

What does the architecture look like?

At a high level, the pipeline looks like this:

Document
   ↓
ETL / Cleaning
   ↓
Chunking
   ↓
Local Embedding
   ↓
Vector Index
   ↓
Query Embedding
   ↓
Cosine Similarity
   ↓
CRAG Evaluation
   ↓
 ┌───────────────┐
 │               │
Relevant      Not Relevant
 │               │
 ↓               ↓
Local Context   Web Search
 │               │
 └───────┬───────┘
         ↓
      LLM

Quite a few things are happening here, so I'll go through the interesting parts.

1. I started with ETL because documents are messy

A lot of RAG tutorials start with clean text.

Real documents don't.

A document can contain:

headers and footers
duplicated text
HTML boilerplate
advertisements
inconsistent whitespace
irrelevant navigation content
personal information
API keys and other secrets

If that content goes straight into chunking and embedding, you're already starting with a problem.

So RAG Studio has a dedicated ETL Studio before the actual RAG pipeline.

Extract

The ingestion layer supports:

PDF
TXT
Markdown
JSON
HTML/article content

The goal is to turn different input formats into a normalized text representation.

Transform

The cleaning stage then removes unnecessary content and looks for sensitive patterns.

For example:

alex.vance@ai-research.org
            ↓
[REDACTED_EMAIL]

+1-555-019-2834
            ↓
[REDACTED_PHONE]

sk-live-xxxxxxxx
            ↓
[REDACTED_API_KEY]

192.168.1.104
            ↓
[REDACTED_IP]

There is also whitespace normalization and HTML/boilerplate cleanup.

The important part is the ordering:

redaction happens before vectorization.

I wanted sensitive information removed before it ever became part of the embedding/indexing pipeline.

2. I didn't want the ETL process to be invisible

Normally, document cleaning happens somewhere in the backend and the user never sees it.

I wanted the opposite.

RAG Studio has a side-by-side diff viewer showing:

RAW DOCUMENT          CLEANED DOCUMENT

Original text    →    Normalized text
PII              →    [REDACTED]
HTML noise       →    Removed
Extra whitespace →    Cleaned

It also shows information such as the original size, cleaned size and compression ratio.

So you can actually see what the ETL stage did before loading the cleaned corpus into the RAG pipeline.

This turned out to be more useful than I initially expected.

When retrieval produces strange results, I can go back and check whether the problem started with the source data.

3. Then I turned the RAG pipeline into a DAG

Once the document is clean, there are still quite a few stages involved.

Instead of hiding them behind a function like:

await runRAG(query);

I represented the pipeline as a Directed Acyclic Graph.

The current pipeline contains nodes such as:

etl_hub
doc_source
chunking
vectorizer
vector_index
query_source
query_embed
cosine_ranker
crag_grader
web_fallback
context_injector
llm_synthesizer

Visually, it becomes something like:

ETL Hub
   ↓
Document Source
   ↓
Chunking
   ↓
Vectorizer
   ↓
Vector Index
   ↓
Query Source
   ↓
Query Embedding
   ↓
Cosine Ranker
   ↓
CRAG Grader
   ↓
Web Fallback
   ↓
Context Injector
   ↓
LLM Synthesizer

The graph isn't just there for visualization.

It gives me a way to inspect the actual pipeline state.

4. Why I wanted an interactive DAG

Let's say the final answer is wrong.

It's very easy to immediately think:

"The LLM hallucinated."

But maybe the LLM wasn't the problem.

Maybe the document wasn't cleaned properly.

Maybe the chunk boundaries were bad.

Maybe the relevant chunk wasn't retrieved.

Maybe the retrieved chunk was semantically similar but didn't actually answer the question.

Maybe the system should have searched the web instead.

With the DAG, I can work through those stages.

Each node can expose things like:

input payload
output payload
execution state
vector dimensions
storage mechanism
latency
memory information

I also added the ability to jump directly from a node in the graph to the corresponding stage in the application.

So the graph becomes something closer to a debugger for the RAG pipeline.

5. The embedding pipeline runs locally

This was another part of the project I really wanted to experiment with.

Instead of sending every document chunk to an external embedding API, RAG Studio uses Transformers.js and ONNX Runtime to generate embeddings locally in the browser.

The model produces 384-dimensional embeddings.

The basic flow is:

Text
 ↓
Tokenizer
 ↓
Transformer Model
 ↓
384D Vector
 ↓
Local Vector Index

The model execution uses WebAssembly, while the heavier inference work runs inside Web Workers so the main UI thread doesn't get blocked.

So the browser is doing a lot more of the work:

Document
   ↓
Browser
   ↓
Embedding Model
   ↓
Vector
   ↓
Local Index

instead of:

Document
   ↓
Server
   ↓
Embedding API
   ↓
Vector Database

For a large production system, I'd obviously consider a different architecture.

But for this project, running the embedding pipeline locally was an interesting trade-off.

It reduces the need for a separate embedding service and keeps more of the processing on the client.

6. Then comes the math

Once everything is embedded, retrieval is essentially a vector similarity problem.

The query is converted into a vector, and that vector is compared against the document vectors.

The main calculation is cosine similarity:

cosine_similarity(A, B)
=
(A · B) / (||A|| ||B||)

So a retrieval result might look like:

Chunk A    0.91
Chunk B    0.86
Chunk C    0.72
Chunk D    0.31

The higher the score, the more similar the vectors are.

But there is a problem.

Similarity doesn't necessarily mean relevance.

7. A high similarity score doesn't mean the context is good

This is where I wanted to go beyond basic RAG.

Imagine the user asks a question and retrieval returns:

Chunk A → 0.91
Chunk B → 0.87
Chunk C → 0.83

It looks great.

But what if those chunks are only related to the question and don't actually contain enough information to answer it?

If we blindly send them to the LLM, we're basically hoping the model figures it out.

I wanted another check before generation.

That's where Corrective RAG (CRAG) comes in.

8. The CRAG layer acts as a quality gate

After retrieval, the selected chunks are passed to a relevance evaluator.

The evaluator produces a structured result containing:

Verdict
Confidence
Justification

For example:

VERDICT: RELEVANT

CONFIDENCE: 92.4%

The retrieved chunks contain information
directly related to the user's question.

The important thing is that the system now has a decision point.

Instead of:

Retrieve → Generate

the pipeline becomes:

Retrieve
   ↓
Evaluate
   ↓
Is the context good enough?

9. What if the local documents don't have the answer?

This is probably my favorite part of the architecture.

If the retrieved context is relevant enough, the pipeline continues normally:

Retrieved Chunks
       ↓
   CRAG Grader
       ↓
    Relevant
       ↓
 Local Context
       ↓
      LLM

But if the evaluator decides the context isn't sufficient:

Retrieved Chunks
       ↓
   CRAG Grader
       ↓
  Not Relevant
       ↓
Optimized Search Query
       ↓
    Web Search
       ↓
 Additional Context
       ↓
      LLM

So the local vector index isn't treated as the absolute source of truth.

It's one knowledge source.

If it can't provide enough evidence, the system has another route.

This is the main idea behind the corrective layer:

Don't force the LLM to answer from context that the system already knows is insufficient.

10. The UI lets me watch the whole thing happen

This is really what separates RAG Studio from a normal RAG demo for me.

I can run the pipeline step by step instead of waiting for one final response.

I can inspect the intermediate state.

I can look at the vectors.

I can see the retrieval scores.

I can inspect the CRAG decision.

I can see whether the web fallback was triggered.

And I can inspect what eventually gets assembled into the LLM context.

So instead of only seeing:

Question → Answer

I can see:

Question
   ↓
Query Embedding
   ↓
Similarity Search
   ↓
Retrieved Chunks
   ↓
Relevance Evaluation
   ↓
Routing Decision
   ↓
Context Assembly
   ↓
LLM
   ↓
Answer

That makes debugging much easier.

The technology behind it

The main stack is:

Technology	Where I use it
TypeScript 5+	Main application, state, APIs and vector operations
Next.js	Application framework
React / JSX	UI
Zustand	Client-side state management
Tailwind CSS	Styling and responsive UI
SVG	Interactive DAG and architecture visualizations
Transformers.js	Browser-side model inference
ONNX Runtime	Local model execution
WebAssembly	Fast browser-side inference
Web Workers	Running embedding workloads away from the UI thread

One thing I particularly enjoyed was building the DAG visualization myself rather than treating it as just an image.

The nodes, connections and interactions are part of the application.

What I actually wanted to build

I wasn't trying to build another chatbot.

I wanted to make the RAG pipeline itself something you can explore.

A normal RAG application might look like:

Upload
  ↓
Embed
  ↓
Search
  ↓
LLM

RAG Studio is closer to:

                 ┌──────────────┐
                 │   Document   │
                 └──────┬───────┘
                        ↓
                 ┌──────────────┐
                 │     ETL      │
                 │ Clean/Redact │
                 └──────┬───────┘
                        ↓
                    Chunking
                        ↓
                Local Embeddings
                        ↓
                   Vector Index
                        ↓
                     Query
                        ↓
                 Similarity Search
                        ↓
                  CRAG Evaluation
                    ↙        ↘
               Relevant    Insufficient
                  ↓             ↓
             Local Context   Web Search
                    ↘        ↙
                     Context
                        ↓
                       LLM

There are more moving parts, but I think that's actually the point.

When something goes wrong, there are more places to look.

What I learned building it

The biggest thing I took away from this project is that RAG isn't just retrieval.

There are a lot of decisions happening before the LLM ever sees the prompt.

The source data matters.

The cleaning matters.

The chunking matters.

The embeddings matter.

The retrieval strategy matters.

And even after retrieving something that looks relevant, you still need to ask whether it is actually sufficient.

That's why I started thinking about RAG less as:

"Search some vectors and ask an LLM."

and more as:

"Build a pipeline that can inspect its own intermediate state and recover when one path isn't good enough."

That's what I wanted RAG Studio to explore.

Not just getting an answer.

But being able to look at the pipeline and understand how the system got there.

Try it

I've made the project available here:

RAG Studio → rayulumukku.com/projects/rag-studio/

If you're building RAG systems yourself, I'd be interested in hearing how you're handling retrieval evaluation, document preprocessing, and fallback strategies.

The interesting part of RAG isn't just the final answer. It's everything that happens before it.

DEV Community: Rayulu Mukku