Hello, I'm Rijul, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product.
You have the information in your document. You know it's there.
But when you ask your RAG system about it, the model gives an answer that misses it completely.
Why?
In RAG, the model can only answer using the pieces of text that are retrieved.
And those pieces of text depend heavily on how the document was split into chunks.
If the information is split badly, it might never reach the model in the right form, even though it is clearly present in the original document.
Let's first go through a quick overview of how RAG works.
How RAG works
- You split your documents into small pieces called chunks.
- Each chunk is turned into a number vector called an embedding and stored.
- When someone asks a question, the system finds chunks that are relevant to the question.
- Those chunks are passed to the LLM, which uses them to generate an answer.
The LLM does not necessarily see your whole document. It usually sees only the chunks that are retrieved.
Chunking determines what information can be retrieved and shown to the model.
Why chunking matters
Retrieval sets the ceiling
If the information you need is not present in the retrieved chunks, the model cannot use it.
This means retrieval puts a ceiling on how accurate the answer can be. Even if the LLM is capable of answering the question, it cannot recover information that was never provided to it.
Chunks that are too big become less focused
Suppose you have a large chunk that covers five different topics.
When this chunk is converted into an embedding, its representation captures information about all of those topics.
Now imagine someone asks about just one specific topic.
The chunk may still be retrieved, but its representation is not focused only on that topic. The other topics can make the match less precise.
So, very large chunks can make retrieval less focused.
Chunks that are too small lose context
Now imagine that a chunk contains only a single sentence.
That sentence might depend on the sentences before it to make sense.
Without that surrounding context, the chunk may not contain enough information to understand what it is talking about.
So, making chunks too small can also hurt retrieval.
Bad cuts can break ideas apart
Suppose a paragraph contains one complete idea, but you split it right in the middle.
Now the two chunks contain only parts of that idea.
If the relevant information is split across chunks, retrieving only one of them may not give the LLM enough context to understand the full idea.
This is why simply cutting text into arbitrary pieces can cause problems.
Because of these issues, there are different ways to decide where one chunk should end and another should begin.
Let's look at some common strategies.
Common chunking strategies
Fixed-size chunking
This splits the document into chunks containing a fixed number of tokens.
For example, you might create a new chunk every 500 tokens.
It is simple, but it does not care about the meaning of the text. It can cut a sentence, paragraph, or idea in the middle.
Fixed-size chunking with overlap
This is similar to fixed-size chunking, but consecutive chunks share some text.
For example:
Chunk 1: A B C D E F
Chunk 2: E F G H I J
The overlap helps preserve some context when an idea happens to cross a chunk boundary.
Recursive or structure-aware chunking
Instead of immediately cutting at an arbitrary token count, the system tries to preserve the document's structure.
It might first split by headings, then paragraphs, then sentences, using smaller units only when necessary.
This helps keep related content together.
Semantic chunking
Semantic chunking tries to identify where the topic or meaning changes.
Instead of asking only:
"Have we reached 500 tokens?"
it asks something closer to:
"Has the topic changed enough that this should become a new chunk?"
This can produce more meaningful chunks, but it generally requires additional processing.
Parent-child chunking
Here, retrieval happens using smaller child chunks, but when a relevant child is found, the system can provide the larger parent section to the LLM.
This gives retrieval a focused unit to search while still providing more surrounding context to the model.
Contextual chunking
Here, additional context is added to each chunk to explain where it came from or what it represents.
For example, a chunk might be accompanied by information about its document, section, or surrounding context.
This can help the retrieval system and the LLM interpret the chunk more accurately.
Wrapping up
There is no single way to split a document into chunks.
The goal is to create chunks that are focused enough to retrieve accurately, while containing enough context to preserve their meaning.
If the chunks are too large, retrieval can become less focused.
If they are too small, important context can be lost.
And if the boundaries break apart ideas, the LLM may never receive the information it needs in a usable form.
So before blaming the LLM for a bad RAG answer, it is worth looking at something much earlier in the pipeline:
How did you split the document in the first place?
Your team's attention is limited, and the deluge of AI-generated code is making it harder to keep production reliable and secure without slowing you down.
I'm building LiveReview, a blast-radius aware AI code review built for your business-critical systems.
Instead of presenting every diff with equal emphasis, LiveReview scores each change by blast radius — how far its impact reaches through your call graph — so you can focus attention where it actually matters.
Spend code review effort where business risk is highest — not spread evenly across every diff.
⭐ Star it on GitHub:
HexmosTech
/
LiveReview
Blast-Radius Aware AI Code Review for Business-Critical Systems
LiveReview: Blast-Radius Aware AI Code Review for Business-Critical Systems
LiveReview is an AI code reviewer that scores every hunk of a diff by blast radius: how far a change reaches through your call graph, how much persistent state it touches, and how well-tested it is. A 3-line change to a shared auth check can outrank a 300-line UI tweak. Your team's attention goes to the highest-risk code first, not spread evenly across every diff.
blast-radius-demo.mp4
LiveReview's Blast Radius & Review Priority scoring, live in the diff viewer.
The exact math, not a black box
Visualize blast radius at a glance
Every factor that feeds the score
Here's the goal:
- A 3-line fix in a function used by 40 other files, that also writes to a database, should score high.
- A 300-line UI change in one file, fully covered by…
Click below to try LiveReview with your codebase:
Top comments (1)
Financial docs killed us with this - same company referred to as 'ACME Corp' in one section and 'the Group' 50 pages later, and the chunk from either end had no idea they were connected. We eventually put a graph layer in front that had entity aliases pre-resolved, and that's what actually moved the needle. Chunk size tuning didn't really help. If I were starting over I'd look at what entities are in the document and how they relate before worrying about chunk boundaries at all.