When retrieval in a RAG pipeline comes back with the wrong passage, the first suspect is almost always the embedding model. Swap text-embedding-3-small for something with a bigger benchmark number, re-index, measure again. That is usually the wrong knob. If the chunk you pulled back stops halfway through the sentence that contained the answer, the answer is not in your index at all, and no embedding model is going to reconstruct it.
Chunking gets decided once, in a constructor argument, and then nobody looks at it again. So I looked at it: four strategies, one document (RFC 9562, the UUID specification, 114,629 characters), a 1,000-character target, and one metric that chunking libraries do not report. Plain fixed-size splitting landed a boundary mid-sentence 98 percent of the time. Paragraph-aware splitting managed zero. The full write-up with the complete script is on DevToolLab.
Four Ways to Cut a Document
Fixed size slices every N characters. Trivial to write, perfectly uniform output, and it has no idea what the text says.
Fixed size with overlap does the same thing, then copies the tail of each chunk onto the head of the next one, betting that whatever got severed will survive intact in at least one of the pair.
Recursive works down a ladder of separators, paragraph break first, then newline, then sentence, then space, and only drops to the next rung when a piece is still too big. Most frameworks ship this as the default.
Paragraph aware treats a paragraph as atomic. It fills a chunk with whole paragraphs and opens a new one when the budget runs out.
The Metric Nobody Reports
Chunk size is not the interesting number. Boundary placement is. A cut that lands on a sentence terminator or a blank line costs you nothing, because both halves are still coherent units. A cut anywhere else takes one idea and turns it into two useless fragments.
So the measurement is: collect every legal boundary in the document up front, then ask each strategy how many of its cuts missed. The test corpus is an RFC because RFCs have real paragraph structure and a plain-text canonical form, which makes them easy to fetch and hard to argue with.
The scoring half is about ten lines of Node with no dependencies:
// Every position where a sentence or a paragraph legitimately ends.
const sentenceEnds = [...doc.matchAll(/[.!?]["')\]]?\s+(?=[A-Z0-9])/g)].map((m) => m.index + m[0].length)
const paragraphEnds = [...doc.matchAll(/\n\s*\n/g)].map((m) => m.index + m[0].length)
const SAFE = new Set([...sentenceEnds, ...paragraphEnds, 0, doc.length])
// A chunker returns [start, end] pairs. Every start after the first is a cut it chose.
const midSentence = (chunks) => chunks.slice(1).filter(([start]) => !SAFE.has(start)).length
And the strategy that scores zero is not complicated either, which is the uncomfortable part:
// Never split a paragraph. Pack whole paragraphs until the budget is gone.
const byParagraph = (text) => {
const out = []
let start = 0, cur = 0
for (const end of [...paragraphEnds, text.length]) {
if (end - start > SIZE && cur > start) { out.push([start, cur]); start = cur }
cur = end
}
if (cur > start) out.push([start, cur])
return out
}
The original article carries the whole runnable file, including the recursive splitter and the corpus cleanup that strips RFC page furniture.
What Came Back
Node 25.5.0, run September 14, 2026:
document: RFC 9562, 114,629 chars, 616 sentences
target chunk size: 1000 chars (overlap 200 where used)
strategy chunks avg size mid-sentence mid-para total chars
------------------------------------------------------------------------------
fixed size 115 997 112 (98%) 112 114,629
fixed + overlap 144 995 142 (99%) 142 143,229
recursive 147 780 15 (10%) 15 114,629
paragraph aware 132 868 0 (0%) 0 114,629
112 bad cuts out of 114 is not a tail case you can ignore, it is what the strategy does on every document that is not pre-chopped into 1,000-character pieces.
The overlap row is the one worth staring at. It cuts mid-sentence at 99 percent, slightly worse than plain fixed size, because shifting the stride by 800 instead of 1,000 does nothing to align it with the prose. What overlap actually buys is a duplicate of the damaged region living inside the neighbor, and the bill for that shows up in the last column: 143,229 characters embedded instead of 114,629. That is 25 percent more vectors, 25 percent more storage, 25 percent more spend at every re-index, to paper over a boundary problem that recursive splitting removes for free.
Recursive drops the bad-cut rate to 10 percent and costs nothing but uniformity: 147 chunks averaging 780 characters against fixed size's tidy 115 at 997.
Where It Stops Being Free
Structure-aware splitting produces uneven chunks, and that is a real tradeoff rather than a rounding error. A 200-character chunk and a 900-character chunk do not carry comparable specificity once embedded, so cosine similarity between them is not quite apples to apples, and short chunks can punch above their weight in a ranked list. Recursive sits in the middle of that tension, which is a decent explanation for why it is everyone's default.
Paragraph-aware hitting exactly zero also depends on the corpus. RFC paragraphs happen to fit under a 1,000-character budget. Point it at a document with 3,000-character paragraphs and it either blows the budget or falls back to something else, so check your source before assuming the zero transfers.
Five Things to Do With This
- Move off plain fixed size today. Recursive is the same single config line and removes roughly 90 percent of the damage. For prose, there is no case where fixed size is the better pick.
- Go paragraph-aware on structured text. Specs, contracts, API docs, anything with genuine paragraph markup can reach zero mid-sentence cuts.
- Treat overlap as a last resort, not a default. Here it added 25 percent to the embedding bill and improved nothing. It earns its keep only when the text has no usable separators at all, like untimed transcripts or raw OCR.
- Score boundaries, not sizes. Every library will happily print its chunk length distribution. None of them tell you how many ideas got bisected, and that is the number correlated with retrieval quality.
- Budget in tokens, not characters. Roughly 250 English tokens fit in 1,000 characters, and far fewer in most other languages, so a character budget quietly shrinks your chunks the moment your corpus is not English.
Clean the Input First
Half the boundary damage in real pipelines comes from garbage the chunker was never designed to see. Markdown link syntax and heading markers eat budget without carrying meaning, so run documents through Markdown to Text before embedding. Zero-width and non-breaking characters from PDF and web scrapes are worse, because they survive every normalization step and sit invisibly inside what should be a clean sentence boundary; Invisible Character Remover strips them.
Conclusion
The chunking default in most RAG stacks severs a sentence at nearly every boundary it creates, and the standard remedy for that, overlap, charges a 25 percent embedding tax while leaving the cut rate untouched. Switching to recursive splitting is a one-line change that eliminates most of the problem, and on structured documents paragraph-aware eliminates all of it. Before you spend another sprint tuning the retriever or shopping for a better embedding model, run the boundary count on one of your own documents. The number is usually embarrassing.
References
- RAG Chunking Strategies, Measured - the original article, with the complete script and full methodology
- RFC 9562: Universally Unique IDentifiers - the test corpus
- Best RAG Platforms and Tools - what runs the pipeline once chunking is settled
- Best PDF Parsers for RAG - the ingestion step where structure is preserved or lost
- How LLM Tokenization Actually Works - why characters and tokens are not interchangeable budgets
Top comments (0)