If you've worked with RAG (Retrieval-Augmented Generation), you've probably run into a surprisingly annoying problem: how should the document actually be split before creating embeddings?
It sounds simple.
Take a document, split it into smaller pieces, generate embeddings, put them into a vector database, and let the LLM retrieve the relevant chunks.
In practice, the way you chunk the document can have a pretty noticeable impact on retrieval quality.
Split things too aggressively and you can lose context. Make the chunks too large and retrieval becomes less precise. Use the wrong separator and you might end up breaking a paragraph, code block, list, or even a sentence in a completely unhelpful place.
That's why I built a small tool to make experimenting with this process easier.
RAG Text Chunking & Semantic Splitter
The RAG Text Chunking & Semantic Splitter lets you paste a document and experiment with different ways of splitting it into chunks.
You can use it to get a quick idea of what your data will look like before sending it through an embedding pipeline.
👉 https://omnikite.vercel.app/tools/ai/chunk-splitter
The tool runs entirely in the browser, so there's no need to upload your document to a server just to test a few chunking strategies.
Why does chunking matter?
Imagine you have documentation like this:
Authentication
Users can authenticate using email/password or OAuth.
OAuth supports Google and GitHub.
Access tokens expire after 60 minutes.
If you split this content into tiny pieces, you could end up with chunks that don't contain enough context.
For example:
OAuth supports Google and GitHub.
That's technically useful, but the chunk doesn't explain that OAuth is one of the authentication methods supported by the application.
On the other hand, putting several pages of documentation into one huge chunk isn't particularly helpful either.
The retriever may return a large amount of unrelated information when the user only needs one small section.
The goal is to find a reasonable balance.
Different documents need different strategies
There isn't really a magical chunk size that works perfectly for every RAG application.
A technical documentation site might work well with one strategy.
A collection of research papers might need another.
Legal documents, Markdown files, source code, product documentation, FAQs, and conversational data can all have very different structures.
That's why experimenting with the actual text is often more useful than blindly choosing a number like 500 tokens and hoping for the best.
What can you experiment with?
The splitter is designed for exactly this kind of quick experimentation.
You can paste your text and see how different chunking approaches affect the resulting pieces.
For example, you can compare things such as:
- Fixed-size chunking
- Character-based splitting
- Separator-based splitting
- Sentence-oriented splitting
- Overlapping chunks
- Semantic-style splitting
Seeing the chunks directly makes it much easier to spot problems.
Maybe a heading gets separated from its content.
Maybe a code example gets split in the middle.
Maybe your chunks are unnecessarily large.
Or maybe you're creating dozens of tiny chunks that don't contain enough information to be useful.
Those things are much easier to notice when you can actually see the output.
A useful tool during RAG development
I think tools like this are most useful before you build the full pipeline.
Instead of repeatedly modifying your ingestion code, uploading documents, generating embeddings, indexing everything into a vector database, and then testing retrieval...
You can first play around with the raw text and figure out what kind of chunks make sense.
Then you can take that configuration into your actual RAG pipeline.
It doesn't replace proper evaluation, of course. Ultimately, you still want to test retrieval quality with real queries and real documents.
But it's a convenient way to understand what's happening between the document and the embedding model.
No upload required
One of the things I wanted to keep simple was privacy.
The tool runs 100% client-side, so your text doesn't need to leave the browser just to experiment with chunking.
That's particularly useful when you're working with internal documentation, private notes, source code, or other content that you don't want to casually paste into another service.
No account is required either.
Just open the tool, paste some text, change the settings, and see what happens.
Give it a try
If you're currently building a RAG application, working with vector databases, or just trying to understand how document preprocessing affects retrieval, you might find it useful.
RAG Text Chunking & Semantic Splitter:
https://omnikite.vercel.app/tools/ai/chunk-splitter
It's a small tool, but sometimes the small parts of a RAG pipeline are where the interesting problems hide.
Top comments (0)