RAG explained: how to give an LLM your own information | Xiliux Blog

#ai #python #llm

A language model knows a great deal about the world, but knows nothing about your company: your manuals, your policies, your products. And if you ask it about something it doesn't know, it may invent an answer that sounds convincing. RAG (Retrieval-Augmented Generation) solves both problems.

The idea

Instead of expecting the model to "already know" your information, you give it to it at the moment of the question:

Index your documents: split them into fragments and turn them into embeddings (numeric vectors that capture the meaning).
When a question comes in, search for the fragments most similar to that question.
Pass the LLM the question together with those fragments and ask it to answer using them.

The model no longer guesses: it answers from a real context that you control. And you can ask it to cite where each fact came from.

The pattern in Python

# 1. Indexado (una vez): fragmentos -> embeddings -> base vectorial
#    (con librerías como sentence-transformers + FAISS, o un servicio gestionado)

# 2. En cada pregunta: recuperar los fragmentos relevantes
fragmentos = base_vectorial.buscar(pregunta, k=4)
contexto = "\n\n".join(fragmentos)

# 3. Generar la respuesta con el contexto
from anthropic import Anthropic
client = Anthropic()

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system="Responde SOLO con la información del contexto. Si no está, dilo.",
    messages=[{
        "role": "user",
        "content": f"Contexto:\n{contexto}\n\nPregunta: {pregunta}",
    }],
)
print(resp.content[0].text)

Note the instruction in the system prompt: asking it to answer only with the context and to admit when it doesn't know is what drastically reduces hallucinations.

What is it good for?

Support assistants that answer with your real documentation.
Internal search in natural language over your manuals or wikis.
Onboarding: a chatbot that knows your processes.

Details that make the difference

How you split the documents (chunking) has a big effect on quality.
Good embeddings and a suitable vector database matter more than the model.
Citing the sources builds trust and allows verification.

RAG is today one of the most cost-effective ways to apply AI in a business: you retrain nothing, and the assistant improves just by updating your documents.

Want an assistant with your data?

At Xiliux I build RAG systems in Python from end to end. Tell me what information you want to put to work: contact.

DEV Community