DEV Community

Cover image for what is rag ai: a no-PhD guide to retrieval-augmented generation
Sam Chen
Sam Chen

Posted on Originally published at getaab.com

what is rag ai: a no-PhD guide to retrieval-augmented generation

#ai #automation #nocode #tutorial

RAG is a technique that combines a large language model (LLM) with a vector store of embedded text chunks to retrieve relevant context at query time. In practice you feed a user prompt to the LLM, first pull the most relevant passages from an external knowledge base, then let the model generate a response that is grounded in those passages. This matters because it dramatically reduces hallucinations, lets you keep the model's knowledge up-to-date without costly fine-tuning, and lets you answer domain-specific questions with a single API call.

What you need

Tool Plan / Price Role
n8n (automation) Self-hosted (Docker, free) or n8n Cloud Free (2 000 executions/month) Orchestrates embedding, storage, and LLM calls
OpenAI API (gpt-4/embeddings) Pay-as-you-go ≈ $0.03 / 1 k input tokens, $0.04 / 1 k output tokens; embeddings $0.0004 / 1 k tokens Generates answers and creates vector embeddings
Pinecone (vector DB) Free tier = 1 M vector operations / month; paid starts at $5/month for additional capacity Stores and retrieves document embeddings
Source documents (PDF/HTML/MD) Your own files (no cost) Raw knowledge you want the LLM to cite
Docker (optional) Free Runs n8n locally if you prefer self-hosted

Estimated build time: 2-3 hours for a minimal proof-of-concept, 1-2 days for a production-ready pipeline with monitoring.

Step-by-step build

  1. Prepare your document corpus

    • Place all text files (PDFs, markdown, HTML) in a folder ./docs.
    • Use a simple Python script (or n8n "Read Binary File" node) to extract raw text.

  2. Chunk the text

    • Split each document into ~200-token chunks to stay within the LLM's context window.
    • In n8n, add a Function node with JavaScript: 
 // This node receives `content` as a string and returns an array of chunks
 const maxTokens = 200;
 const words = $json["content"].split(/\s+/);
 const chunks = [];
 for (let i = 0; i < words.length; i += maxTokens) {
 chunks.push(words.slice(i, i + maxTokens).join(' '));
 }
 return [{ json: { chunks } }];
Enter fullscreen mode Exit fullscreen mode
  1. Create embeddings for each chunk
    • Add an HTTP Request node that calls OpenAI's embedding endpoint. 
 {
 "method": "POST",
 "url": "https://api.openai.com/v1/embeddings",
 "headers": {
 "Authorization": "Bearer {{ $env.OPENAI_API_KEY }}",
 "Content-Type": "application/json"
 },
 "body": {
 "model": "text-embedding-ada-002",
 "input": "{{$json.chunk}}"
 },
 "responseFormat": "json"
 }
Enter fullscreen mode Exit fullscreen mode

What this does: Sends each 200-token chunk to OpenAI and receives a 1536-dimensional vector.

  1. Store embeddings in Pinecone
    • Use another HTTP Request node pointing to Pinecone's upsert endpoint (https://{index}.svc.{region}.pinecone.io/vectors/upsert).
    • Payload example: 
 {
 "vectors": [
 {
 "id": "doc-{{ $json.docId }}-{{ $json.chunkIdx }}",
 "values": {{ $json.response.data[0].embedding }},
 "metadata": { "text": "{{ $json.chunk }}" }
 }
 ]
 }
Enter fullscreen mode Exit fullscreen mode
  1. Build the retrieval-augmented query workflow
    • Trigger: HTTP Webhook node (exposes /query).
    • Step A - Embed the user query: Same HTTP Request node as in step 3, but with the user's prompt as input.
    • Step B - Retrieve top-k similar chunks: HTTP Request to Pinecone's query endpoint, requesting topK=5. 
 {
 "method": "POST",
 "url": "https://{index}.svc.{region}.pinecone.io/query",
 "headers": {
 "Authorization": "Bearer {{ $env.PINECONE_API_KEY }}",
 "Content-Type": "application/json"
 },
 "body": {
 "vector": {{ $json.response.data[0].embedding }},
 "topK": 5,
 "includeMetadata": true
 },
 "responseFormat": "json"
 }
Enter fullscreen mode Exit fullscreen mode

What this does: Finds the five most relevant document chunks for the user's question.

  1. Compose the final prompt for the LLM
    • Use a Set node to concatenate the retrieved texts into a system prompt: 
 You are an assistant that answers using only the provided context. Context:
 {{ $json.results.map(r => r.metadata.text).join('\n---\n') }}

 Question: {{ $json.question }}
Enter fullscreen mode Exit fullscreen mode
  1. Call the LLM
    • Add an HTTP Request node to OpenAI's chat completion endpoint (https://api.openai.com/v1/chat/completions). 
 {
 "method": "POST",
 "url": "https://api.openai.com/v1/chat/completions",
 "headers": {
 "Authorization": "Bearer {{ $env.OPENAI_API_KEY }}",
 "Content-Type": "application/json"
 },
 "body": {
 "model": "gpt-4",
 "messages": [
 { "role": "system", "content": "{{ $json.composedPrompt }}" }
 ],
 "temperature": 0.2,
 "max_tokens": 500
 },
 "responseFormat": "json"
 }
Enter fullscreen mode Exit fullscreen mode
  1. Return the answer
    • Connect the response to the Webhook node's return JSON: 
 {
 "answer": "{{ $json.choices[0].message.content }}",
 "sources": {{ $json.results.map(r => r.id) }}
 }
Enter fullscreen mode Exit fullscreen mode
  1. Deploy and test
    • Run the workflow locally (docker compose up -d n8n) or on n8n Cloud.
    • POST a JSON payload to https://your-n8n-instance.com/webhook/query with {"question":"What is rag ai?"}.
    • Verify that the answer cites the retrieved chunks (the sources array).

Result: You now have a live endpoint that answers "what is rag ai" (or any domain question) by grounding the response in your own knowledge base, dramatically reducing hallucinations.

Where this breaks

Failure mode Why it happens Mitigation
Rate limits on OpenAI embeddings Free tier caps at 3 000 requests/minute; higher usage can be throttled. Batch chunks, add a Delay node, or upgrade to a paid plan.
Pinecone vector-store quota Free tier limits 1 M operations/month; large corpora exceed it quickly. Monitor usage via Pinecone dashboard; switch to a paid plan before hitting the limit.
Context window overflow gpt-4's window is 8 192 tokens; concatenating too many chunks exceeds it. Restrict topK to 3-5 chunks and truncate each to ≤ 200 tokens (as done in step 2).
Embedding drift Adding new docs without re-embedding old ones can skew similarity scores. Re-run the ingestion pipeline nightly or trigger on document change.
Token cost blowout Each query incurs embedding + LLM tokens; heavy traffic can become pricey. Cache query embeddings for repeated questions, set temperature=0 to reduce token usage, and enforce rate limiting at the webhook.
Authentication expiry API keys rotated or expire after 90 days in some orgs. Store keys in n8n's Credentials and set a reminder to rotate them; the workflow fails gracefully if a 401 is returned.

Key truth: RAG does not eliminate hallucinations outright, but it cuts them by roughly 30-40 % when the retrieved context is high-quality (see OpenAI's best-practice guide).

For a deeper technical reference, see n8n's documentation.

FAQ

What is rag ai in plain language?

RAG (retrieval-augmented generation) is a method that first looks up relevant pieces of text from a vector database and then feeds those pieces to an LLM so the answer is anchored in real content.

How does chunking affect retrieval quality?

Chunking creates uniformly sized snippets that fit inside the LLM's context window. Smaller, well-defined chunks improve similarity matching because each vector represents a coherent idea, reducing noise in the top-k results.

Can I use a self-hosted vector store instead of Pinecone?

Yes. Open-source options like Weaviate, Milvus, or Qdrant run in Docker for free. Replace the Pinecone HTTP nodes with the equivalent endpoints of your chosen DB; the rest of the workflow stays identical.

Will RAG work with any LLM, not just OpenAI's?

In principle, any model that accepts a prompt can be used. You just need an embedding model compatible with your vector store (e.g., Cohere, HuggingFace's sentence-transformers) and adjust the chat-completion request format.

How do I prevent the system from leaking proprietary data?

Store embeddings in a private VPC-isolated Pinecone index or a self-hosted vector DB behind your firewall. Ensure the webhook is authenticated (API key or OAuth) and audit query logs regularly.

Where can I get more help building RAG pipelines?

Check out the RAG Support Agent for a ready-made n8n template and step-by-step walkthrough, or grab the free guide for a deeper dive into advanced chunking strategies and monitoring practices.

Ready to ship a production-grade RAG service? Grab the template from the RAG Support Agent and start scaling today.

Related reading

Top comments (0)