DEV Community

Autonomous Studio
Autonomous Studio

Posted on

How to use RAG Production Automation Kit

Introduction

Retrieval‑augmented generation (RAG) has become the de‑facto pattern for building AI‑enabled SaaS platforms that need up‑to‑date, factual answers from large knowledge bases. While the concept is simple—retrieve relevant chunks, feed them to a LLM, and let the model generate a response—the engineering effort to make it reliable, scalable, and maintainable can be daunting.

The RAG Production Automation Kit from VectorFlow abstracts away the repetitive plumbing and gives you a ready‑to‑run, production‑grade pipeline. In this tutorial we’ll walk through a typical use‑case, set up the kit, and programmatically turn a raw document collection into a live RAG service.


The Problem

  1. Data ingestion – Raw PDFs, markdown files, or API feeds must be chunked, embedded, and indexed.
  2. Retriever selection – Choosing the right similarity search (e.g., dense vector vs. BM25) and tuning its parameters is non‑trivial.
  3. Prompt engineering – You need a consistent prompt template that injects retrieved context without exceeding token limits.
  4. Observability – Production systems require logging, latency metrics, and fallback strategies.

Doing all of this manually leads to duplicated code, hidden bugs, and a long time‑to‑value for your product team.


How the Automation Kit Solves It

The kit bundles:

  • Data pipelines that automatically chunk, embed (via any OpenAI‑compatible or local model), and push vectors to a configurable vector store.
  • Retriever adapters for popular stores (FAISS, Pinecone, Qdrant) with sane defaults and easy hyper‑parameter overrides.
  • Prompt templates that respect token budgets and expose a clean API for customisation.
  • Telemetry hooks (OpenTelemetry, Prometheus) baked into the request flow.

All of this is orchestrated through a small declarative YAML file, letting you focus on business logic instead of glue code.


Setting Up the Environment

Prerequisite – Python 3.9+, a virtual environment, and an API key for your embedding model (e.g., OpenAI, Cohere, or a local HuggingFace endpoint).

# 1️⃣ Create and activate a venv
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate

# 2️⃣ Install the kit and its dependencies
pip install rag-production-automation-kit
Enter fullscreen mode Exit fullscreen mode

Step 1 – Install the Kit

The package is distributed on Gumroad, but the pip install pulls the latest public release. Verify the installation:

python -c "import rag_kit; print(rag_kit.__version__)"
Enter fullscreen mode Exit fullscreen mode

You should see a version string like 0.4.2.


Step 2 – Prepare Your Knowledge Base

Place all source files (PDF, markdown, HTML) under a single directory, e.g., ./data/docs. The kit will walk the tree recursively.

# data_pipeline.yaml
source:
  path: ./data/docs
  formats: [pdf, md, html]

chunking:
  strategy: sliding_window
  size: 512          # tokens
  overlap: 64

embedding:
  provider: openai
  model: text-embedding-3-large
  api_key: ${OPENAI_API_KEY}
Enter fullscreen mode Exit fullscreen mode

Run the ingestion command:

rag-kit ingest data_pipeline.yaml
Enter fullscreen mode Exit fullscreen mode

The command:

  • extracts text,
  • splits it into overlapping chunks,
  • generates dense embeddings,
  • writes vectors to the default FAISS index (./index/faiss.index).

Step 3 – Configure the Retriever

Create a lightweight config that tells the kit how to fetch the most relevant chunks at query time.

# retriever.yaml
store:
  type: faiss
  path: ./index/faiss.index

search:
  top_k: 5
  distance_metric: cosine
  score_threshold: 0.65
Enter fullscreen mode Exit fullscreen mode

You can swap faiss for pinecone or qdrant by changing the type and providing the required connection credentials.


Step 4 – Build the Generation Pipeline

Now wire the retriever to a language model and a prompt template. The kit supports any OpenAI‑compatible endpoint.

# rag_pipeline.yaml
retriever: retriever.yaml

generator:
  provider: openai
  model: gpt-4o-mini
  api_key: ${OPENAI_API_KEY}
  temperature: 0.2
  max_tokens: 256

prompt:
  template: |
    You are a helpful assistant for a SaaS knowledge base.
    Use the following context to answer the question concisely.

    Context:
    {{retrieved_chunks}}

    Question: {{user_query}}
    Answer:
Enter fullscreen mode Exit fullscreen mode

Launch the service locally:

rag-kit serve rag_pipeline.yaml --host 0.0.0.0 --port 8000
Enter fullscreen mode Exit fullscreen mode

The kit starts a FastAPI server exposing two endpoints:

  • POST /query – body { "question": "…" } returns { "answer": "…", "sources": [...] }
  • GET /health – simple health check.

Step 5 – Deploy and Test

For production you’ll likely containerise the service. The kit ships a Dockerfile that copies your configs and the index:

FROM python:3.11-slim
WORKDIR /app
COPY . /app
RUN pip install rag-production-automation-kit
EXPOSE 8000
CMD ["rag-kit", "serve", "rag_pipeline.yaml", "--host", "0.0.0.0", "--port", "8000"]
Enter fullscreen mode Exit fullscreen mode

Build and push:

docker build -t vectorflow/rag-kit:latest .
docker push vectorflow/rag-kit:latest
Enter fullscreen mode Exit fullscreen mode

Deploy to your preferred orchestrator (Kubernetes, ECS, etc.). The kit automatically emits Prometheus metrics at /metrics and logs structured JSON for easy ingestion into ELK or Loki.

Quick sanity test

curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question":"How do I reset my API key?"}'
Enter fullscreen mode Exit fullscreen mode

You should receive a concise answer plus a list of source document IDs, enabling traceability for compliance.


Common Pitfalls & Tips

Symptom Likely Cause Fix
Empty sources array score_threshold too high Lower score_threshold or increase top_k.
Latency > 2 s Using a remote vector store without connection pooling Enable HTTP keep‑alive or switch to a local FAISS index for low‑latency workloads.
Token limit errors Prompt + retrieved chunks exceed model context Reduce chunking.size or enable dynamic truncation (rag-kit truncate).
Inconsistent answers across runs Non‑deterministic temperature Set temperature: 0 for deterministic responses in production.

Conclusion

By following the steps above you’ve turned a raw document collection into a fully‑instrumented RAG service with just a few YAML files and a single command. The RAG Production Automation Kit handles ingestion, retrieval, prompt construction, and observability, letting AI‑enabled SaaS platforms focus on product differentiation rather than infrastructure plumbing.


Next steps – Explore the live demo to see the kit in action and grab the full package for deeper customisation:


Autonomous Studio — Self-hosted AI Micro-APIs & Developer Tools.

Get the full package · Live demo

Top comments (0)