Simple and cheap RAG - genai-toolbox and pgvector

#rag #ai #database #postgres

I recently hit a common architectural fork in the road while building my ADK (Agent Development Kit) application.

Initially, I was using Chroma as my RAG (Retrieval-Augmented Generation) backend. It works perfectly for local development, but things got complicated when moving to the cloud. I needed a production-ready, resilient solution that didn't involve managing stateful assets on a Mac Mini or paying for a separate managed vector database.

The solution was already right in front of me: Postgres 🐘.

By using pgvector, you can turn your relational database into a powerful vector store. This is especially seamless if you use Supabase, which can host small databases for free or provide a "Pro" tier for a reasonable price.

Why I Swapped MCP for genai-toolbox

In my previous setup, I used an chroma mcp server. However, for my relational data, I was already relying heavily on genai-toolbox.

I realized I could simplify my architecture significantly by dropping the separate MCP server and Chroma dependency in favor of extending my tools.yaml. In engineering, if I can remove a dependency without losing functionality, I don't hesitate.

Here is the configuration change:

embeddingModels:
  gemini-emb:
    type: gemini
    model: text-embedding-004
    dimension: 768

tools:
  add-document:
    kind: postgres-sql
    source: naati-db
    description: >-
      Adds a document to the RAG database. The 'content' will be automatically embedded.
    parameters:
      - name: content
        type: string
        description: The raw text content to be stored in the database.
      - name: vector_string
        type: string
        # This parameter is hidden from the LLM.
        # It automatically copies the value from 'content' and embeds it.
        valueFromParam: content
        embeddedBy: gemini-emb
        description: This parameter is hidden from the LLM.
      - name: metadata
        type: string
        description: >-
          JSON string representing metadata (e.g., '{"file_name": "tos.docx"}').

  query-documents:
    kind: postgres-sql
    source: naati-db
    description: >-
      Performs a semantic similarity search on the RAG database using the provided query text.
    parameters:
      - name: query
        type: string
        description: >-
          The search query text to embed and search for.
        embeddedBy: gemini-emb
      - name: limit
        type: integer
        description: >-
          Maximum number of results to return.
        default: 5
    statement: >-
      SELECT content, metadata
      FROM documents
      ORDER BY embedding <=> $1::vector
      LIMIT $2::int;