Building a Production RAG API in Go with pgvector and Gemini

Lessons from building the Restaurant Suggestion System, a Go/Gin backend with a conversational recommendation API.

A RAG demo takes an afternoon: embed some documents, run a similarity search, paste the results into a prompt. A RAG API you can put in front of real users takes longer, because most of the work is not the LLM call. It's everything around it.

I built the Restaurant Suggestion System (RSS) as a Go/Gin backend where users chat to get dish and restaurant suggestions. The answers come from RAG over real menu data in PostgreSQL with pgvector and Gemini embeddings. Because the recommendations have to respect allergies, "the model usually gets it right" was never good enough.

This post walks through the shape of that pipeline and the Go patterns that made it dependable. The snippets are simplified and focused on the ideas, not copied from the private repo.

The pipeline

Every chat request goes through the same steps:

1. Embed the user's message (plus stored preferences) with Gemini.
2. Retrieve candidates from Postgres with pgvector, with allergen and health filters applied in SQL.
3. Score candidates deterministically in Go.
4. Generate a conversational answer with the LLM, using only the top candidates as context.
5. Validate the answer against the candidates we sent.
6. Fall back to a vector-only response if the LLM is slow or unavailable.

The rule behind all of it: the LLM explains and converses. It does not decide what is safe or what exists.

Schema: keep filters as real columns

It's tempting to put everything into the embedded text and let similarity sort it out. Don't do that for anything safety-related. Allergens belong in a column you can filter on exactly.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE menu_items (
    id            BIGSERIAL PRIMARY KEY,
    restaurant_id BIGINT NOT NULL REFERENCES restaurants(id),
    name          TEXT NOT NULL,
    description   TEXT NOT NULL,
    allergens     TEXT[] NOT NULL DEFAULT '{}',
    price_cents   INT NOT NULL,
    is_available  BOOLEAN NOT NULL DEFAULT TRUE,
    embedding     vector(768) NOT NULL
);

CREATE INDEX ON menu_items USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON menu_items USING gin (allergens);

The dimension (768 here) must match the embedding model's output configuration. Pick one, store it in config, and fail fast at startup if they disagree. A silent dimension mismatch is a miserable bug to find in production.

Embeddings with Gemini

RSS uses Gemini embeddings in two places: when indexing menu items, and during onboarding to capture user preferences as a vector. Wrap the provider behind a small interface so the rest of the code doesn't care which SDK you use:

type Embedder interface {
    Embed(ctx context.Context, text string) ([]float32, error)
}

type GeminiEmbedder struct {
    client *genai.Client
    model  string
    dims   int32
}

func (g *GeminiEmbedder) Embed(ctx context.Context, text string) ([]float32, error) {
    ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
    defer cancel()

    res, err := g.client.Models.EmbedContent(ctx, g.model,
        genai.Text(text),
        &genai.EmbedContentConfig{OutputDimensionality: &g.dims},
    )
    if err != nil {
        return nil, fmt.Errorf("embed: %w", err)
    }
    if len(res.Embeddings) == 0 || len(res.Embeddings[0].Values) != int(g.dims) {
        return nil, errors.New("embed: unexpected embedding shape")
    }
    return res.Embeddings[0].Values, nil
}

Two habits worth copying: a hard timeout on every external call, and a shape check on the response. The interface also makes tests easy, since a fake embedder returns fixed vectors.

Retrieval: filter in SQL, before the LLM

This is the most important design decision in the system. Allergen and health filters run in the retrieval query, so excluded items never reach the model. A prompt like "don't suggest peanuts" is a request. A WHERE clause is a guarantee.

type Candidate struct {
    ID         int64
    Name       string
    Allergens  []string
    PriceCents int
    Distance   float64
}

const retrieveSQL = `
SELECT id, name, allergens, price_cents, embedding <=> $1 AS distance
FROM menu_items
WHERE is_available
  AND NOT (allergens && $2::text[])
ORDER BY embedding <=> $1
LIMIT $3`

func (r *Repo) Retrieve(ctx context.Context, q []float32, exclude []string, k int) ([]Candidate, error) {
    rows, err := r.db.Query(ctx, retrieveSQL, pgvector.NewVector(q), exclude, k)
    if err != nil {
        return nil, err
    }
    defer rows.Close()

    var out []Candidate
    for rows.Next() {
        var c Candidate
        if err := rows.Scan(&c.ID, &c.Name, &c.Allergens, &c.PriceCents, &c.Distance); err != nil {
            return nil, err
        }
        out = append(out, c)
    }
    return out, rows.Err()
}

One pgvector gotcha: with an approximate index such as HNSW, filtering happens after the index returns its nearest neighbours. With a strict filter you can get fewer than k rows back. pgvector 0.8+ supports iterative index scans (SET hnsw.iterative_scan = relaxed_order;), which keep scanning until enough rows pass the filter. On older versions, raise hnsw.ef_search or over-fetch and trim. Either way, write a test where most rows are filtered out and check you still get results.

Deterministic scoring

Vector distance is one signal, not the ranking. RSS ranks filtered candidates with a deterministic scoring engine, so the same input gives the same order and every recommendation can be explained.

type Weights struct {
    Similarity, Preference, Budget float64
}

func Score(c Candidate, prefSim float64, budgetCents int, w Weights) float64 {
    s := w.Similarity*(1-c.Distance) + w.Preference*prefSim
    if budgetCents > 0 && c.PriceCents <= budgetCents {
        s += w.Budget
    }
    return s
}

func Rank(cands []Candidate, scores map[int64]float64) {
    sort.SliceStable(cands, func(i, j int) bool {
        si, sj := scores[cands[i].ID], scores[cands[j].ID]
        if si != sj {
            return si > sj
        }
        return cands[i].ID < cands[j].ID // stable tie-break
    })
}

Note the tie-break on ID. Without it, equal scores can shuffle between runs, and your tests become flaky. The signals here are illustrative; the point is that ranking lives in plain Go you can unit-test, not in a prompt.

Generation: ground it, then check it

Only the top candidates go into the prompt, each with its ID. A pattern I recommend on top of that: ask the model for structured output that references those IDs, and validate the response before returning it:

type LLMAnswer struct {
    Message string  `json:"message"`
    ItemIDs []int64 `json:"item_ids"`
}

func validateAnswer(a LLMAnswer, sent []Candidate) error {
    allowed := make(map[int64]bool, len(sent))
    for _, c := range sent {
        allowed[c.ID] = true
    }
    for _, id := range a.ItemIDs {
        if !allowed[id] {
            return fmt.Errorf("model referenced unknown item %d", id)
        }
    }
    return nil
}

If the model mentions something you didn't give it, that's a hallucination by definition, and you can detect it in a few lines.

The fallback: vector-only results

LLM APIs time out, hit rate limits and have outages. RSS handles that by degrading gracefully: if the LLM call fails, the API still returns the filtered, scored candidates from vector search. In the sketch below, an answer that fails validation takes the same path:

func (s *Service) Recommend(ctx context.Context, req ChatRequest) (ChatResponse, error) {
    cands, err := s.retrieveAndRank(ctx, req)
    if err != nil {
        return ChatResponse{}, err // no data, no answer
    }

    genCtx, cancel := context.WithTimeout(ctx, 8*time.Second)
    defer cancel()

    ans, err := s.llm.Answer(genCtx, req.Message, cands)
    if err == nil {
        err = validateAnswer(ans, cands)
    }
    if err != nil {
        s.log.Warn("llm unavailable, using vector-only fallback", "err", err)
        return fallbackResponse(cands), nil
    }
    return ChatResponse{Message: ans.Message, ItemIDs: ans.ItemIDs, Source: "llm"}, nil
}

Because the safety filters ran before the LLM, the fallback is just as safe as the full path. It's less chatty, not less correct. Returning a Source field also helps the frontend and your dashboards see how often you're degrading.

The rest of the production checklist

The RAG pipeline is maybe a third of the system. The rest is ordinary backend work that still matters:

• Auth: passwordless email OTP, JWT, and role-based access control so admin endpoints stay admin-only.
• Data ingest: CRUD plus CSV/JSON bulk upload and AWS file upload. Whatever your ingest path, make sure every write also refreshes the item's embedding, or search goes stale.
• Privacy: admin analytics built without exposing PII.
• Ops: Postgres with pgvector in Docker, so local, CI and production run the same database.
• Tests: a Newman (Postman) end-to-end suite that hits the real API. Make sure the fallback path has its own tests too.

Takeaways

• Put safety rules in SQL and Go, never only in the prompt.
• Rank deterministically, with stable tie-breaks.
• Validate model output against the context you sent.
• Design the fallback first. If the LLM disappears, users should still get a correct answer.

Go suits this well: strict types for model output, context deadlines on every call, and a single binary that's easy to run next to Postgres.

---

Pradip Parmar is a Go backend and AI agent engineer who has been writing Go since 2018. He builds RAG systems, MCP servers and LLM agents for businesses, and previously led the Python-to-Go migration of the Flights backend at MakeMyTrip. If you need a Go AI agent developer for a RAG or agent project, you can reach him on LinkedIn, Upwork or Contra, or see his code on GitHub.