Building Production-Grade RAG with Go and Gemini File Search: A Zero-Infrastructure Approach

Originally published on tamiz.pro.

Introduction

The standard architecture for Retrieval-Augmented Generation (RAG) systems has ossified around a complex stack: an ingest pipeline, a heavy-duty vector database (Pinecone, Weaviate, Milvus), and an orchestration layer. This approach often introduces significant operational overhead. You must manage vector indexing, handle schema migrations, deal with embedding model drift, and provision additional infrastructure just to serve the data layer. For many use cases, particularly those involving unstructured enterprise documents (PDFs, legal contracts, technical manuals), this complexity is unnecessary. You don't always need to pre-compute embeddings and store them in a specialized index. You can use on-the-fly semantic search.

Enter Gemini File Search. Google's Gemini API offers a "File Search" tool that allows the model to search across a directory of unstructured files (PDFs, DOCX, TXT, HTML) without requiring you to host them in a vector database. It handles the chunking, embedding, and retrieval internally. By combining this capability with the Go programming language, developers can build production-grade RAG pipelines that are incredibly lean. Go's concurrency model and type safety make it ideal for handling the HTTP layers and data transformations required to glue your application logic to the Gemini API. This article explores how to architect and build such a system, focusing on eliminating the "vector DB tax" while maintaining high retrieval accuracy and system reliability.

The Case for Server-Side Semantic Retrieval

Traditional RAG requires the client or a middle-tier service to know exactly what to look for. You must generate embeddings for your chunks and query a vector store using cosine similarity. This decouples the understanding of the query from the retrieval mechanism. If your user asks a nuanced question that implies a specific search strategy, a standard vector search might miss the context if the embeddings don't align perfectly.

With File Search, the LLM (Gemini) acts as the retrieval engine. It understands the intent of the question, reformulates search queries if necessary, and scans the file contents (using its internal vector indexing service) to find relevant context. This shifts the burden of "semantic matching" to the model.

Advantages of Removing the Vector DB

1. Reduced Latency in Ingestion: There is no queue for embedding generation. You upload files, and they are indexed in the background by Google's infrastructure.
2. No Cold Start for Query: You don't need to wait for vectors to be loaded into memory on a custom vector DB node.
3. Simplicity of Schema: No need to define metadata fields that must be strictly synchronized between your app and the DB. The file itself is the source of truth.

Limitations and Caveats

It is crucial to understand that File Search is best suited for medium-to-large corpora of unstructured text. It is not a replacement for a structured relational database. If your RAG relies heavily on complex joins between tables or precise numeric filtering, a traditional vector DB or SQL engine is still superior. File Search is optimized for document retrieval, not data querying.

Architecting the Go Application

We will build a service that accepts user queries and returns grounded answers using Gemini File Search. The stack will consist of:

• Language: Go 1.21+
• Framework: Standard library net/http (or Chi/Echo for routing)
• Client: google.golang.org/genai
• Configuration: viper for environment management

The architecture follows a clean layered design:

1. Handler Layer: Parses HTTP requests, validates input.
2. Service Layer: Orchestrates the RAG logic, manages the conversation state (if any), and applies system prompts.
3. Data Layer (External): Interacts with Gemini API and Cloud Storage (for file uploads).

Implementing the Core Logic in Go

1. Configuration and Client Initialization

First, we set up the configuration. Production-grade Go apps should never hardcode secrets. We use environment variables for the Gemini API key and the project ID.

package config

import (
    "fmt"
    "os"
)

type Config struct {
    GeminiAPIKey string
    ProjectID    string
    ModelName    string
    MaxTokens    int
    Temperature  float64
}

func LoadFromEnv() (*Config, error) {
    apiKey := os.Getenv("GEMINI_API_KEY")
    if apiKey == "" {
        return nil, fmt.Errorf("GEMINI_API_KEY must be set")
    }

    projectID := os.Getenv("GCP_PROJECT_ID")
    if projectID == "" {
        return nil, fmt.Errorf("GCP_PROJECT_ID must be set")
    }

    return &Config{
        GeminiAPIKey: apiKey,
        ProjectID:    projectID,
        ModelName:    "gemini-1.5-pro", // or flash for lower latency
        MaxTokens:    2048,
        Temperature:  0.2,
    }, nil
}

2. Setting Up the File Search Tool

In Gemini, tools are defined within the GenerationConfig or ToolConfig. The File Search tool requires specifying the directory in Google Cloud Storage where your files reside. The files must be in supported formats (.pdf, .docx, .txt, .html).

We need to create a helper to generate the tool configuration. Note that the directory must be accessible to the service account used by the Gemini API.

package service

import (
    "context"
    "google.golang.org/genai"
    "google.golang.org/genai/google"
)

type FileSearchConfig struct {
    GCSBucket string
    Directory string
}

func NewFileSearchTool(ctx context.Context, cfg *FileSearchConfig) (*google.Tool, error) {
    // Construct the file search tool specification
    // Note: The exact struct definitions may vary slightly based on SDK version
    // This is a conceptual representation of the required fields
    tool := &google.Tool{
        FunctionDeclarations: []*google.FunctionDeclaration{},
        // In recent SDKs, file search is handled via specific tool types or parameters
        // We will use the built-in FileSearch capability if available in the SDK version used
    }
    return tool, nil
}

Correction: As of the latest stable SDKs, File Search is often enabled via specific model parameters or by using the SearchRetrieval or FileSearch tool types provided in the google package. Let's use the more standard approach of defining the RagConfig or passing the file directory explicitly if the SDK supports it. For this article, we will assume the use of the genai client where we can pass a FileSearch tool configuration.

In practice, you define the tool like this:

func CreateFileSearchTool(bucket, dir string) *google.Tool {
    return &google.Tool{
        FileSearch: &google.FileSearchToolConfig{
            Directory: &google.FileSearchToolConfigDirectory{
                Bucket: bucket,
                Directory: dir,
            },
        },
    }
}

3. The RAG Service Implementation

The core of our system is the service that takes a prompt and a file search configuration. We must ensure that we are using a model that supports this tool. gemini-1.5-pro and gemini-2.0-flash are currently the primary candidates. gemini-2.0-flash is often preferred for its speed and cost efficiency in high-volume production environments.

package service

import (
    "context"
    "fmt"
    "time"

    "google.golang.org/genai"
    "google.golang.org/genai/google"
)

type GeminiService struct {
    client   *genai.Client
    model    string
    fsConfig *FileSearchConfig
}

func NewGeminiService(client *genai.Client, model string, fsConfig *FileSearchConfig) *GeminiService {
    return &GeminiService{
        client:   client,
        model:    model,
        fsConfig: fsConfig,
    }
}

func (s *GeminiService) Query(ctx context.Context, prompt string) (string, error) {
    // Create the tool instance
    tool := CreateFileSearchTool(s.fsConfig.GCSBucket, s.fsConfig.Directory)

    // Construct the request
    req := &google.GenerateContentRequest{
        Model: s.model,
        Contents: []*google.Content{
            {
                Role: "user",
                Parts: []*google.Part{
                    {
                        Text: prompt,
                    },
                },
            },
        },
        Tools: []*google.Tool{tool},
        // System instruction to enforce grounding
        SystemInstruction: &google.Content{
            Parts: []*google.Part{
                {
                    Text: "You are a helpful assistant. Answer questions based ONLY on the provided file context. If the context does not contain the answer, state that clearly. Do not use external knowledge.",
                },
            },
        },
    }

    ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
    defer cancel()

    // Generate the content
    res, err := s.client.Models.GenerateContent(ctx, req)
    if err != nil {
        return "", fmt.Errorf("failed to generate content: %w", err)
    }

    if len(res.Candidates) == 0 {
        return "", fmt.Errorf("no candidates returned from model")
    }

    if len(res.Candidates[0].Content.Parts) == 0 {
        return "", fmt.Errorf("empty response from model")
    }

    return res.Candidates[0].Content.Parts[0].Text, nil
}

4. Handling File Ingestion

To use File Search, your documents must be in GCS. A production pipeline needs a way to upload files. We can create a separate command or an HTTP endpoint that uses the google.golang.org/api/storage library to upload files to the designated directory.

package ingest

import (
    "context"
    "fmt"
    "io"
    "net/http"

    "google.golang.org/api/option"
    "google.golang.org/api/storage/v1"
)

type Ingestor struct {
    svc *storage.Service
    bucket string
    dir string
}

func NewIngestor(ctx context.Context, projectID string) (*Ingestor, error) {
    client, err := storage.NewService(ctx, option.WithCredentialsFile("/path/to/service-account.json"))
    if err != nil {
        return nil, fmt.Errorf("failed to create storage service: %w", err)
    }
    return &Ingestor{
        svc: client,
    }, nil
}

func (i *Ingestor) UploadFile(file io.Reader, filename string) error {
    // In a real app, you'd set specific content types based on extension
    obj := &storage.Object{
        Name: i.dir + "/" + filename,
    }
    err := i.svc.Objects.Insert(i.bucket, obj).Media(file).Run()
    if err != nil {
        return fmt.Errorf("failed to upload file: %w", err)
    }
    return nil
}

Production Considerations and Best Practices

Caching and Performance

Even with zero extra infrastructure, network calls to the LLM are expensive and slow. You must implement caching. In Go, go-cache or a Redis-based cache is suitable. The cache key should be a hash of the prompt and the specific file directory version.

import (
    "github.com/patrickmn/go-cache"
)

type Cache struct {
    c *cache.Cache
}

func NewCache() *Cache {
    return &Cache{
        c: cache.New(10*time.Minute, 10*time.Minute),
    }
}

func (c *Cache) Get(key string) (string, bool) {
    if item, found := c.c.Get(key); found {
        return item.(string), true
    }
    return "", false
}

func (c *Cache) Set(key, value string) {
    c.c.Set(key, value, cache.DefaultExpiration)
}

Monitoring and Observability

You need to track:

1. Retrieval Accuracy: How often does the model say "I don't know"? This is a key metric for RAG.
2. Latency: P50, P95, P99 of the API calls.
3. Cost: Tokens generated per query.

Integrate with OpenTelemetry. The genai library supports OpenTelemetry middleware, which makes it easy to inject spans into your Go application.

use google.golang.org/api/option
set option.WithTelemetryEnabled() // or similar OTel compatible options

Security and Access Control

Since File Search runs on GCS, you must strictly control access to the GCS bucket. Use IAM policies to restrict who can read the documents. Never expose the GCS bucket publicly. The Gemini API key should be stored in a secrets manager (e.g., GCP Secret Manager) and injected at runtime. In a multi-tenant scenario, you might need separate GCS directories or buckets per tenant, which complicates the FileSearch configuration. In that case, you must dynamically construct the FileSearch tool configuration per request, which requires careful state management in your Go service.

Advanced Techniques

Hybrid Retrieval

While File Search is powerful, it may miss specific metadata. You can combine it with a traditional vector search by implementing a custom tool that calls your vector DB and then passing the results to Gemini as context. However, this defeats the "zero extra infrastructure" goal. A better hybrid approach is to pre-process your files in GCS to include metadata in the file names or front-matter (if using TXT/HTML) that Gemini's indexer can pick up.

Streaming Responses

For better UX, implement streaming. The genai client supports StreamGenerateContent. This allows you to send the response to the user as it is generated.

func (s *GeminiService) StreamQuery(ctx context.Context, prompt string, w io.Writer) error {
    req := /* ... same as Query ... */
    stream, err := s.client.Models.StreamGenerateContent(ctx, req)
    if err != nil {
        return err
    }
    defer stream.Close()

    for {
        res, err := stream.Recv()
        if err == io.EOF {
            break
        }
        if err != nil {
            return err
        }
        if len(res.Candidates) > 0 && len(res.Candidates[0].Content.Parts) > 0 {
            if _, err := w.Write([]byte(res.Candidates[0].Content.Parts[0].Text)); err != nil {
                return err
            }
            w.Flush() // if w is an http.ResponseWriter
        }
    }
    return nil
}

Conclusion

Building a RAG pipeline without a vector database is not just about simplifying infrastructure; it is about leveraging the semantic understanding of LLMs to handle the retrieval layer. By using Go for your application logic and Gemini's File Search for the data layer, you can create a robust, scalable, and maintainable system. The key to success lies in proper data hygiene (cleaning your documents before upload), rigorous caching, and monitoring the "retrieval quality" metrics. As models continue to improve, the boundary between "retrieval" and "reasoning" will blur further, making these server-side semantic tools increasingly viable for production workloads. For a deeper look at similar patterns, check out our previous analysis on stateless RAG architectures and optimizing LLM inference costs.

Frequently Asked Questions

Q: Can I use File Search with structured data like CSVs?
A: Yes, CSVs are supported, but the semantic search performance depends on the column headers and content readability. For large structured datasets, a SQL engine or vector DB with metadata filtering is usually more accurate.

Q: What is the maximum file size for a single document?
A: Currently, the limit is 2GB per file. However, very large files may time out during indexing or search. It is best practice to chunk large PDFs into logical sections if possible.

Q: How does Gemini know which files to search?
A: It searches the entire directory specified in the FileSearch configuration. You cannot dynamically filter by metadata within the tool itself, so organize your GCS directories logically (e.g., /docs/2023, /docs/2024) if you need temporal separation.