I Built a Multimodal RAG System with Gemini File Search — Here’s What I Learned

RAG has become one of the most practical ways to build AI applications that need to work with private or domain-specific information.

Instead of expecting an LLM to already know everything, we can give it access to our own documents and retrieve the relevant information when a user asks a question.

That sounds simple until the data stops being simple.

Real-world knowledge bases rarely contain only plain text. They can contain PDFs, screenshots, product images, diagrams, tables, architecture documents, technical manuals, and other visual information.

That was the problem I wanted to solve.

I wanted a RAG system where text and images could live in the same searchable knowledge base instead of maintaining separate pipelines for documents and visual data.

This is where Gemini API File Search becomes interesting.

Google's Gemini API provides File Search as a managed Retrieval-Augmented Generation system. It handles the ingestion, chunking, embedding, indexing, and retrieval pipeline, while the model uses the retrieved information to generate a grounded response. The newer multimodal capabilities extend this workflow to images as well.

In this article, I'll walk through how I think about the architecture, why multimodal embeddings matter, how the implementation works, and where this approach makes sense compared with building a traditional RAG stack yourself.

What Problem Does RAG Actually Solve?

A normal LLM has a knowledge boundary.

Even if the model is extremely capable, it doesn't automatically have access to your company's internal documentation, private PDFs, product catalog, support tickets, or architecture diagrams.

You could fine-tune a model, but that is often the wrong solution.

If the information changes frequently, you don't want to retrain a model every time a document changes.

RAG solves this differently.

Instead of modifying the model's knowledge, you retrieve relevant information from an external knowledge source and provide that information as context to the model.

The basic workflow looks like this:

User Question
     ↓
Convert Question to Search Representation
     ↓
Search Knowledge Base
     ↓
Retrieve Relevant Chunks
     ↓
Send Retrieved Context to LLM
     ↓
Generate Grounded Answer

This is powerful because the knowledge can be updated without retraining the model.

But traditional RAG becomes more complicated when your knowledge isn't purely text.

Consider a technical PDF containing:

• Written documentation
• Architecture diagrams
• Screenshots
• Tables
• Product images
• Flow charts

If your retrieval system only understands extracted text, you may lose valuable information contained inside those visual elements.

That's where multimodal retrieval changes the architecture.

Why Multimodal RAG Is Different

Traditional text RAG generally converts text into embeddings.

Those embeddings represent the semantic meaning of the text.

A user query is also converted into an embedding, and the system searches for chunks that are semantically similar.

Multimodal RAG extends this idea beyond text.

With Gemini's gemini-embedding-2, images can also be embedded and searched alongside textual content. This means an application can retrieve visual information rather than relying entirely on OCR or manually generated image descriptions.

For example, imagine a product catalog containing:

Product description
+
Product specifications
+
Product image

A user could ask:

"Show me the product that has a red design."

A multimodal retrieval system can search the visual representation of the product image along with the associated textual information.

That opens up use cases that are difficult to implement cleanly with text-only RAG.

The Architecture I Would Use

The architecture is surprisingly small when using managed File Search.

                 ┌──────────────────────┐
                 │   Documents / Images  │
                 └──────────┬───────────┘
                            ↓
                 ┌──────────────────────┐
                 │   File Search Store  │
                 │                      │
                 │ Chunking              │
                 │ Embeddings            │
                 │ Indexing              │
                 └──────────┬───────────┘
                            ↓
User Question ─────────────→ Semantic Search
                            ↓
                    Relevant Context
                            ↓
                     Gemini Model
                            ↓
                    Grounded Response
                            ↓
                         Citations

The important part is that I don't have to manually operate a vector database, embedding service, document chunking pipeline, and retrieval layer.

File Search manages those parts for me.

Google describes a File Search Store as a persistent container for processed document data and embeddings. When files are imported, the content is chunked, converted into embeddings, and indexed for semantic retrieval.

That removes a significant amount of infrastructure from the application.

Creating a Multimodal File Search Store

The first important decision is the embedding model.

If you're building a text-only knowledge base, the text embedding model may be sufficient.

But if you want native image retrieval, the File Search Store needs to be created using:

models/gemini-embedding-2

For example:

from google import genai

client = genai.Client()

store = client.file_search_stores.create(
    config={
        "display_name": "my-multimodal-knowledge-base",
        "embedding_model": "models/gemini-embedding-2"
    }
)

print(store.name)

This decision matters because the embedding model is associated with the store configuration.

So don't blindly create a store with the default embedding model and later discover that your application needs multimodal retrieval.

Google's current documentation specifically recommends gemini-embedding-2 for processing both text and images in a multimodal File Search Store.

Uploading Documents

Once the store exists, documents can be uploaded directly.

For example:

import time

operation = client.file_search_stores.upload_to_file_search_store(
    file_search_store_name=store.name,
    file="technical-documentation.pdf",
    config={
        "display_name": "Technical Documentation"
    }
)

while not operation.done:
    time.sleep(5)
    operation = client.operations.get(operation)

print("Document indexed")

Behind the scenes, the service processes the document and prepares it for retrieval.

The important thing for developers is that we don't need to manually implement every stage of the ingestion pipeline.

Instead of writing:

PDF parser
      ↓
Text extraction
      ↓
Chunking service
      ↓
Embedding API
      ↓
Vector database
      ↓
Metadata layer
      ↓
Retriever

we can work with the managed File Search abstraction.

That can significantly reduce development and operational overhead.

Adding Images

This becomes more interesting with multimodal stores.

Images can also be uploaded:

images = [
    "product-red.png",
    "product-blue.png",
    "architecture-diagram.png"
]

for image_file in images:

    operation = client.file_search_stores.upload_to_file_search_store(
        file_search_store_name=store.name,
        file=image_file,
        config={
            "display_name": image_file
        }
    )

    while not operation.done:
        time.sleep(5)
        operation = client.operations.get(operation)

Google's current documentation states that PNG and JPEG images are supported for multimodal File Search, with an image resolution limit of 4K × 4K pixels. Audio and video are currently not supported by File Search.

That last limitation is important.

Multimodal does not mean "every media format."

You should design your ingestion pipeline around the formats File Search actually supports.

Querying the Knowledge Base

After indexing the data, the actual application logic becomes simple.

The model can use File Search as a tool:

response = client.models.generate_content(
    model="gemini-3-flash-preview",
    contents="Which product is available in red?",
    config={
        "tools": [{
            "file_search": {
                "file_search_store_names": [store.name]
            }
        }]
    }
)

print(response.text)

The query is converted into a representation that can be matched against the indexed content.

The retrieval system then finds relevant chunks from the File Search Store.

Those results are supplied as context for the model.

The model generates an answer based on the retrieved information rather than relying exclusively on its general knowledge.

That distinction is extremely important.

RAG doesn't magically make an LLM truthful.

It gives the model access to better evidence.

Semantic Search Instead of Keyword Search

One reason this architecture is useful is semantic search.

Consider a document containing:

"Our infrastructure automatically increases compute capacity when traffic exceeds predefined thresholds."

A user might ask:

"How does the system handle traffic spikes?"

A basic keyword search might struggle because "traffic spikes" doesn't exactly match the original wording.

Semantic retrieval is designed to understand the relationship between the query and the underlying meaning.

Gemini File Search uses embeddings to represent the semantic meaning of indexed content and retrieve relevant chunks based on the user's prompt.

This is one of the fundamental reasons RAG systems are more useful than simple text search.

Metadata Makes Retrieval More Useful

Another feature I would use in a real application is metadata.

Suppose the same knowledge base contains documents from different categories:

product
support
engineering
billing
legal

Instead of searching everything every time, metadata can help narrow the retrieval scope.

For example:

config={
    "display_name": "Spring Product Catalog",
    "custom_metadata": [
        {
            "key": "category",
            "string_value": "product"
        },
        {
            "key": "season",
            "string_value": "spring-2026"
        }
    ]
}

Then the query can use a metadata filter.

This matters because retrieval quality isn't only about the embedding model.

The quality of your data organization also matters.

A huge unstructured knowledge base can produce weaker retrieval than a well-organized one.

Citations Are More Important Than They Look

One of the biggest problems with AI-generated answers is verification.

If the model says:

"This product supports feature X."

I want to know where that information came from.

File Search provides grounding information that can identify the source material used to generate the response. Depending on the response and API surface, this can include document/page information or other citation metadata.

This changes the UX of a RAG application.

Instead of:

AI says something.

you can build:

AI says something.

Source:
Technical Documentation
Page 14

For enterprise applications, this can be far more valuable than simply generating a fluent response.

A good RAG system should make it easier to verify the answer.

RAG Does Not Automatically Fix Hallucinations

This is where many RAG tutorials oversell the technology.

Adding a vector database does not guarantee correct answers.

There are several failure points:

Bad document
    ↓
Bad chunking
    ↓
Bad embedding
    ↓
Bad retrieval
    ↓
Wrong context
    ↓
Wrong answer

The LLM is only the final stage.

If retrieval returns irrelevant information, the model can still generate a confident-looking answer.

That's why I would test a RAG system with an evaluation dataset instead of judging it from five successful demos.

Create questions where you already know:

• The correct answer
• The source document
• The expected section/page
• Whether the answer should exist at all

Then measure retrieval and answer quality.

Chunking Still Matters

Managed retrieval doesn't mean chunking becomes irrelevant.

A document can be split into chunks before embedding.

If chunks are too large, retrieval can become less precise.

If chunks are too small, important context can be separated.

Google's File Search API also supports custom chunking configuration. For example, the documentation demonstrates configuring a whitespace-based strategy with a maximum number of tokens per chunk and overlap between chunks.

For example:

"chunking_config": {
    "white_space_config": {
        "max_tokens_per_chunk": 200,
        "max_overlap_tokens": 20
    }
}

I wouldn't copy these numbers blindly into production.

Chunking should be tested against your actual documents.

Technical manuals, legal documents, product catalogs, and source-code documentation don't necessarily benefit from the same chunking strategy.

Where Multimodal RAG Becomes Really Useful

This architecture is particularly interesting for applications where visual information carries meaning.

Technical Documentation

Imagine a developer asking:

"Find the architecture diagram that shows the authentication flow."

A text-only system may retrieve a paragraph describing authentication.

A multimodal system can also work with the actual diagram.

Product Catalogs

A product database can combine:

Product name
Description
Specifications
Price information
Product images

Users can search using both natural language and visual characteristics.

Real Estate

A property knowledge base can contain:

• Property descriptions
• Floor plans
• Interior images
• Specifications

The retrieval layer can work with more than just text.

Design Systems

Imagine indexing UI screenshots, component documentation, and design specifications.

A developer could search for a component based on its visual appearance or associated documentation.

These are the situations where multimodal retrieval becomes more than a marketing feature.

What About Cost?

Cost is another reason managed File Search is interesting.

According to Google's current documentation, storage and query-time embedding generation are free. You pay for embedding creation when files are initially indexed, along with normal Gemini model input/output token usage. Retrieved document tokens are charged as regular context tokens.

That doesn't mean the system is free.

Large knowledge bases still create indexing and model-token costs.

You should monitor:

Number of indexed files
+
Indexing frequency
+
Retrieved context size
+
Model requests
+
Response tokens

A poorly designed application can retrieve too much context and increase token usage unnecessarily.

The goal isn't to retrieve everything.

The goal is to retrieve the smallest useful amount of evidence needed to answer the question.

Important Limits

Production systems also need to respect service limits.

Google's current File Search documentation lists a maximum individual document size of 100 MB. Project-level File Search storage limits vary by usage tier, and Google recommends keeping individual stores under 20 GB for better retrieval latency.

There are also feature limitations.

For example, File Search isn't currently supported in the Live API, and built-in grounding tools have compatibility restrictions when combined in the same request.

These details are easy to ignore in a demo and painful to discover in production.

When I Would Use File Search Instead of Building My Own RAG Stack

I wouldn't say managed File Search replaces every vector database.

It doesn't.

If I need highly customized retrieval logic, specialized ranking, complex hybrid search, or complete infrastructure control, I may still build a custom RAG architecture.

But if my goal is:

Upload documents
        ↓
Index them
        ↓
Search them semantically
        ↓
Ask Gemini questions
        ↓
Return grounded answers

then a managed solution can remove a lot of unnecessary infrastructure.

I don't have to maintain:

• My own vector database
• Embedding workers
• Chunking pipelines
• Retrieval services
• Index synchronization
• Separate image retrieval infrastructure

That can mean fewer moving parts and less code to maintain.

The Architecture I Would Build for a Real Application

For a production application, I'd put an application layer in front of File Search.

                User
                 │
                 ▼
          Web / Mobile App
                 │
                 ▼
             API Server
                 │
        ┌────────┴─────────┐
        │                  │
        ▼                  ▼
 Authentication       Query Validation
                           │
                           ▼
                    Gemini File Search
                           │
                           ▼
                    Retrieved Context
                           │
                           ▼
                     Gemini Model
                           │
                           ▼
                Answer + Citations
                           │
                           ▼
                       User

The API server should control access.

Users shouldn't automatically get access to every document in every store.

I'd separate stores or use metadata according to the application's authorization model.

For example:

Company A
 ├── Engineering
 ├── Support
 └── Product

Company B
 ├── Engineering
 ├── Support
 └── Product

This becomes especially important when the RAG system contains private business information.

My Biggest Takeaway

The most interesting part of modern RAG isn't simply that an LLM can "chat with PDFs."

That's old news.

The bigger shift is that retrieval is becoming multimodal and increasingly managed.

Documents aren't just text.

Knowledge can exist inside:

• Diagrams
• Screenshots
• Images
• Tables
• Product photos
• Technical documentation

A retrieval system that ignores those formats is potentially throwing away useful information.

Gemini File Search's multimodal support makes it possible to build a single retrieval workflow around both text and images, while managed indexing, semantic search, metadata, and citations reduce the amount of infrastructure developers need to operate themselves.

But I wouldn't treat it as a magic "add RAG and everything works" button.

The real engineering work is still in:

Data quality
     ↓
Document organization
     ↓
Chunking
     ↓
Metadata
     ↓
Retrieval evaluation
     ↓
Prompt design
     ↓
Access control
     ↓
Answer verification

The technology can handle much of the infrastructure.

It cannot decide what good data looks like for your application.

Final Thoughts

If you're building a knowledge-based AI application today, I think the right question isn't:

"Which vector database should I use?"

The better first question is:

"How much retrieval infrastructure do I actually need to own?"

For many applications, a managed File Search system can get you from a folder full of documents to a grounded AI assistant much faster.

And when the data includes images and diagrams, multimodal retrieval makes the architecture considerably more useful.

The best RAG system isn't the one with the most components.

It's the one that retrieves the right evidence, gives the model enough context to reason over it, and makes the final answer easy to verify.

That's the part that actually matters in production.