What is Gemini Embedding 2?

Gemini Embedding 2: A Practical Guide to Multimodal Embeddings

Google’s Gemini Embedding 2 handles text, images, video, audio, and documents in a single embedding space, making it easier to build multimodal AI applications. Released in March 2026, this is Google’s first embedding model that natively processes multiple content types without separate pipelines.

Try Apidog today

If you’re building semantic search, RAG systems, or testing APIs that work with different media types, this model simplifies your architecture and improves accuracy.

What Makes Gemini Embedding 2 Different?

Most embedding models specialize in one type of content. Text embedding models process text, while image embedding models process images. Supporting multiple formats usually requires separate models and additional logic to combine their results.

Gemini Embedding 2 maps multiple content types into one embedding space:

• Text: up to 8,192 tokens
• Images: up to 6 per request
• Video: up to 128 seconds
• Audio: up to 80 seconds
• PDF documents: up to 6 pages

This lets you search across different media types with a single query. A text question can return relevant videos, images, PDFs, or audio because all of them are represented in the same vector space.

How Multimodal Embeddings Work

A typical multimodal retrieval pipeline looks like this:

1. Collect content in different formats.
2. Generate an embedding for each item.
3. Store the vectors and metadata in a vector database.
4. Embed the user’s query.
5. Run a similarity search.
6. Return results from any supported media type.

For example, a product knowledge base might store:

Product description → text embedding
User manual page    → PDF embedding
Demo recording      → video embedding
Support call        → audio embedding
Product photo       → image embedding

The query does not need to match the content format. A text query such as how do I replace the filter? can retrieve a PDF page, a video demonstration, or an audio explanation.

Key Features

1. Interleaved Multimodal Input

Gemini Embedding 2 supports multiple content types in a single request. You can send combinations such as:

• Text plus an image
• Video plus audio
• A product description plus product photos
• A question plus a document page

This is useful when the meaning depends on the relationship between media types. For example, a product’s description, image, and demo video can be embedded together rather than processed as unrelated records.

When indexing mixed content, keep the source metadata with the resulting vector:

{
  "id": "product-123",
  "content_type": "product",
  "source_ids": [
    "description-123",
    "image-123",
    "demo-video-123"
  ],
  "embedding_dimensions": 768
}

The exact metadata schema depends on your vector database, but retaining the original source references makes retrieval and debugging easier.

2. Matryoshka Representation Learning

Gemini Embedding 2 produces 3,072-dimensional embeddings by default, but the vectors can be truncated to smaller sizes.

The available dimensions range from 128 to 3,072. Smaller vectors require less storage and can reduce the cost of similarity searches. The model is designed so that truncated vectors retain much of the quality of the full representation.

A practical starting point is:

• 3,072 dimensions: highest quality and largest storage requirement
• 1,536 dimensions: balanced quality and storage
• 768 dimensions: near-peak quality with 75% less storage

For many production systems, 768 dimensions are a reasonable starting point. Validate this with your own retrieval benchmark before standardizing on it.

Do not mix dimensions in the same vector index. If you change the configured dimensionality, create a compatible index or re-embed the affected data.

3. Custom Task Instructions

Task instructions tell the model how an embedding will be used. Supported task types include:

• RETRIEVAL_QUERY: user queries
• RETRIEVAL_DOCUMENT: documents being indexed
• SEMANTIC_SIMILARITY: comparing content
• CLASSIFICATION: categorizing content

For retrieval, use the query task for user input and the document task for indexed content:

User input:
  task = RETRIEVAL_QUERY

Indexed content:
  task = RETRIEVAL_DOCUMENT

Using the appropriate task type helps align the embeddings with the intended operation.

4. Native Audio Processing

Gemini Embedding 2 processes audio directly instead of requiring transcription first. This preserves audio information such as tone, emotion, and context that may not survive a text-only transcription pipeline.

This can be useful for searching:

• Customer support calls
• Audio instructions
• Interviews
• Voice notes
• Video soundtracks

Technical Specifications

Text

• Up to 8,192 tokens per request
• 100+ languages supported
• Supports code and long documents

Images

• Up to 6 images per request
• PNG and JPEG formats

Video

• Up to 128 seconds per request
• MP4 and MOV formats
• H264, H265, AV1, and VP9 codecs

Audio

• Up to 80 seconds per request
• MP3 and WAV formats
• No transcription required

PDF Documents

• Up to 6 pages per request
• Processes text and visual content
• Includes built-in OCR

Validate these limits against the current public preview documentation before deploying, because preview APIs may change.

Building a Multimodal Search Pipeline

A practical implementation separates ingestion from querying.

Ingestion flow

1. Load a source item
2. Detect its media type
3. Split it if it exceeds the model limit
4. Generate an embedding
5. Store the vector and metadata

Example metadata:

{
  "id": "manual-page-42",
  "source": "product-manual.pdf",
  "page": 42,
  "content_type": "pdf",
  "language": "en",
  "embedding_model": "gemini-embedding-2-preview",
  "dimensions": 768
}

Query flow

1. Receive the user’s text query
2. Generate a RETRIEVAL_QUERY embedding
3. Search the vector database
4. Filter or rerank results using metadata
5. Return the original media references

Store the original content location rather than only the vector. A vector is useful for retrieval, but the application still needs the source document, image, video, or audio file to display or pass to a downstream model.

Real-World Use Cases

Semantic Search Across Media Types

Build a search engine that returns relevant content regardless of format.

A user searches for:

How do I fix a leaky faucet?

The result set might include:

• A plumbing tutorial video
• A step-by-step article
• A diagram image
• An audio instruction
• A PDF repair guide

All results can be ranked using the same similarity-search workflow.

RAG Systems with Multimodal Context

A retrieval-augmented generation system can gather context from multiple sources, such as:

• Product descriptions
• User manual pages
• Demo videos
• Customer review audio

The embedding stage finds relevant material across formats. Your application can then provide the retrieved content to the generation model in the format it supports.

Keep chunk sizes and metadata consistent during ingestion. For PDFs, store page numbers. For videos, store timestamps. For audio, store segment boundaries. This makes citations and result previews easier to implement.

API Testing with Semantic Similarity

In Apidog, Gemini embeddings can help test API responses semantically. Instead of comparing response strings exactly, compare the response embedding with an embedding generated from the expected meaning.

This is useful when:

• Wording can change while the meaning remains the same
• An endpoint returns natural-language responses
• You are testing an LLM-powered API
• Responses contain equivalent but differently formatted content

A semantic assertion can follow this pattern:

1. Send a request to the API
2. Extract the relevant response field
3. Generate an embedding for the actual response
4. Generate an embedding for the expected response
5. Calculate similarity
6. Fail the test if the score is below your threshold

Choose the threshold from representative passing and failing examples instead of selecting one arbitrarily.

You can also add semantic search to API documentation. Developers could search for:

Create a customer and return its identifier

instead of needing to know the exact endpoint name or parameter names.

Content Clustering and Organization

Use embeddings to group related content across formats. Product photos, descriptions, and videos can cluster around the same product category or topic.

A typical workflow is:

1. Generate embeddings for all assets.
2. Store the vectors with category and source metadata.
3. Run a clustering algorithm in your data-processing layer.
4. Review or label the resulting groups.
5. Use the groups for navigation, recommendations, or content cleanup.

Sentiment Analysis Across Channels

You can create a unified feedback search across:

• Text reviews
• Video testimonials
• Audio support calls
• Social media images

Embeddings help retrieve and group semantically related feedback. For precise sentiment labels, use a classification step in addition to similarity search.

Performance and Benchmarks

Google claims Gemini Embedding 2 outperforms leading models in text, image, and video tasks. It also introduces speech capabilities that were not available in previous embedding models.

The model is designed to represent relationships between different content types more effectively than single-modality models. Benchmark the model with your own data, especially if your application depends on a specific language, media format, or retrieval threshold.

Useful evaluation metrics include:

• Recall@k
• Precision@k
• Mean reciprocal rank
• Retrieval latency
• Storage per vector
• Cost per indexed item

Pricing

Text embeddings cost $0.20 per million tokens. If you do not need real-time responses, the batch API offers 50% off.

Image, audio, and video use standard Gemini API media token rates.

Your total cost depends on:

• The number of items indexed
• Input length and media duration
• The number of re-embedding operations
• The selected vector dimensions
• Whether you use real-time or batch processing

For most applications, the cost is reasonable. A RAG system processing thousands of documents might cost a few dollars to embed the corpus, depending on the document size and media content.

Gemini Embedding 2 vs. Other Embedding Models

Feature	Gemini Embedding 2	OpenAI text-embedding-3	Cohere Embed v3
Modalities	Text, image, video, audio, PDF	Text only	Text only
Max input	8,192 tokens for text	8,191 tokens	512 tokens
Dimensions	128–3,072	256–3,072	1,024
Languages	100+	100+	100+
Task instructions	Yes	No	Yes
Pricing	$0.20 per million tokens	$0.13 per million tokens	$0.10 per million tokens
Best for	Multimodal applications	Text-only applications	Text classification

The primary differentiator is multimodal support. If you only need text embeddings, OpenAI or Cohere may be cheaper. If you need images, video, or audio in the same embedding space, Gemini Embedding 2 provides that capability through one model.

Integration and Availability

Gemini Embedding 2 is available in public preview as gemini-embedding-2-preview through:

• Gemini API
• Vertex AI
• LangChain
• LlamaIndex
• Haystack
• Weaviate
• Qdrant
• ChromaDB
• Vector Search

The public preview status means the API may change before general availability. Isolate the embedding provider behind an application interface so you can update request formats or replace the model without changing your retrieval logic.

For example:

EmbeddingService.embedQuery(query)
EmbeddingService.embedDocument(document)
EmbeddingService.embedMedia(media)

This also gives you one place to manage:

• Model names
• Task instructions
• Output dimensions
• Retry behavior
• Rate limits
• Logging
• Re-embedding jobs

Important Migration Note

If you are using the older gemini-embedding-001 model, its embedding space is incompatible with Gemini Embedding 2.

Do not mix vectors from the two models in the same vector index. Upgrading requires re-embedding your entire dataset because there is no migration path that preserves existing vectors.

A safe migration plan is:

1. Create a new vector index.
2. Record the new model and dimension in your metadata.
3. Re-embed the existing dataset.
4. Run retrieval benchmarks against the old index.
5. Switch reads to the new index.
6. Remove the old index after validation.

Choosing an Output Dimension

The model supports dimensions from 128 to 3,072:

• 3,072: highest quality and largest storage requirement
• 1,536: balanced quality and size
• 768: near-peak quality with 75% less storage

Start with 1,536 or 3,072 dimensions during evaluation if storage allows. Compare those results with 768 dimensions using your real queries. If the retrieval quality is similar, the smaller representation can reduce storage and search costs.

Keep the selected dimension consistent across:

• Query embeddings
• Document embeddings
• Vector index configuration
• Similarity calculations
• Re-embedding jobs

When to Use Gemini Embedding 2

Use Gemini Embedding 2 when:

• Your data includes text, images, video, audio, or PDFs
• You need semantic search across different content types
• You are building a RAG system with diverse sources
• You want to cluster or classify mixed-media content
• You need embeddings that represent relationships between modalities

A text-only model may be a better fit when:

• You only work with text
• You need the highest possible text-only performance
• You already have embeddings that cannot be regenerated
• Multimodal retrieval is not part of your product requirements

What This Means for Developers

Gemini Embedding 2 reduces the number of pipelines required for multimodal applications. Instead of selecting a separate embedding model for each media type and combining their results, you can use one model and one vector space.

That simplifies:

• Ingestion code
• Vector storage
• Query logic
• Cross-media retrieval
• Metadata management

The Matryoshka representation also gives you a practical optimization path. Start with 3,072 dimensions during development, then evaluate 1,536 or 768 dimensions for production.

Task instructions provide another optimization point. You can use different instructions for queries, documents, similarity comparisons, and classification without training a custom model.

Getting Started

To use Gemini Embedding 2:

1. Get a Gemini API key from Google AI Studio.
2. Install the Google Generative AI SDK.
3. Call the embedding endpoint with your content.
4. Specify the appropriate task type.
5. Select an output dimension.
6. Store the vectors and source metadata.
7. Use the vectors for search, RAG, clustering, or classification.

The request flow is conceptually:

Embedding request
├── model: gemini-embedding-2-preview
├── content: text, image, video, audio, or PDF
├── task type: retrieval, similarity, or classification
└── output dimensions: 128–3,072

For production code, verify the exact request and response field names in the current Gemini API or Vertex AI documentation because this model is in public preview.

A basic implementation should also include:

• Input validation for media limits
• Chunking for long text and documents
• Source metadata for every vector
• Retry handling for transient API errors
• A re-embedding job for model or dimension changes
• Retrieval evaluation using real application queries

The Bottom Line

Gemini Embedding 2 is Google’s approach to multimodal embeddings. It handles text, images, video, audio, and documents in one unified embedding space.

The Matryoshka representation provides flexibility in choosing vector dimensions. Custom task instructions help adapt embeddings to specific operations. Native audio processing preserves information that can be lost in a transcription-only workflow.

If you are building semantic search, multimodal RAG, content organization, or API tests based on semantic similarity, this model is worth evaluating. The public preview is available through the Gemini API and Vertex AI.

For API testing with Apidog, embeddings can help validate whether responses are semantically equivalent even when their wording changes. That is especially useful for LLM-powered endpoints and natural-language API responses.