DEV Community: Sébastien Descloux

EmbeddingGemma 2: Building Local Multimodal RAG with Python

Sébastien Descloux — Wed, 07 Oct 2026 21:54:23 +0000

What if an AI search application could retrieve information from internal documentation, source code, screenshots, and recordings without sending those files to an external embedding API?

On October 6, 2026, Google DeepMind introduced EmbeddingGemma 2, an open-weight multimodal embedding model released under the Apache 2.0 license. It brings text, code, images, video, and audio into a shared 768-dimensional vector space.

For developers building Retrieval-Augmented Generation (RAG) systems, the most interesting question is architectural: which parts of retrieval can run directly on the user's device?

Why local multimodal retrieval matters

A typical RAG pipeline prepares documents, creates embeddings, retrieves relevant passages, and optionally sends those passages to a generative language model. Depending on its design, the pipeline may transmit content to third-party services.

Local retrieval offers a different trade-off:

Data locality: encoding and searching can take place on the device holding the files.
Offline access: once the model, runtime, and index are available, retrieval can work without a network connection.
Infrastructure control: fewer per-query embedding API calls, but more memory, CPU/GPU, and storage requirements on the device.

This does not make cloud APIs obsolete. For public data or centrally managed services, a remote API can still be the right engineering choice.

EmbeddingGemma 2 in numbers

The model uses modular encoders that share a compatible embedding space:

Configuration	Parameters	Typical content
Text and code	270M	Documentation, codebases, knowledge bases
Text + vision	440M	Images, diagrams, video frames
Text + audio	570M	Recordings and speech
Full multimodal	740M	Text, code, images, video, audio

The shared representation makes it possible to compare a text query with an image or audio segment without requiring a separate caption or transcript in every workflow.

Important: EmbeddingGemma 2 is an embedding model, not a generative LLM. It helps find relevant content. If you want a written answer, you need a separate generation step. For fully on-device RAG, that generative model must also run locally.

Compare three RAG architectures

1. Remote embedding API

A service provider computes the vectors; the application stores and queries an index locally or on a server. This can simplify deployment, but introduces considerations around network access, data handling, and recurring costs.

2. Local text-only RAG

A compact text encoder and retrieval index live on the user's device. This is a practical starting point for notes, internal documentation, and code search. Document parsing, chunking, and access control still matter.

3. Local multimodal RAG

Additional encoders make screenshots, diagrams, sound, and video searchable within a common representation. This can improve retrieval when essential information is not purely textual, at the cost of more processing and memory.

A minimal Python example

Google documents EmbeddingGemma 2 with sentence-transformers 6.1.0 or newer. Install the dependencies in your Python environment:

pip install -U "sentence-transformers[image,audio,video]" transformers

For a text-only prototype, omit the encoders you do not need:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "google/embeddinggemma-2",
    config_kwargs={
        "vision_config": None,
        "audio_config": None,
    },
)

documents = [
    "The application enforces document access permissions.",
    "The meeting notes describe a local retrieval architecture.",
]
question = "Where can I find information about offline search?"

document_vectors = model.encode(
    documents,
    prompt_name="Document",
    normalize_embeddings=True,
)
query_vector = model.encode(
    question,
    prompt_name="SearchQuery",
    normalize_embeddings=True,
)

scores = model.similarity(query_vector, document_vectors)[0]
best_match = int(scores.argmax())
print(documents[best_match])

This returns a retrieved passage, not an AI-generated answer. The first run requires downloading model weights. Subsequent encoding can work offline when all necessary resources have been cached.

A production search system should also store stable document identifiers and source locations, handle updates incrementally, and enforce document-level permissions.

Cross-modal search with images and audio

With the full model, media can be embedded by passing a dictionary keyed by modality:

model = SentenceTransformer("google/embeddinggemma-2")

image_vector = model.encode({"image": "architecture.png"})
audio_vector = model.encode({"audio": "meeting.wav"})
query_vector = model.encode(
    "discussion about application architecture",
    prompt_name="SearchQuery",
)

print("Image:", model.similarity(query_vector, image_vector))
print("Audio:", model.similarity(query_vector, audio_vector))

The sample files must exist and be compatible with the runtime. For longer recordings, index meaningful segments with timestamps: a vector for an entire video cannot reliably locate a particular moment. PDFs with diagrams may require image-based retrieval alongside extracted text and, for scanned pages, OCR.

Memory and vector storage

EmbeddingGemma 2 supports Matryoshka Representation Learning, so embeddings can be shortened from 768 dimensions to 512, 256, or 128. For example, one million raw float32 vectors require approximately:

3.07 GB at 768 dimensions.
1.02 GB at 256 dimensions.

These figures exclude metadata and index overhead. Queries and indexed content need matching dimensions, and smaller vectors should be evaluated for retrieval quality on your actual dataset.

Google has also published memory figures for optimized configurations on specific devices. Those are not universal memory guarantees for a Python application or every browser.

Does on-device retrieval guarantee privacy?

Not automatically. A privacy-oriented deployment also needs to answer:

Do model downloads, analytics, logs, or synchronization transmit sensitive data?
How are files, embeddings, and metadata protected at rest?
Does retrieval enforce user permissions before displaying results?
Is retrieved content treated as untrusted input to prevent prompt injection?
Does the final LLM run on-device or receive selected documents through an API?

Local embeddings reduce certain data transfers; they do not replace a complete security design.

How to evaluate it reproducibly

A useful evaluation can compare remote embeddings, local text retrieval, and local multimodal retrieval using the same authorized corpus, questions, and target hardware.

Prepare representative documents and media, plus 30–50 questions with expected retrieval results.
Measure recall@k, source relevance, indexing time, retrieval latency, peak memory, and storage.
Repeat with 768-dimensional and 256-dimensional vectors to measure the quality/storage trade-off.
Add a generative LLM only if the user experience truly needs written answers, and test cases where the system should abstain.

This is a proposed evaluation method, not a report of independent benchmarks or a description of a deployed product.

Final thoughts

EmbeddingGemma 2 provides an intriguing building block for multimodal search over sensitive, heterogeneous content on local devices. It does not remove the need for good segmentation, source metadata, evaluation, permissions, or a separate generative model.

The key engineering decision is where each stage of the retrieval pipeline should run, given the application's performance, privacy, and usability constraints.

For a more detailed discussion of the architecture, examples, and practical limitations, see the original English article on SDX Development.

Official references

Discussion: Which stage would you move on-device first: embeddings, retrieval, or generation?

EmDash 1.0 vs WordPress: When Should You Consider This Astro CMS?

Sébastien Descloux — Wed, 30 Sep 2026 22:00:00 +0000

Cloudflare recently released EmDash 1.0, an open-source CMS built around Astro and TypeScript.

It comes with an admin interface, structured content, media management, localization, APIs, a CLI, a plugin system, and even an integrated MCP server.

That sounds promising.

But the interesting question is not whether EmDash is a new “WordPress killer”.

The real question is:

For which projects does an Astro-based CMS like EmDash actually make more sense than WordPress?

And if you already have a WordPress website, what does migrating to EmDash really involve?

TL;DR

EmDash 1.0 looks particularly interesting for:

custom Astro websites
TypeScript-heavy projects
structured-content platforms
multilingual sites
projects where developers want full control over the frontend

WordPress still has a major advantage when a project depends on:

a large plugin ecosystem
WooCommerce
page builders
LMS or membership plugins
mature third-party integrations

The biggest mistake would be to treat a WordPress → EmDash migration as a simple content import.

In many cases, it is closer to a technical rebuild.

What is EmDash?

EmDash adds CMS capabilities to an Astro project.

Instead of starting with a traditional CMS and building the frontend inside it, the architecture is closer to:

Astro
  ↓
Your frontend
  ↓
EmDash CMS
  ↓
Structured content / database / media

Astro remains the foundation of the application.

EmDash provides the editorial layer.

That distinction matters.

With WordPress, the CMS is usually both the runtime and the content platform.

With EmDash, your Astro project stays at the center of the architecture.

For teams already working with TypeScript, this can result in a much more consistent stack.

A TypeScript-first CMS architecture

One of the most attractive aspects of EmDash is that developers can stay inside a modern JavaScript/TypeScript ecosystem.

A project can combine:

Astro for routing and rendering
TypeScript for application logic
EmDash for content management
SQL-based storage
local or S3-compatible media storage
APIs and CLI tools for automation

That is very different from maintaining a WordPress installation containing PHP templates, JavaScript, multiple plugins and several layers of custom code.

But “different” does not automatically mean “better”.

It depends heavily on the project.

EmDash vs WordPress by use case

1. A custom marketing website

This is probably one of the most interesting use cases for EmDash.

Imagine a company website with:

a custom design
20–50 pages
a blog
case studies
team profiles
a few forms
multilingual content

With WordPress, this could involve:

a custom theme
Advanced Custom Fields or similar tooling
SEO plugins
form plugins
multilingual plugins
performance plugins

With Astro + EmDash, much of the frontend can simply be implemented as components while EmDash manages editable content.

For a development team already comfortable with Astro, this can be a very clean architecture.

2. A multilingual blog

EmDash also has an interesting approach to localization.

Translations can be managed as separate content variants with their own slugs and publication states.

This is useful when, for example, the English version of an article is published while the French translation is still a draft.

For a new multilingual project, that model is attractive.

Migrating an existing WordPress multilingual site, however, is more complicated.

A WordPress export does not automatically reproduce every relationship created by tools such as WPML or Polylang.

That means multilingual migrations need to be tested carefully.

3. A highly customized application

This is where EmDash becomes especially interesting.

Consider a website containing:

custom content types
business APIs
application-specific forms
structured relationships
automation
custom dashboards
third-party services

At some point, a heavily customized WordPress project starts behaving more like a web application than a traditional CMS.

In that situation, using Astro and TypeScript directly can be attractive.

Instead of forcing application logic into WordPress hooks and plugin conventions, developers can build the application using the same tools they would use for any other TypeScript project.

4. A WordPress site with 20+ plugins

This is almost the opposite scenario.

Suppose a WordPress site relies on:

WooCommerce
Elementor
an SEO suite
advanced forms
memberships
analytics plugins
newsletter integrations
custom payment gateways
several business-specific plugins

Moving that site to EmDash is not simply:

Export WordPress
→
Import EmDash
→
Done

Every important dependency needs to be reviewed.

For each plugin, you have to decide whether the feature will be:

replaced by an EmDash feature
replaced by another service
implemented through an EmDash plugin
rebuilt in TypeScript
removed completely

This is why plugin inventory is one of the most important parts of migration planning.

What actually migrates from WordPress?

EmDash includes tooling for WordPress migration.

Content can be imported using WordPress export data, and EmDash also provides tooling designed to extract additional data from an existing WordPress installation.

Typical migration candidates include:

posts
pages
custom post types
taxonomies
authors
media references
some metadata

But importing content is only one part of the project.

Your WordPress theme does not migrate

A PHP WordPress theme cannot simply be installed inside EmDash.

The frontend must be rebuilt with Astro.

For example, a WordPress template such as:

single.php

might become an Astro route similar to:

src/pages/posts/[slug].astro

The visual design can obviously be recreated.

The implementation is what changes.

In many projects, this is actually an opportunity.

A migration can be used to remove years of accumulated technical debt instead of reproducing an old WordPress architecture exactly.

WordPress plugins do not magically become EmDash plugins

This is another critical point.

Existing WordPress plugins are not directly compatible with EmDash.

Their functionality needs to be analyzed and, where necessary, ported or rebuilt.

For a custom WordPress plugin, developers may need to review things such as:

hooks
REST endpoints
custom database tables
cron jobs
admin screens
shortcodes
blocks
external integrations

Some plugins may no longer be required at all.

A caching plugin designed around a traditional PHP WordPress runtime, for example, might have no equivalent purpose in a completely different architecture.

The useful question is therefore not:

How do we migrate every plugin?

It is:

What functionality does each plugin provide, and do we still need it?

What about plugins in EmDash?

EmDash 1.0 also introduces its own plugin architecture.

One interesting part of the model is the ability to run some plugins inside isolated environments with explicit permissions.

That creates a different security model from the traditional WordPress approach where plugins frequently execute inside the same PHP runtime as the application.

EmDash also supports native plugins for features that require deeper integration.

The architecture is promising.

The ecosystem, however, is still extremely young compared with WordPress.

And that matters.

Ecosystem maturity is still WordPress' biggest advantage

WordPress has been around for more than two decades.

Its ecosystem includes:

tens of thousands of plugins
thousands of themes
hosting providers
agencies
freelancers
integrations
documentation
tutorials
existing solutions for almost every common website requirement

EmDash 1.0 cannot reproduce that overnight.

So if your project depends heavily on ready-made functionality, WordPress still offers something extremely valuable:

maturity.

On the other hand, if you were going to build most of the functionality yourself anyway, the difference becomes much smaller.

A WordPress → EmDash migration is often a rebuild

For a professional project, I would approach the migration in roughly this order.

1. Audit the existing WordPress site

Identify:

content types
taxonomies
plugins
custom fields
shortcodes
theme customizations
forms
APIs
redirects
SEO metadata
multilingual content
scheduled jobs
business integrations

2. Classify every dependency

Decide whether each feature should be:

imported
replaced
rebuilt
connected to an external service
removed

3. Import a small content sample

Don't migrate thousands of posts immediately.

Start with representative examples.

4. Rebuild the frontend in Astro

Create routes, layouts and components based

I Built a RAG AI Assistant That Runs in the Browser with WebGPU

Sébastien Descloux — Sun, 27 Sep 2026 19:28:01 +0000

A website may already contain the answers visitors need.

The real problem is often finding them.

So I built and deployed a first version of a RAG AI assistant for my own website, with one particular constraint:

The response model runs directly in the visitor's browser using WebGPU.

This was not meant to be another chatbot connected to an API.

I wanted to explore what happens when retrieval, local inference, source-based answers, conversation context, and a CMS are designed as one complete workflow.

How the architecture works

The system splits the work between the website server and the browser.

1. The CMS prepares the knowledge base

Selected website content is split into passages and indexed.

The CMS also lets me manage:

which documents are available to the assistant
model profiles
search parameters
assistant instructions
activation and validation

2. The browser understands the question

The browser can use recent conversation context to reformulate follow-up questions.

An embedding model then converts the query into a vector.

3. The server performs vector search

The vector is sent to the website API.

The server searches the document index and returns the most relevant passages together with their sources.

4. WebGPU generates the answer locally

The response model runs on the visitor's GPU.

It receives:

the question
recent conversation context
retrieved passages
CMS instructions

The first result is not displayed immediately.

A second pass checks the draft against the retrieved sources, and the system can perform one corrective search if necessary.

5. The visitor gets an answer with sources

The final response contains links back to the original website content.

If the available information is not sufficient, the assistant is expected to say so instead of inventing an answer.

Why run the model in the browser?

The main experiment was to avoid calling a hosted generation API for every message.

This changes several things.

No per-response generation API call

Once the models are loaded, generation happens on the visitor's device.

That does not make the system free — hosting, downloads, development and maintenance still have costs — but there is no token bill for every generated response in this workflow.

More control over conversation data

The conversation is not sent to an external generation API to produce the answer.

Optional conversation memory is also stored locally in the browser.

The server is still involved in document retrieval, so this is not a fully offline system.

Local memory

I also added optional semantic memory using IndexedDB.

It can keep previous exchanges locally for a limited period and retrieve an older, related question when useful.

An important design decision was to keep three concepts separate:

documents are the factual source
recent conversation maintains context
local memory helps recover previous intent

An old AI-generated answer never becomes a new factual source.

The limitations are just as important

Running AI models in the browser comes with real constraints.

The current response model requires roughly 2 GB of downloads, plus about 279 MB for the embedding model.

Browser support is another factor.

Having navigator.gpu available does not automatically mean that every model and inference engine will work correctly on that device.

Drivers, GPU capabilities, memory limits and browser implementation all matter.

Latency is also very different from a fast cloud API.

With the models already cached, one documented test produced a reviewed answer in:

36.11 seconds on a Samsung Galaxy S25 Ultra
13.06 seconds on a desktop with an NVIDIA Ampere GPU

These are measurements from my own test devices, not performance guarantees.

More complex mobile requests can take over a minute.

Why I still find this architecture interesting

The goal is not to replace every search box with an AI assistant.

The interesting use cases are situations where users have a real question and the website already has reliable information that can answer it.

For example:

product catalogs
technical documentation
training websites
tourism or accommodation
business software
real-estate search

A user can express a need naturally, while structured data and RAG keep the response connected to information that can actually be verified.

The model is only one part of the solution.

The real value comes from combining:

data + retrieval + business rules + UX + performance + validation

What I learned from this first version

Building this project reinforced something I increasingly see in applied AI work:

A better model does not automatically create a better product.

The difficult parts are often around the model:

preparing reliable knowledge
keeping retrieval consistent
managing context
handling model downloads and caching
keeping the interface responsive
validating answers
testing on real devices
deciding when the system should simply say "I don't know"

This first version is now running in production on my own website, and I am continuing to test lighter models, additional browsers, GPUs and possible business applications.

I wrote a more complete article with the architecture, limitations, benchmarks and potential use cases:

👉 RAG and WebGPU: When Your Website Becomes an AI Assistant

There is also a deeper technical case study covering the complete RAG pipeline, CMS integration, embeddings, local memory and model choices.