DEV Community: Emanuele Strazzullo

Building a Browser-Based RAG System with WebGPU

Emanuele Strazzullo — Fri, 07 Nov 2025 13:20:25 +0000

I built a proof-of-concept that lets you chat with PDF documents using AI models running entirely in your browser via WebGPU. No backend, no API keys, complete privacy.

🔗 Demo: https://webpizza-ai-poc.vercel.app/
📦 Code: https://github.com/stramanu/webpizza-ai-poc

Why?

I've been following the progress of WebGPU and WebLLM, and I was curious: Can we run a full RAG pipeline in the browser?

RAG (Retrieval-Augmented Generation) typically requires:

A vector database
An embedding model
A language model
Orchestration logic

Turns out, modern browsers can handle all of this!

Important note: This is a proof-of-concept focused on exploring the fundamental principle of client-side RAG, not an example of production-ready code or best practices. The goal was experimentation with WebGPU and LLMs in the browser, so expect rough edges and architectural shortcuts.

The Stack

Frontend: Angular 20 (standalone components, zoneless)
LLM: WebLLM v0.2.79 + WeInfer (optimized fork)
Embeddings: Transformers.js (all-MiniLM-L6-v2)
Vector Store: IndexedDB with cosine similarity
PDF Parser: PDF.js
Deployment: Vercel

How It Works

1. Model Loading

WebLLM downloads pre-compiled MLC models (Phi-3, Llama, Mistral). First load is slow (1-4GB), but they're cached in the browser.

await this.llm.initialize();

2. Document Ingestion

Upload a PDF → Parse with PDF.js → Chunk into ~500 char pieces → Embed each chunk → Store in IndexedDB.

const chunks = await this.parser.parseFile(file);
for (const text of chunks) {
  const embedding = await this.embedder.embed(text);
  await this.vectorStore.addChunk({ text, embedding });
}

3. Query Processing

User asks question → Embed query → Similarity search in IndexedDB → Get top-k chunks → Feed to LLM with context.

const queryEmbedding = await this.embedder.embed(question);
const relevantChunks = await this.vectorStore.search(queryEmbedding, k=3);
const context = relevantChunks.map(c => c.text).join('\\n');
const response = await this.llm.generate(context, question);

Challenges

1. Cross-Origin Isolation

WebGPU requires SharedArrayBuffer, which needs these headers:

Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Opener-Policy: same-origin

Vercel makes this easy with vercel.json, but it breaks if you have external resources without CORS.

2. Memory Management

Browsers aren't designed for 4GB models. I had to:

Clear vector store before loading new documents
Implement proper cleanup for embeddings
Handle model caching effectively

3. WebGPU Compatibility

Not all browsers support WebGPU yet. Fallback to WebAssembly works, but it's significantly slower. Added detection logic to guide users to compatible browsers.

What I'd Do Differently

Implement proper vector indexing - Currently brute force cosine similarity
Add model quantization options - Let users choose speed vs quality
Better chunking strategies - Currently just splitting at 500 chars
Streaming for large documents - Don't embed everything at once
Support for multiple document formats - Not just PDFs

Privacy Win

One unexpected benefit: Complete privacy by default.

Your documents never leave your device. No API calls, no server uploads, no tracking. Everything happens in your browser.

This makes it useful for:

Sensitive documents (legal, medical, personal)
Offline environments
Privacy-conscious users
Demos without infrastructure costs

Try It Yourself

🔗 Live Demo: https://webpizza-ai-poc.vercel.app/

Requirements:

Chrome/Edge 113+ (WebGPU support)
4GB+ RAM
Modern GPU (or patience for CPU fallback)

Quick start:

git clone https://github.com/stramanu/webpizza-ai-poc
cd webpizza-ai-poc
npm install
npm start

Closing Thoughts

This is a proof-of-concept, not production software. It has bugs, rough edges, and questionable architectural decisions.

But it proves that browser-based AI is getting real. WebGPU + WebAssembly + modern JS frameworks = surprisingly capable local inference.

What would you build with this stack?

Inspired by: DataPizza AI

Questions? Issues? Drop a comment or open an issue on GitHub!

How I built a CLI tool to simplify my daily terminal workflow

Emanuele Strazzullo — Tue, 21 Oct 2025 15:01:28 +0000

As a developer, I live in the terminal. Every day I end up typing the same long commands: boot servers, build artifacts, sync repos, SSH into boxes, run one-off scripts. It’s fine… until it isn’t. I’d tweak flags, forget exact arguments, or copy–paste broken snippets.

I wanted something tiny, fast, and mine: a way to name the commands I use most and run them from anywhere with muscle‑memory simplicity.

That’s how mcl — My Command Line — was born.

🧠 The idea behind mcl

The “aha” moment came after repeating the same flows across projects. I didn’t want another framework. I wanted a thin layer over the shell where I could:

Describe commands once
Reuse them with arguments and variables
Keep them organized across projects
See what I have at a glance

With mcl you write small JSON recipes. For example:

{
  "scripts": {
    "say-hello": "echo 'Hello!'",
    "start-dev": "npm run dev"
  }
}

Then you just run:

mcl say-hello

No DSL. No ceremony. Just your shell, with a nicer memory.

⚙️ How it works (local vs global, args, vars)

mcl reads from two places and merges:

Local project config: ./mcl.json
Global config: ~/.mcl/global-mcl.json

Local overrides global, so you can keep common shortcuts globally (e.g., open-docs, deploy) and specialize per project.

It also supports:

Positional placeholders: $1, $2, … and optional ones like ?$1
Named vars: $project, $version via a vars object
Nested flows: scripts can be nested objects, e.g. example.date.utc
Dry-run mode: see exactly what would run before executing
Env sharing: --share-vars exports config vars and args to subprocesses

A richer example:

{
  "vars": {
    "project": "mcl",
    "version": "0.2.0"
  },
  "scripts": {
    "example": {
      "hello": "echo Hello, $1!",
      "date": {
        "utc": "date -u"
      }
    },
    "build": {
      "win": "GOOS=windows GOARCH=amd64 wails build"
    }
  }
}

Run them like this:

mcl --dry-run example date utc
mcl build win

And if you run plain mcl, it prints a handy list of available scripts (local first, then global).

Under the hood: it’s Python + Click, with strict type hints, pytest, mypy, and black. The CLI resolves your script path, applies substitutions, and executes steps in order.

🚀 What’s next

Here’s what I’m exploring next to make mcl smarter, safer, and more shareable:

AI‑assisted recipe generator: analyze your repo (Dockerfile, package.json, pyproject, Makefile, CI) and propose a ready‑to‑use mcl.json with common tasks.
Natural language → command recipes: “build and publish the docker image” becomes a reproducible script (with a dry‑run preview first).
Smart discovery and autocomplete: fuzzy search across local/global scripts, inline arg hints, and quick previews of what will run.
Safety checks: secret/unsafe flag linting and an optional “explain this command” powered by an LLM before you execute.
Team sharing & sync: share curated script packs across repos (and optionally sync via a gist or a small registry).
Plugin hooks & marketplace: pre/post hooks, custom resolvers, and reusable packs (e.g., Docker, Git, Node, Python).
Config schema validation (Pydantic) and optional YAML support.
Multi‑platform test matrix (tox) to keep behavior consistent across OSes and shells.

📦 Try it out

GitHub: https://github.com/stramanu/mcl-tool
PyPI: https://pypi.org/project/mcl-tool/

Quick start:

# install (recommended)
pipx install mcl-tool

# or in a virtual environment
pip install mcl-tool

# initialize a local config
mcl init

# run your scripts
mcl <script> [args...]

If this sounds useful, a ⭐️ on GitHub helps a ton — and I’d love to hear your feedback or feature ideas. Let’s make the terminal a bit more ergonomic together.