PixelRAG makes a blunt bet: for retrieval, a screenshot of a web page beats the text you would parse out of it. Instead of converting a page or PDF into text chunks and hoping the parser kept the table intact, the project renders the document to images and searches over the pixels directly.
That claim is the title of the paper behind the repo, "Web Screenshots Beat Text for Retrieval-Augmented Generation," from Berkeley SkyLab, BAIR, and Berkeley NLP. The README's own diagram makes the case with one example: a text-based pipeline parses a page, loses the table, and the reader model has nowhere to find the answer. PixelRAG renders that same page to screenshot tiles, retrieves the right tile, and the reader reads the number straight off the image.
What actually changes
Two pieces carry the idea. First, capture: the pixelshot command turns any URL, PDF, or local file into image tiles using headless Chromium over CDP. Second, embedding: a Qwen3-VL-Embedding model, LoRA-fine-tuned on screenshot data, maps those page images into a vector space where visual content is retrievable. Tables, charts, infographics, and layout survive, because nothing was ever flattened into text in the first place.
The rest is a conventional retrieval stack. pixelrag chunk, embed, and build-index turn tiles into vectors and then into a FAISS index. pixelrag serve puts a FastAPI search endpoint in front of that index on CPU or GPU. You install only the stages you need, and each stage runs standalone without the orchestrator, so you can render on one machine and embed on another.
You can also skip the build entirely. The team hosts an endpoint at api.pixelrag.ai with a pre-built index of 8.28M Wikipedia pages, no key and no setup, and it answers a POST with either a text query or an image query. If you want the index locally, it lives on Hugging Face, though the base Wikipedia index is about 217G, so plan disk accordingly.
Give a coding agent eyes
The part most people can try in a few minutes is the Claude Code plugin. The same renderer ships as a skill called pixelbrowse. Install the pixelshot CLI so it sits on your PATH, add the marketplace, install the plugin, and then Claude screenshots a page and reads the image instead of fetching raw HTML:
claude -p "screenshot https://news.ycombinator.com and summarize the top stories"
There is no MCP server and no backend. The skill just calls pixelshot (Playwright/CDP) on your machine, in an isolated throwaway Chrome profile, so it works even while you have Chrome open. For an agent, charts and dashboards that are invisible to a plain HTML fetch become readable.
Where it fits, and the fine print
The honest catch is hardware and disk. Embedding runs a vision-language model, so it wants a GPU on Linux or Apple Silicon on a Mac. The README's own smoke test indexes a single PDF and searches it locally, about three minutes on an M-series chip and about a minute on a GPU, which is a fair way to feel the whole loop before committing to anything large. Training is walled off in its own train/ uv project with a pinned CUDA stack, and you do not need it: the LoRA adapters and the full screenshot training set are both published, so adapting a larger backbone is a supported path rather than a rewrite.
PixelRAG is a research codebase with a working product attached: a paper, a live demo, a hosted API, and a plugin. The question it poses is not whether pixels can beat text on every corpus, but how much retrieval quality your current pipeline is quietly losing at the parse step.
GitHub: https://github.com/StarTrail-org/PixelRAG
Curated by Agent Palisade — practical AI for small and mid-sized businesses.
Top comments (0)