DEV Community

GridPort
GridPort

Posted on

Using PixelRAG with Claude Code (August 2026) — Visual RAG for Documents with Tables and Diagrams

#ai #python #rag #claude

PixelRAG is a visual RAG tool that treats web pages, PDFs, and images as screenshots rather than text — preserving the layout of tables and charts so you can search and reference them as-is.

This post covers installing it as a plugin, actual usage, how it differs from traditional text-based RAG, and the gotchas you're likely to hit — all from a Claude Code user's perspective.

What you'll get out of this post

  • You're an intermediate engineer already using Claude Code for real work
  • You need to search or understand documents where tables, diagrams, and layout matter
  • You want to know how this differs from traditional text RAG (embeddings + a vector DB)

What is PixelRAG?

PixelRAG renders documents — web pages, PDFs, images — as screenshots and feeds those images directly to the model. The visual structure that HTML parsing normally destroys — tables, charts, layout, infographics — stays intact, so the model can actually answer questions about them.

PixelRAG is an open-source project built around "Visual Retrieval-Augmented Generation," made up of 5 components:

Component Role
pixelrag-render Converts documents (web pages, PDFs) into image tiles (via Playwright/CDP)
pixelrag-embed Vectorizes tile images and builds a FAISS index
pixelrag-index Runs the full source → ingest → embed → index pipeline
pixelrag-serve Serves a FAISS search API (CPU/GPU)
pixelrag-train Fine-tunes Qwen3-VL-Embedding via LoRA

These are all now bundled into a single pixelrag package, installable with one pip install pixelrag. As a Claude Code user, the first thing you'll actually touch is the pixelshot command (shipped by pixelrag-render) and the "pixelbrowse" plugin that wires it into Claude Code.

flowchart LR
    A[URL / PDF] --> B["pixelshot<br/>(generates image tiles)"]
    B --> C["tile_0000.jpg ..."]
    C --> D["Claude Code's Read tool"]
    D --> E["Claude understands it visually"]
    C -.->|optional| F[pixelrag-embed / index]
    F --> G[FAISS index]
    G --> H[pixelrag-serve search API]
Enter fullscreen mode Exit fullscreen mode

How this differs from traditional text RAG

  • Row/column relationships in tables (these tend to break down under text extraction)
  • The actual content of charts, diagrams, and infographics
  • Meaning carried by layout itself (emphasis, the position of annotations, etc.)

Setting up PixelRAG

Setup is straightforward: either clone the repo and run it locally, or add the plugin via the marketplace.

Before you start

  • Python 3.12+ (per requires-python in pyproject.toml)
  • Claude Code CLI already installed
  • License: Apache-2.0 (commercial use allowed)
  • Note: the repo's pyproject.toml includes environments = ["sys_platform == 'linux'"], meaning the GPU-dependent parts (embed/serve/train) assume Linux. This shouldn't matter much if you're only using the screenshot feature (pixelshot), but on Mac/Windows you're safer running it through WSL.
  • Cost note: the pixelshot screenshot feature just runs Playwright/Chromium locally, so there's no extra cost beyond your normal Claude Code token usage. Building your own index with embed/serve/train, however, needs a GPU, and if you use a cloud GPU for that, you'll pay for that usage separately.

Installation

The official plugin/setup.sh looks like this:

#!/bin/bash
# One-liner that installs pixelrag and registers it as a Claude Code plugin
set -e

# Install pixelrag into an isolated environment via uv
uv tool install --from "$REPO_DIR" pixelrag 2>/dev/null || \
    uv tool upgrade --from "$REPO_DIR" pixelrag

# Install Chromium for screenshots
uvx playwright install chromium 2>/dev/null || true
Enter fullscreen mode Exit fullscreen mode

Option 1: Clone the repo and run it locally

git clone https://github.com/StarTrail-org/PixelRAG.git
cd PixelRAG
./plugin/setup.sh
claude --plugin-dir ./plugin
Enter fullscreen mode Exit fullscreen mode

Option 2: Install via the marketplace

pip install pixelrag                                # installs the pixelshot command
claude plugin marketplace add StarTrail-org/PixelRAG
claude plugin install pixelbrowse@pixelrag-plugins
Enter fullscreen mode Exit fullscreen mode

With the plugin installed, Claude is set up to call pixelshot via Bash and then read the generated images with the Read tool.

Using PixelRAG with Claude Code

Once installed, you just pass a URL in regular conversation and it works.

claude -p "Look at https://news.ycombinator.com and summarize the top stories"
Enter fullscreen mode Exit fullscreen mode

In an interactive session, you can also use the slash command:

claude --plugin-dir ./plugin
# inside the session
/screenshot https://example.com
Enter fullscreen mode Exit fullscreen mode

Under the hood, Claude runs something like the following pixelshot command via Bash:

# Screenshot a URL (tile height optimized to 1568px for Claude's vision model)
pixelshot https://example.com --output /tmp/pixelbrowse --tile-height 1568 --wait-network-idle

# Process multiple URLs in parallel
pixelshot url1 url2 --output /tmp/pixelbrowse --tile-height 1568 --wait-network-idle --workers 4

# Render a PDF
pixelshot document.pdf --output /tmp/pixelbrowse
Enter fullscreen mode Exit fullscreen mode

Output is saved with a naming pattern like /tmp/pixelbrowse/<domain>.png.tiles/tile_0000.jpg, and Claude reads it in as an image to understand the content.

Gotchas, troubleshooting, and where this is useful

The following notes come straight from the official SKILL.md and matter in practice.

Forgetting --wait-network-idle gives you a blank page
Sites that render via JavaScript will get captured before they've finished loading if you skip this flag, leaving you with an empty screenshot. Always include it when targeting a URL.

Stick with the default --tile-height of 1568px
Claude's vision models downscale images whose long edge exceeds 1568px (Sonnet/Haiku) or 2576px (Opus) before processing them. Leave the default at 8192px and the text becomes too compressed to read.

If text is too small to read, crop and re-read it
The official workflow is to crop the relevant region with Pillow and feed it back through the Read tool.

Where PixelRAG is useful

  • Searching specs, IR documents, and anything else with tables and charts that text extraction tends to mangle
  • Checking your own site's UI for visual bugs (e.g. "screenshot http://localhost:3000 and tell me if anything looks broken")
  • Understanding papers or scanned PDFs where layout carries meaning

Wrapping up

PixelRAG specializes in exactly what traditional text RAG struggles with: searching documents while preserving tables, diagrams, and layout.

Wiring it into Claude Code doesn't require an MCP server at all — it's a skill-only setup that comes down to a single pixelshot command. Before using it in production, check the official repo for the latest status.

📌 This post reflects information as of August 2026. Since Claude Code updates frequently, check the official docs for the latest specifics.

This article was edited with AI assistance.
*Originally published in Japanese on EdgeHUB.

Top comments (0)