Building a Private, On-Device AI Dungeon Master in the Browser

I’ve been spending the last few months wrestling with a specific constraint: how do you bring the generative power of large language models into a real-time game loop without turning the experience into a slideshow of API calls?

The industry standard answer right now is "cloud inference." You send a prompt, wait for the stream, render the text, and hope the latency doesn’t break immersion. For a text-based adventure, that’s tolerable. For a dungeon crawl where you’re clicking every two seconds, it’s a friction point that kills the flow.

So I built Grimhollow. It’s an infinite dark-fantasy dungeon crawler, but the hook isn’t the art or the combat mechanics. The hook is that the entire narrative engine runs locally on your machine using WebGPU. There is no server-side AI processing. Your prompts never leave your browser. And because it’s on-device, it works entirely offline once the model is cached.

The Wedge: Latency vs. Privacy

Most indie devs looking to integrate AI are forced to choose between speed and privacy. If you want speed, you use a small, quantized model hosted on a fast edge server, but you’re trusting that server with your context window. If you want privacy, you run a model locally, but you’re often stuck waiting 3-5 seconds for a response on a mid-tier laptop.

I wanted to prove that with WebGPU acceleration, we can get the best of both. By offloading the matrix multiplications to the GPU, we can achieve token generation speeds that feel native to the UI, not like a chat interface bolted onto a game.

Under the Hood

The architecture is surprisingly simple, which is part of the appeal for anyone looking to replicate this pattern.

1. The Runtime: The game is a standard web app, but it leverages the WebGPU API to load a quantized language model directly into the browser’s memory.
2. The Context Window: Unlike a chat app where history grows indefinitely, a dungeon crawl has a natural "scene" boundary. When you enter a new room or encounter a new enemy, the context resets. This keeps the memory footprint low and the inference speed high.
3. The Prompt Structure: The AI doesn’t just generate prose; it generates structured JSON alongside the narrative. This allows the game engine to parse HP changes, inventory updates, and state flags without post-processing the text.

Here is a simplified look at how the inference call is structured in the code. Note that we never specify a model name in the user-facing code because the model is bundled as a binary blob loaded by the WebGPU runtime:

// The inference loop runs on the main thread but delegates to GPU
async function generateNextStep(currentState) {
  const prompt = buildPrompt(currentState);

  // 'localModel' is the compiled WebGPU graph
  const response = await localModel.generate(prompt, {
    maxTokens: 64,
    temperature: 0.7
  });

  return parseResponse(response);
}

The "Infinite" Problem

The biggest challenge wasn’t technical; it was design. When you have an AI GM, the world is theoretically infinite. But infinite choices lead to player paralysis. To solve this, Grimhollow uses a "constrained randomness" approach. The AI is given strict genre tags (dark fantasy, gothic horror) and a limited vocabulary of items and enemies. This ensures that while every dungeon is unique, the feel remains consistent. You don’t get a sci-fi spaceship in a medieval crypt because the system prompt explicitly forbids it.

Performance Realities

I want to be honest about the hardware requirements. This isn’t magic. It’s heavy lifting.

On a modern MacBook Pro or a Windows laptop with a dedicated GPU, the experience is seamless. The model loads in a few seconds, and narration appears almost instantly as you explore. On older hardware or integrated graphics, there is a noticeable delay—about 1-2 seconds per turn. It’s not "real-time" in the sense of an action game, but it’s fast enough for a tactical dungeon crawler.

Also, because the model is running locally, the initial download is substantial. We’re talking about a 2-4GB download for the model weights depending on the quantization level. This is a trade-off for the privacy and offline capability. If you’re on a metered connection, you’ll feel that upfront cost.

Trying It Out

If you want to see what this feels like in practice, you can check out the game here: Grimhollow.

It’s a paid tool, but there is a 7-day trial period. For those who prefer to test before committing, the games included in the trial offer free turns so you can experience the local AI generation without a credit card.

Why This Matters for Devs

I’m sharing this because I think we’re at an inflection point for browser-based AI. We’ve spent the last two years treating the browser as a thin client for cloud APIs. WebGPU changes that equation. It allows us to treat