What if a game-engine AI agent could work without an API key or send the project prompt to a cloud model?
I have been building that option into Feather Engine while keeping its existing OpenAI, Anthropic, and Google providers intact. Users can now choose Local AI (WebGPU) in the same agent panel, download the model deliberately, and let it use the engine's real tools rather than a separate demo chatbot.
The architecture
The local path uses AI SDK 6, browser-ai's Transformers.js provider, and a dedicated Web Worker. The first curated model is Qwen3 0.6B in q4f16 form: small enough to be realistic in a browser, but still a roughly 570 MB download.
Model weights live in a Feather-owned browser cache. Nothing downloads until the user confirms it. The UI exposes hardware compatibility, progress, cancellation, loading, unloading, and cache clearing. Local mode does not require a provider key and never silently falls back to a cloud model.
Making the local model control the whole engine
The hard part was not text generation. Feather's agent has 223 engine tools covering scenes, objects, materials, physics, scripts, UI, animation, assets, and project workflows. Sending every definition and the full cloud prompt to a 0.6B model would overwhelm its useful context.
Instead, every tool belongs to a tested capability group. A lightweight router selects the relevant groups for each request while retaining a small core set. The local model therefore receives a compact prompt and a focused tool surface, but every engine capability remains reachable. Both local and cloud models call the same tool instances and mutation layer.
Safety and lifecycle details
- Explicit consent before the first model download
- WebGPU and shader-f16 checks before loading
- Abortable generation without destroying an already loaded model
- Worker termination when unloading or switching models
- Cache deletion limited to Feather's own model cache
- Persisted provider settings that preserve existing cloud keys and model choices
Verification
The implementation currently passes 512 automated tests, a production build, and a rendered browser smoke test for consent, provider switching, and cloud-setting preservation. A real-GPU inference run in the supported desktop/browser matrix remains an important release gate because headless CI cannot meaningfully prove driver behavior.
What I learned
Small local models benefit more from disciplined context design than from pretending they are cloud-scale models. Tool routing, compact instructions, visible download state, and a strict no-fallback privacy contract make the feature feel like part of the engine instead of an embedded model demo.
I am excited about where browser-native agents can go next: stronger curated models, better capability routing, and more work that stays on the user's machine.
Top comments (0)