DEV Community: Xuepoo Foter

Rethinking Web UIs: A Canvas-Native Retained-Mode UI Runtime

Xuepoo Foter — Thu, 09 Jul 2026 07:29:32 +0000

The browser DOM is a masterpiece of software engineering. For the last three decades, it has served as the bedrock of the internet, excelling at flowing text, document-first layouts, SEO-heavy prose, and standard form interfaces.

But the DOM has a fundamental bottleneck: it is designed around a document flow model.

When you build applications that behave more like interactive scenes than static documents—such as complex mathematical simulations, node-based diagrams, streaming data dashboards, 3D WebGL scenes, or WebXR interfaces—the "one DOM element per visual item" model breaks down.

At thousands of entities, the styling, layout computation, and paint operations of the browser’s render pipeline become a massive bottleneck. The traditional workaround has been to drop down into a <canvas> element. However, doing so immediately strips away the most crucial layers of the modern web stack: layout helper systems, event propagation, accessibility, and automation.

That is why we built VectoJS (available at vectojs.org) — a zero-dependency, canvas-native UI runtime driven by a Virtual Math Tree.

What is VectoJS?

VectoJS is a retained-mode UI runtime. Instead of placing styled HTML elements in the browser DOM, you construct a hierarchical JavaScript scene graph called the Virtual Math Tree (VMT). VectoJS manages the layout, computes coordinate transforms, handles pointer events, and paints the final result to high-performance, canvas-backed rendering layers.

       [ Application State Updates ]
                    │
                    ▼
       ┌───────────────────────────┐
       │   Virtual Math Tree (VMT) │  <── Hierarchical Entities & Layouts
       └────────────┬──────────────┘
                    │
         ┌──────────┴──────────┐
         ▼                     ▼
┌─────────────────┐   ┌─────────────────┐
│ Canvas/GPU Paint│   │  Semantic DOM   │  <── Assistive Tech, Screen Readers,
│  (WebGL/WebGPU) │   │   Projection    │      and AI Coding Agents
└─────────────────┘   └─────────────────┘

By bypassing the browser DOM's layout engine, VectoJS gives you pixel-perfect rendering control and high-performance operations, without requiring you to write custom layout math or hit-testing algorithms from scratch.

The Core Pillars of VectoJS

We designed VectoJS to solve the three historic problems of canvas-rendered interfaces: accessibility, AI automation, and layout complexity.

1. Semantic DOM Projection

The biggest critique of canvas-rendered sites is that they are completely inaccessible to screen readers and assistive technology.

VectoJS solves this with Semantic DOM Projection. While the visible UI is painted to the canvas, VectoJS projects real, invisible semantic HTML nodes (<button>, <input>, <a>, <div role="dialog">, etc.) over the corresponding coordinates on the canvas.

When a screen reader navigates the page, it reads the projected DOM tree. When a user tabs to a button and presses Enter, the browser triggers the event on the projected node, which VectoJS captures and routes back to the canvas entity. This keeps your canvas controls accessible, supports native browser autofill and soft keyboards, and complies with modern無障碍 (accessibility) standards.

2. Agent-Friendly Architecture

We are entering the era of AI-driven software engineering. But to an AI Coding Agent (like Gemini or Claude), a traditional WebGL canvas is a black box. An agent cannot read the rendering pixels to understand where a button is or if a layout is broken.

VectoJS exposes a structured, queryable scene graph. Because the semantic projection maps canvas coordinates to real, accessible HTML elements, AI agents can read the DOM projection, understand the interface structure, trigger clicks, input text, and programmatically verify UI states. VectoJS was built from day one to be pair-programmed by AI.

3. Unified GPU and UI Threading

If you are building a Three.js scene, a custom shader, or a high-velocity particle simulation, you often need interactive controls directly embedded inside the rendering loop.

VectoJS provides adapters (like @vectojs/three) that let you project a VectoJS scene graph directly onto a 3D texture, routing 3D raycast pointer inputs back to 2D UI events seamlessly. You can stack standard buttons, dropdowns, and markdown text elements directly inside WebGL and WebXR viewports without introducing layout lag.

How VectoJS Differs from Typical Canvas Libraries

Most canvas libraries provide low-level drawing primitives (like rectangles, circles, and path APIs) and leave high-level concepts to the application developer. VectoJS provides a complete runtime stack.

Feature Layer	VectoJS	Typical Canvas Libraries (PixiJS, EaselJS, etc.)
Layout System	Built-in hierarchical stacks, grids, lists, and cards	Manual coordinate calculation
Hit-Testing	Automatic transform conversion and shape testing	Manual bounding-box or distance calculations
Event Routing	Full capture and bubbling phases (similar to DOM)	Callback-only or flat hit-testing
Accessibility	Built-in Semantic DOM Projection	Usually absent; requires manual overlay syncing
Text Handling	Wrapping, BiDi, Arabic support, and MSDF font paths	Simple single-line `fillText` only
Components	Buttons, Markdown, Toggle, Scroll, Tables, Dropdowns	Custom-built by the application
Video Export	Deterministic fixed-step frame-by-frame recorder	External canvas recording hacks

Getting Started: Hello World in 20 Lines

VectoJS is built to be modern and lightweight. It runs on Bun, supports npm, and compiles cleanly with TypeScript.

1. Install the Core Packages

bun add @vectojs/core @vectojs/ui

2. Build Your First Scene

import { Scene, Stack } from '@vectojs/core';
import { Button, Text } from '@vectojs/ui';

// 1. Initialize the canvas-backed Scene
const canvas = document.getElementById('gallery-canvas') as HTMLCanvasElement;
const scene = new Scene(canvas, { maxFPS: 60 });

// 2. Create a responsive vertical stack layout
const layout = new Stack({
  direction: 'vertical',
  gap: 16,
  padding: 24,
  background: '#1a1a24',
  borderRadius: 12
});

// 3. Add UI components to the layout
layout.add(new Text('VectoJS Engine', { font: 'bold 24px sans-serif', color: '#ffffff' }));
layout.add(new Text('Bypassing the DOM for high-performance visual scenes.', { font: '14px sans-serif', color: '#888899' }));
layout.add(new Button('Start Simulation', { 
  onClick: () => console.log('Simulation starting...'),
  style: { background: '#5d5dff', color: '#ffffff' }
}));

// 4. Mount the layout to the scene and start the loop
scene.add(layout);
scene.start();

When to Choose VectoJS

VectoJS Shines in:

High-Entity Visuals: Interfaces containing thousands of moving items or interactive data points.
Math-Driven Layouts: Flowcharts, diagrams, graphing interfaces, and custom physics animations.
3D & Spatial Web: UI overlays embedded directly inside WebGL/WebGPU frames or WebXR environments.
Deterministic Recording: Interactive workflows that need to be exported frame-by-frame to high-quality H.264 video.

Discover More & Support Us

VectoJS is fully open-source and ready for exploration.

🌐 Documentation & Guide: Check out the official website at vectojs.org for detailed guides on accessibility, performance optimizations, and package structures.
📦 GitHub Repository: Visit github.com/vectojs/vectojs to star the project, browse the source code, or contribute to our Core, UI, or Three.js packages.
🎨 Interactive Gallery: Experience VectoJS Native apps in action at vectojs-gallery.

I Threw Out the DOM, Kept Accessibility

Xuepoo Foter — Thu, 09 Jul 2026 05:48:13 +0000

Canvas-native UI runtime with a Virtual Math Tree, semantic accessibility projection, WebGL/WebGPU backends, and agent-drivable controls.

VectoJS is an open-source TypeScript UI framework that renders an entire application to a single <canvas> element instead of the DOM. Every UI framework of the last decade has made a different bet: the DOM is the substrate, and the framework's job is to manage it faster, or hide it behind a nicer syntax. React manages it with a virtual diff. Svelte compiles away the runtime cost of managing it. Tailwind makes styling it less painful. All of them still assume that a UI, underneath, is a tree of HTML elements.

VectoJS doesn't make that assumption. It renders everything — text, buttons, scrollable lists, thousands of live data points — to one <canvas> element. There is no HTML tree underneath. A Button is a plain TypeScript object with a position, a size, and a render() method; the whole UI is a retained scene graph of these objects, walked and drawn every frame like a game engine's scene, not laid out like a document.

class DangerButton extends Button {
  override render(r: IRenderer): void {
    r.beginPath();
    r.roundRect(0, 0, this.width, this.height, 8);
    r.fill('#dc2626');
    super.render(r); // draw the label on top
  }
}

No CSS specificity, no cascade, no layout thrashing from reading a style back after writing it — the object's own fields are the only state that exists. Inheritance and composition work exactly like they do in any other TypeScript codebase, because this is just TypeScript, not a templating language pretending to be one.

The obvious objection

If you've gotten this far, you're probably thinking: fine, but you've just reinvented <canvas> games, and canvas is an accessibility black hole. Screen readers see a single opaque bitmap. Ctrl+F finds nothing. That's true of every canvas app that came before this one, and it's the actual reason most teams don't consider canvas for real UI.

Here's the mechanism we built to close that gap, concretely: every entity that's interactive projects a single, invisible, real DOM node — a <button>, an <input>, an <a> — into a layer positioned exactly over the canvas. That node's transform is re-synced every frame, so if the entity moves, scales, or rotates, the invisible DOM node moves with it. Keyboard focus, native pointer events, and screen readers all land on that real element, because it is one — just transparent, sitting precisely on top of what's drawn.

This is not free, and we're not going to pretend it is: it's a real DOM node per interactive entity, kept in sync every frame. It's the right cost for buttons, links, and form controls — the things a person actually needs to reach — and it's exactly why we don't set it on the other 99% of a typical VectoJS scene: individual particles, glyphs, and data points cost nothing but a draw call, because nothing about them is interactive.

What that split buys you

Because the non-interactive path is free, the workloads VectoJS is actually built for are the ones that make a DOM-based UI fall over: a full node-graph editor with hundreds of draggable, connected nodes and live bezier curves, redrawn at 60fps with zero DOM elements for the graph itself; a WebGPU compute pass moving tens of thousands of particles, with a CPU fallback when WebGPU isn't there; a text reader that reflows an entire book around your cursor in real time, which is a layout operation no browser engine is built to do smoothly at that scale.

Try the node editor · Try the particle field · Try the reflowing reader

We're not going to hand you a single "N× faster" number, because there isn't one — it depends on how much of your scene is actually interactive, your GPU, and your browser. What we do ship is a live FPS/entity-count readout on the demos themselves, so you can measure your own hardware instead of trusting ours.

Try it yourself

Everything here is MIT-licensed and on GitHub: the engine, the docs site you might be reading this on, and a community gallery where anyone can submit their own VectoJS creation as a one-file pull request.