DEV Community

Cover image for Rethinking Web UIs: A Canvas-Native Retained-Mode UI Runtime
Xuepoo Foter
Xuepoo Foter

Posted on

Rethinking Web UIs: A Canvas-Native Retained-Mode UI Runtime

The browser DOM is a masterpiece of software engineering. For the last three decades, it has served as the bedrock of the internet, excelling at flowing text, document-first layouts, SEO-heavy prose, and standard form interfaces.

But the DOM has a fundamental bottleneck: it is designed around a document flow model.

When you build applications that behave more like interactive scenes than static documents—such as complex mathematical simulations, node-based diagrams, streaming data dashboards, 3D WebGL scenes, or WebXR interfaces—the "one DOM element per visual item" model breaks down.

At thousands of entities, the styling, layout computation, and paint operations of the browser’s render pipeline become a massive bottleneck. The traditional workaround has been to drop down into a <canvas> element. However, doing so immediately strips away the most crucial layers of the modern web stack: layout helper systems, event propagation, accessibility, and automation.

That is why we built VectoJS (available at vectojs.org) — a zero-dependency, canvas-native UI runtime driven by a Virtual Math Tree.


What is VectoJS?

VectoJS is a retained-mode UI runtime. Instead of placing styled HTML elements in the browser DOM, you construct a hierarchical JavaScript scene graph called the Virtual Math Tree (VMT). VectoJS manages the layout, computes coordinate transforms, handles pointer events, and paints the final result to high-performance, canvas-backed rendering layers.

       [ Application State Updates ]
                    │
                    ▼
       ┌───────────────────────────┐
       │   Virtual Math Tree (VMT) │  <── Hierarchical Entities & Layouts
       └────────────┬──────────────┘
                    │
         ┌──────────┴──────────┐
         ▼                     ▼
┌─────────────────┐   ┌─────────────────┐
│ Canvas/GPU Paint│   │  Semantic DOM   │  <── Assistive Tech, Screen Readers,
│  (WebGL/WebGPU) │   │   Projection    │      and AI Coding Agents
└─────────────────┘   └─────────────────┘
Enter fullscreen mode Exit fullscreen mode

By bypassing the browser DOM's layout engine, VectoJS gives you pixel-perfect rendering control and high-performance operations, without requiring you to write custom layout math or hit-testing algorithms from scratch.


The Core Pillars of VectoJS

We designed VectoJS to solve the three historic problems of canvas-rendered interfaces: accessibility, AI automation, and layout complexity.

1. Semantic DOM Projection

The biggest critique of canvas-rendered sites is that they are completely inaccessible to screen readers and assistive technology.

VectoJS solves this with Semantic DOM Projection. While the visible UI is painted to the canvas, VectoJS projects real, invisible semantic HTML nodes (<button>, <input>, <a>, <div role="dialog">, etc.) over the corresponding coordinates on the canvas.

When a screen reader navigates the page, it reads the projected DOM tree. When a user tabs to a button and presses Enter, the browser triggers the event on the projected node, which VectoJS captures and routes back to the canvas entity. This keeps your canvas controls accessible, supports native browser autofill and soft keyboards, and complies with modern無障碍 (accessibility) standards.

2. Agent-Friendly Architecture

We are entering the era of AI-driven software engineering. But to an AI Coding Agent (like Gemini or Claude), a traditional WebGL canvas is a black box. An agent cannot read the rendering pixels to understand where a button is or if a layout is broken.

VectoJS exposes a structured, queryable scene graph. Because the semantic projection maps canvas coordinates to real, accessible HTML elements, AI agents can read the DOM projection, understand the interface structure, trigger clicks, input text, and programmatically verify UI states. VectoJS was built from day one to be pair-programmed by AI.

3. Unified GPU and UI Threading

If you are building a Three.js scene, a custom shader, or a high-velocity particle simulation, you often need interactive controls directly embedded inside the rendering loop.

VectoJS provides adapters (like @vectojs/three) that let you project a VectoJS scene graph directly onto a 3D texture, routing 3D raycast pointer inputs back to 2D UI events seamlessly. You can stack standard buttons, dropdowns, and markdown text elements directly inside WebGL and WebXR viewports without introducing layout lag.


How VectoJS Differs from Typical Canvas Libraries

Most canvas libraries provide low-level drawing primitives (like rectangles, circles, and path APIs) and leave high-level concepts to the application developer. VectoJS provides a complete runtime stack.

Feature Layer VectoJS Typical Canvas Libraries (PixiJS, EaselJS, etc.)
Layout System Built-in hierarchical stacks, grids, lists, and cards Manual coordinate calculation
Hit-Testing Automatic transform conversion and shape testing Manual bounding-box or distance calculations
Event Routing Full capture and bubbling phases (similar to DOM) Callback-only or flat hit-testing
Accessibility Built-in Semantic DOM Projection Usually absent; requires manual overlay syncing
Text Handling Wrapping, BiDi, Arabic support, and MSDF font paths Simple single-line fillText only
Components Buttons, Markdown, Toggle, Scroll, Tables, Dropdowns Custom-built by the application
Video Export Deterministic fixed-step frame-by-frame recorder External canvas recording hacks

Getting Started: Hello World in 20 Lines

VectoJS is built to be modern and lightweight. It runs on Bun, supports npm, and compiles cleanly with TypeScript.

1. Install the Core Packages

bun add @vectojs/core @vectojs/ui
Enter fullscreen mode Exit fullscreen mode

2. Build Your First Scene

import { Scene, Stack } from '@vectojs/core';
import { Button, Text } from '@vectojs/ui';

// 1. Initialize the canvas-backed Scene
const canvas = document.getElementById('gallery-canvas') as HTMLCanvasElement;
const scene = new Scene(canvas, { maxFPS: 60 });

// 2. Create a responsive vertical stack layout
const layout = new Stack({
  direction: 'vertical',
  gap: 16,
  padding: 24,
  background: '#1a1a24',
  borderRadius: 12
});

// 3. Add UI components to the layout
layout.add(new Text('VectoJS Engine', { font: 'bold 24px sans-serif', color: '#ffffff' }));
layout.add(new Text('Bypassing the DOM for high-performance visual scenes.', { font: '14px sans-serif', color: '#888899' }));
layout.add(new Button('Start Simulation', { 
  onClick: () => console.log('Simulation starting...'),
  style: { background: '#5d5dff', color: '#ffffff' }
}));

// 4. Mount the layout to the scene and start the loop
scene.add(layout);
scene.start();
Enter fullscreen mode Exit fullscreen mode

When to Choose VectoJS

VectoJS Shines in:

  • High-Entity Visuals: Interfaces containing thousands of moving items or interactive data points.
  • Math-Driven Layouts: Flowcharts, diagrams, graphing interfaces, and custom physics animations.
  • 3D & Spatial Web: UI overlays embedded directly inside WebGL/WebGPU frames or WebXR environments.
  • Deterministic Recording: Interactive workflows that need to be exported frame-by-frame to high-quality H.264 video.

Discover More & Support Us

VectoJS is fully open-source and ready for exploration.

  • 🌐 Documentation & Guide: Check out the official website at vectojs.org for detailed guides on accessibility, performance optimizations, and package structures.
  • 📦 GitHub Repository: Visit github.com/vectojs/vectojs to star the project, browse the source code, or contribute to our Core, UI, or Three.js packages.
  • 🎨 Interactive Gallery: Experience VectoJS Native apps in action at vectojs-gallery.

Top comments (0)