DEV Community: Yorkie Liu

Rethinking the Web: Inside the Spatial Browser Engine (JSAR)

Yorkie Liu — Wed, 17 Sep 2025 05:14:45 +0000

The web we know today is built around 2D documents. HTML, CSS, and JavaScript together create interfaces inside a flat, rectangular viewport. But what happens if we bring the entire web into 3D space?

That’s the question the JSAR Spatial Browser Engine is tackling. Instead of patching existing 2D browsers, JSAR reimagines the browser from the ground up as a spatial-first engine.

👉 Full article here: Understanding the Spatial Web Browser Engine | YODAOS JSAR

🌐 What is a Spatial Web Browser?

A spatial browser doesn’t just render content on a 2D plane. Instead, it loads HTML, CSS, JS, WebGL/WebXR, and media into a 3D coordinate system.

DOM elements (text, images, forms, canvas, video, etc.) exist as 3D objects with position, rotation, scale, and depth.
Users interact through spatial inputs like gaze, gestures, and controllers.
Web content and 3D models coexist seamlessly in the same space.

🏗 Why not extend existing browsers?

Traditional engines (Blink, Gecko, WebKit) are optimized for 2D rendering pipelines. Forcing them to handle 3D scenes adds complexity and inefficiency.

JSAR instead designs a new architecture from scratch, aligned with the needs of immersive and spatial experiences.

✨ Key Features of JSAR

Here are some highlights from the engine’s design:

Spatial DOM → Elements come with 3D transforms (translate3d, rotate, scale) and real-world units (meters, not pixels).
Unified Rendering Pipeline → HTML UI and 3D models share the same rendering loop, with proper depth sorting and blending.
Batch Rendering Optimization → DOM elements are rendered as textured quads with instancing to minimize draw calls.
Automatic Spatial Audio → elements automatically gain positional sound effects without extra Web Audio setup.
Input Mapping → XR input sources (gaze, gestures, controllers) map directly to DOM events like click and mousemove.
Embedded Integration → Works as a library inside Unity (Unreal planned), rendering web content as native 3D UI components.

👩‍💻 Who benefits from this?

Web Developers → Bring existing web apps into 3D space without rewriting everything in XR frameworks.
WebXR Developers → Use HTML/CSS for spatial UI instead of building custom 3D GUI systems.
Desktop & Workflow Innovators → Even outside XR, spatial browsers can support infinite 3D canvases for multitasking and data visualization.

🔮 Looking Ahead

JSAR is still evolving, with current support for Unity + macOS/Android (OpenGL/GLES3). Future directions include Unreal integration, advanced transparency handling, and scaling across more platforms.

But the vision is clear: move from a 2D-first web to a space-first web—while keeping the openness and accessibility of existing web standards.

📖 Want to dive deeper?

This post is just a surface-level overview. The original article covers the architecture in much more detail, with diagrams and examples:

👉 Understanding the Spatial Web Browser Engine | YODAOS JSAR
👉 GitHub Repository: https://github.com/M-CreativeLab/jsar-runtime

Faster Pipcook 1.2, machine learning in JavaScript

Yorkie Liu — Tue, 08 Sep 2020 12:31:46 +0000

2 months later, Pipcook ushered in the release of the second stable version (v1.2). Let’s take a look at the improvements and enhancements in this version.

Features at a glance

In the past 2 months, the development team has made targeted optimizations for service startup, plug-in installation, and pipeline execution time, especially the time when the pipeline was executed the most by internal users to start training. It takes more than 5 minutes to start training the model, and the now optimized Pipeline only needs 10 seconds to start.

Train the model faster

In the v1.0 version, each pipeline is divided into different stages, such as DataCollect for collecting data sets, ModelDefine for defining models, or DatasetProcess for processing data sets. In the last stable version, training a simple component (picture) classification task took nearly 2 minutes to process the data (the time grows linearly with the size of the data set).

There are two reasons for this:

In the definition of v1.0 Pipeline, before the previous stage has completely processed the data, it will not enter the next stage, but in fact, for example, in the process of data collection and processing, there is a lot of I/O waiting time and CPU idle time.
In the definition of v1.0 Pipeline, data type plug-ins (DataCollect, DataAccess, DataProcess) were previously passed through the path of the file, which not only caused a large number of repeated disk read and write operations in a pipeline process, but also It makes it impossible to start some calculations for numbers like normalization (Normalization).

So in PR#410, the mechanism of asynchronous Pipeline is introduced and Sample is used as the unit of data transfer between plug-ins. The advantages of this are:

Once the first sample is produced by the previous plug-in, the subsequent plug-ins can be loaded. This solves the problem that the subsequent plug-ins need to wait for all the data to be processed, which greatly advances the training start time.
Reduce unnecessary and repetitive read and write operations. The Sample is transferred to the memory between plug-ins, and the processed value is stored in the memory for later use by the plug-ins.

With the help of the asynchronous pipeline, we successfully reduced the pipeline entry time from 1 minute 15 seconds to 11 seconds, and also shortened the overall training time.

Plug-in installation is faster

In the new version, we have also optimized the plug-in installation process. At present, most of the Pipelines in Pipcook still rely on the Python ecology. Therefore, when installing these plug-ins, the dependencies of Python and Node.js will be installed at the same time. In v1 Before .2, Pipcook was installed serially, so in PR#477, we parallelized the installation of Python and Node.js packages to reduce the overall installation time.

In subsequent versions, we will continue to explore the optimizations brought about by parallelization, and try to analyze each installation task (Python and Node.js packages) and schedule the installation tasks to achieve a more reasonable parallel installation.

Faster initial startup

Starting from Pipcook 1.2, users no longer need to install Pipboard locally. We deployed Pipboard as an online service through Vercel and migrated all the code to imgcook/pipboard.

Users can use the functions of Pipboard through https://pipboard.vercel.app/, but there are still some parts that need to be adjusted, such as the remote Pipcook Daemon is not supported.

The subsequent release cycle of Pipboard will be independent of Pipcook, that is to say, we encourage everyone to develop their own Pipboard based on Pipcook SDK, and Pipboard itself will be provided as a demo or a sample application provided by default.

Support Google Colab

If you continue to follow Pipcook users, you must have noticed early on that a link to Google Colab has been added at the beginning of some tutorials in the official document! Yes, Pipcook supports running on Google Colab, which means that for beginners who are trapped without GPU, they can learn Pipcook through the free GPU/TPU on Google Colab, just start from the following two links , You can start your front-end component identification journey:

Plug-in Python runtime for algorithm engineers

In order to facilitate algorithm engineers to contribute models to Pipcook at a lower threshold, we have added support for pure Python runtime. For contributors, in addition to defining an additional package.json, it can be completed without writing any JavaScript code. The development of plug-ins (model classes), and in order to facilitate algorithm engineers to get started easily, we developed a NLP (NER) Pipeline when the Python-based plug-in is running. The related plug-ins are as follows:

Pipcook SDK released

As mentioned earlier, we moved Pipboard out of Pipcook and released it independently. We hope that developers can develop Pipboard or any other form of application that suits their needs through Pipcook SDK. Therefore, we will officially release Pipcook SDK in v1.2. It supports the use of designated Pipcook services in the Node.js and JavaScript runtime environment to complete the management of Pipeline and training tasks.

const client = new PipcookClient('your pipcook daemon host', port);
const pipelines = await client.pipeline.list(); // list all pipelines

Pipcook SDK API documentation: Click here.

Daily (Beta) version and Release version

In order to allow users to use Pipcook selectively, we have updated our version release cycle in the past two months. The specific rules are as follows:

The Beta version or Daily version will be automatically built and released by the CI (GitHub Actions) system every day. If you want to try the latest version, users can use pipcook init beta or pipcook init --beta to get it.
Release version
- The base version (such as 1.1, 1.3, etc.) is an unstable version, which mainly incorporates some larger experimental features
- The even-numbered version (such as 1.0, 1.2, etc.) is a stable version, mainly for stability, performance, etc. more repairs and optimization
- All Release versions will follow the Semver2.0 specification

Next version plan (v1.4)

According to the plan, we will release Pipcook v1.4 in two months. The development team will still focus on how to make Pipcook "faster".

For example, after training the model, if you want to use it in the Node.js environment, you still need very lengthy steps to perform the installation of NPM (which will install Python and related dependencies), we hope that the model after training can be directly Use it without any tedious pre-steps.

In terms of models, we will support a more lightweight target detection model (YOLO/SSD), which can easily perform target detection tasks in some simple scenarios.

Boa: Use Python functions in Node.js

Yorkie Liu — Tue, 28 Jul 2020 15:22:43 +0000

Hey, everybody, this time I will mainly bring you a good thing. Its main purpose is to allow you to use Python interfaces and functions in Node.js. Maybe you will be curious, confused, and puzzled when I see it here. My Node.js magic is so good. Why use Python? If you have tried some applications of JavaScript for machine learning before, you will understand the reason behind this.

The current situation is that the machine learning ecosystem is almost tied to Python. The language is iterating at high speed, and JavaScript can only count on its own. If we expect to achieve Python’s current scale from zero, the amount of work required is huge. When I wrote tensorflow-nodejs years ago, I already thought so.

Therefore, we must change our thinking. Since we can’t surpass Python, then will use Python. For developers of scripting languages, they don’t really care how the low-level layer is implemented, as long as the high-level language and interface are familiar to me, so Boa is a Node.js library born for this, which bridges CPython to provide JavaScript with the ability to access the complete Python ecosystem, and in addition, with the help of ES6 new features to provide users with a seamless development experience, so what is the experience?

Let’s look at a simple example:

const boa = require('@pipcook/boa');  
const os = boa.import('os');  
console.log(os.getpid()); // prints the pid from python.  

// using keyword arguments namely \`kwargs\`  
os.makedirs('..', boa.kwargs({  
  mode: 0x777,  
  exist\_ok: false,  
}));  

// using bult-in functions  
const { range, len } = boa.builtins();  
const list = range(0, 10); // create a range array  
console.log(len(list)); // 10  
console.log(list\[2\]); // 2

Isn’t it simple enough? Just load the Python object through boa.import, and the remaining object access, function call, and array access are no different from our JavaScript.

const boa = require('@pipcook/boa');  
const tf = boa.import('tensorflow');  
const { layers, Model } = tf.keras;

class TestModel extends Model {  
  constructor() {  
    super();  
    this.conv1 = layers.Conv2D(32, 3, boa.kwargs({ activation: 'relu' }));  
    this.flatten = layers.Flatten();  
    this.d1 = layers.Dense(128, boa.kwargs({ activation: 'relu' }));  
    this.d2 = layers.Dense(10, boa.kwargs({ activation: 'softmax' }));  
  }  
  call(x) {  
    return this.conv1(x)  
      .flatten(x)  
      .d1(x).d2(x);  
  }  
}

The above example shows how to use TensorFlow to create a model, in addition to demonstrating how to inherit from a Python class from JavaScript. Is this very JavaScript style?

It is worth mentioning that, in the Boa internals, there is no encapsulation of TensorFlow and other frameworks, as long as you install the corresponding package through Python locally, it can be used like the above code, so in theory you can use any Python package does what is done above.

Next, we introduce some main methods separately.

builtins()

Python will build some common functions in its builtins, the specific API list is at: https://docs.python.org/3.7/library/functions.html, then Boa also provides corresponding way to use them:

const { len, list, range } = boa.builtins();

import(name)

In addition to the built-in methods, the most important function is to load Python packages, so import is to do this.

const np = boa.import('numpy');

kwargs(map)

Next is the keyword arguments. In Python, there is a way to use map to represent parameters, such as:

foobar(100, x=10, y=20)

It helps the caller understand the meaning of each parameter better. For this reason, the kwargs method has been added to Boa to support this usage:

foobar(100, boa.kwargs({ x: 10, y: 20 }));

with(ctx, fn)

“with” may be familiar to some people who are familiar with the history of JavaScript, but “with” in Python, its usage and purpose are not the same as JavaScript. The with-statement in Python is a bit similar to Block Scoping in JavaScript:

with(localcontext()) {  
  \# balabala  
}

The above code saves the state of localcontext(), then starts executing the block code in the with-statement, and finally, releases the state of localcontext().

The internal implementation mechanism is that each variable passed into the with-statement needs to implement two magic methods: __enter__ and __exit__, and then called before and after the block code execution.

eval(str)

The last to tell is to evaluate Python expressions (single line). Why should we provide such a method? This still has to talk about the advantages of Python. In some very complex data processing scenarios, Python expressions can still be expressed very simply and understandably, which greatly reduces the complexity of the code. Let’s take a look at an example:

const line = (boa.eval\`'\\t'.join(\[str(x) for x in ${vec}\])\`);

If the above code is to be replaced with JavaScript:

vec.map(x => x.toString()).join('\\t');

How much does it seem to be almost right? Then take a look at the following example:

boa.eval\`{u:i for i, u in enumerate(${vocab})}\`;  
boa.eval\`\[${char2idx}\[c\] for c in ${text}\]\`  
boa.eval\`${chunk}\[:-1\]\`  
boa.eval\`${chunk}\[0:-1:2\]\`

How about it, does it feel like the above example can’t be done with a simple line of JavaScript?

However, it is worth mentioning that JavaScript is gradually making up in this regard. there are some related standards that TC39 is doing, including the above Slice Notation.

Speaking of returning to eval, it is like a supplement to JavaScript. Before some standards have been implemented and stabilized, it allows us to use Python expressions to express more simply, and all we need is some low-cost learning. That’s it.

Next, let’s talk about how eval is used. It accepts a “string”, but we generally pass Template String when using it. Let’s look at two examples first:

boa.eval('print("foobar")');  
boa.eval(\`print("${txt}")\`);

After reading the above 2 lines of code, they are relatively rare usages. The most commonly used and most effective eval is the use of Tagged Template String. This usage is just like what we saw at the beginning. The content of the template string is directly followed by eval. The advantage of this is that the eval function will receive To all the template parameters so that we can open up JavaScript objects and Python expressions to achieve a smoother user experience, as follows:

const chunk = range(0, 10);  
boa.eval\`${chunk}\[0:-1:2\]\`;

The above is to transfer the chunk to the expression, and then get the corresponding value through the Slice Notation syntax of Python, and finally return to the world of JavaScript.

Summary

Well, the simple API introduction will come here first. If you want to learn more about API and Boa, you can go to our documentation: https://github.com/alibaba/pipcook/blob/master/docs/tutorials/want-to-use-python.md.

In addition, as a subproject of Pipcook, Boa is also very welcome to join. For those who want to join, you can use these Issues as a good start: https://github.com/alibaba/pipcook/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22.

Finally, let’s talk about the original intention of Boa, which is to allow Node.js developers to use Python’s rich machine learning ecosystem more seamlessly. It can be said that starting today, you can start looking at Python documentation and use JavaScript to “learn and use” machine learning and deep learning!