Rendering an Angular Application in Minecraft

This is an English translation of my article originally published in Russian on Habr: Rendering an Angular Application in Minecraft.

The idea behind this experiment is to render an Angular application in Minecraft. Building a redstone computer capable of running Node.js could take half a lifetime, so the application runs externally in Node.js, with Minecraft acting as its screen.

The experiment involves building a weather application that can be controlled from inside the game world. Behind the interface are ordinary Angular building blocks.

To make this work, we need to connect three things: Angular's operations on elements, layout calculations, and Minecraft commands. Let's start by choosing how to draw the interface.

Drawing in Minecraft with RCON and block_display

RCON is the most convenient option for programmatic drawing. However, it requires running the world on a server with access to its internal settings. We do this locally by downloading the server from the official website. RCON is an interface for executing console commands remotely. Using the npm library rcon-client, the application sends commands from Node.js to the game world.

The first option is obvious: treat each block as a pixel. The setblock command places a single block, while fill fills a rectangular region. You can build a screen this way, and the project supports this mode too. But even a 200 × 100 pixel image requires 20,000 block positions. Updating the interface means changing regions of the world, and each pixel is limited to the size of a whole block. This is a fairly expensive operation, yet a screen with that resolution is still too small. That is why we will use block_display.

This entity displays a block model and lets you change its size and position. These entities have been available since Minecraft Java Edition 1.19.4. It is a virtual block: visible in the world, but without physical interactions. We can turn a block model into a thin rectangle and place it anywhere on the screen.

The configuration uses pixelsPerBlock: 4: four pixels horizontally and four vertically across the area of one block face. That gives us 16 pixels instead of one. An 880 × 550 pixel screen occupies 220 × 137.5 blocks. This resolution is sufficient to display and comfortably use a small application. Now we need to work out the rendering itself.

Angular rendering and a custom platform

In a typical browser application, an Angular component contains data and a template describing the interface. Compilation turns that template into Angular instructions. When the view is created, these instructions create elements and text nodes, connect them, set attributes, and register event handlers.

When component state changes, Angular evaluates bindings during change detection and updates those whose values differ from the previous ones. For example, a new temperature value updates the text in the corresponding node. There is no need to recreate the entire interface.

Node operations go through a renderer. In the browser implementation, they modify the DOM, after which the browser calculates element positions and draws the result. Angular manages the view structure and bindings; the browser handles geometry and pixels. To output to Minecraft, we need to replace that last part: store our own nodes instead of DOM nodes, calculate their layout with Yoga, and display the result as entities in the game world.

Angular lets us create custom platforms and renderer implementations. A platform assembles the dependencies for a particular environment, while the renderer determines what happens when an element is created, text changes, or a style is set.

Let's call the platform platformMinecraft. It is created with createPlatformFactory, using platformCore as its base. At startup, it receives the RCON connection, screen parameters, and font settings:

platformMinecraft([
  { provide: MC_RCON, useValue: rcon },
  { provide: MC_SCREEN, useValue: screen },
  { provide: MC_FONT, useValue: font },
]).bootstrapModule(AppModule);

Internally, it registers MinecraftRendererFactory, which returns the Minecraft renderer. This implements Renderer2: createElement, createText, appendChild, setStyle, setValue, and the other methods Angular uses to work with interface nodes.

Here is a simple template:

<div style="background: black_concrete; color: white_wool">
  {{ temperature() }}
</div>

When the view is created, the renderer receives operations to create the element and text, connect them, and set attributes. When temperature() changes, Angular updates the text through setValue. The Minecraft renderer turns these operations into an image.

To do this, it maintains a separate tree of McNode objects. Each node has a parent, children, attributes, and styles. Text nodes contain a string; image nodes contain a decoded image. Angular calls renderer methods, and those methods modify this tree. For example, adding a child node looks like this:

appendChild(parent: McNode, child: McNode): void {
  if (!parent || !child) return;
  this.detach(child);
  child.parent = parent;
  parent.children.push(child);
  parent.yogaNode.insertChild(
    child.yogaNode,
    parent.yogaNode.getChildCount(),
  );
  this.markDirty();
}

The child is added to two trees at once: the McNode element tree and the Yoga tree, which handles sizes and positions. Finally, the scene is marked as needing an update.

Integrating Yoga and calculating layout

Creating a tree is not enough. We need to know where a card starts, how much space its heading takes, and where to put a button after the text. In a browser, the layout engine handles this. On the Minecraft platform, Yoga will do that work.

Yoga calculates rectangle sizes and coordinates using rules based primarily on Flexbox. The engine itself draws nothing. It receives an element structure and supported styles, and returns geometry. For each node, we can get its position relative to its parent and its size, then convert them into screen coordinates { x, y, w, h } for drawing.

For example, a container 400 pixels wide and 120 pixels high arranges two elements in a row. It has 16 pixels of padding on all sides and a 12 pixel gap between elements. The first element has a width of 120 and a height of 80 pixels. The second has flex: 1 and stretches vertically because its parent uses align-items: stretch. After calculateLayout(), calling getComputedLayout() on each child gives the following values. These are the four result fields we need for drawing:

// First element
{ left: 16, top: 16, width: 120, height: 80 }

// Second element
{ left: 148, top: 16, width: 236, height: 88 }

The second element starts after the left padding, the first element, and the gap: 16 + 120 + 12 = 148. Its width fills the remaining space: 400 − 16 − 16 − 120 − 12 = 236. Its height is the available height between the padding: 120 − 16 − 16 = 88.

If the container itself is at { x: 32, y: 24 } on the screen, the renderer adds this offset and gets:

// Rectangles in screen coordinates
// [{ x: 48, y: 40, w: 120, h: 80 }, { x: 180, y: 40, w: 236, h: 88 }]

These are not Minecraft block coordinates yet. Yoga calculates layout in interface units, which we treat as pixels here. The renderer converts them into world coordinates using the screen's scale.

Connecting renderer elements to Yoga nodes

Yoga is integrated directly into the custom renderer. Creating an McNode also creates its corresponding Yoga node:

import Yoga from 'yoga-layout';

function createNode(type: McNode['type'], tag?: string): McNode {
  return {
    type,
    tag,
    children: [],
    attrs: {},
    parent: null,
    yogaNode: Yoga.Node.create(),
    style: {},
  };
}

The renderer's createElement and createText methods use this function. When adding a child, the appendChild method shown above calls parent.yogaNode.insertChild(...). Removal uses the Yoga node's removeChild. This keeps the layout structure synchronized with the tree managed by Angular.

Converting styles into Yoga parameters

For example, a template can define this layout:

<div style="flex-direction: row; padding: 16px; gap: 12px">
  <div style="width: 120px; height: 80px; background: blue_concrete"></div>
  <div style="flex: 1; background: gray_concrete"></div>
</div>

The first element gets a width of 120 pixels. The second takes the remaining space. Yoga accounts for the padding and the gap between them. The renderer translates style properties into calls such as setWidth, setPadding, and setFlex.

For dynamic styles, Angular calls setStyle. In the renderer, this method passes the value to applyStyle and marks the scene as needing an update:

setStyle(
  el: McNode,
  style: string,
  value: any,
  flags?: RendererStyleFlags2,
): void {
  applyStyle(el, style, value);
  this.markDirty();
}

Inside applyStyle, property names are normalized to camelCase, and size values are parsed by toLen. Here is part of the property handling:

const y = el.yogaNode;
switch (name) {
  case 'width': return y.setWidth(toLen(str));
  case 'height': return y.setHeight(toLen(str));
  case 'flex': return y.setFlex(toLen(str) as number | undefined);
  case 'gap': return void y.setGap(Gutter.All, toLen(str));
}

A static style="..." attribute arrives through setAttribute. Its contents are split into individual declarations, each of which is also passed to applyStyle. A static width and a [style.width] binding therefore end up changing the same Yoga node. Colors are handled separately: background and color are stored in McNode for painting and do not participate in layout calculation.

Recalculating layout when the view changes

The renderer factory sets the root Yoga node's width and height to the screen dimensions. The markDirty callback passed to MinecraftRenderer sets the factory's dirty flag, indicating that the scene needs updating.

After a view update pass, Angular calls RendererFactory2.end(). The Minecraft implementation schedules recalculation here:

end(): void {
  if (!this.dirty || this.scheduled) return;
  this.scheduled = true;
  setTimeout(() => {
    this.scheduled = false;
    this.dirty = false;
    this.flush();
  }, 0);
}

This combines several element changes into one scheduled flush() call. It starts by running Yoga's layout calculation:

this.root.yogaNode.calculateLayout(
  this.screen.width,
  this.screen.height,
  Direction.LTR,
);

Then collectPaint traverses the tree and reads each visible node's computed layout. Yoga coordinates are relative to the parent, so accumulated offsets are added:

const l = node.yogaNode.getComputedLayout();
const x = offX + l.left;
const y = offY + l.top;
const w = Math.round(l.width);
const h = Math.round(l.height);

These coordinates are used to assemble background rectangles and text and image pixels. The resulting collection is passed to MinecraftPainter, which builds the Minecraft commands.

Figure: Rendering the layout.

Measuring text and images

Text and images present an additional problem. Yoga does not know how much space a line occupies in the selected font, or the original dimensions of a PNG. For these elements, we provide a measure function. The engine passes constraints to it, and it returns the content's dimensions.

For an image with only its width specified, the height can be derived from the original aspect ratio. For text, we need to measure characters and calculate line wrapping. If a label gets longer, its height may change and move the elements below it. That is why changing text also marks its Yoga node with markDirty(), so its dimensions are recalculated.

There are two separate change indicators here: the factory's dirty flag schedules a screen update, while yogaNode.markDirty() tells Yoga that the content needs measuring again. When text changes, the renderer sets both:

setValue(node: McNode, value: string): void {
  node.value = value;
  if (node.type === 'text') node.yogaNode.markDirty();
  this.markDirty();
}

After layout, we know the sizes and coordinates of the element rectangles. Filling their backgrounds is straightforward. Now we need to fill them with content.

Rasterizing text

We could draw text on a canvas, read its pixels, and transfer them to Minecraft. The current implementation uses opentype.js without a canvas.

The library reads TTF and OTF files and provides access to glyph outlines—the graphical shapes of characters. The rasterizer approximates the outline's curves with line segments and checks which pixels fall inside each letter.

A single check at the center of a pixel is not enough for thin strokes: the result depends too much on how the outline aligns with the grid. Instead, we test nine points inside each pixel, arranged in a 3 × 3 grid. If the proportion of points inside the outline reaches a threshold, the pixel is filled.

This is supersampling. Here, it helps determine the shape of a letter on a coarse grid. Each resulting pixel is either filled or omitted; there is no semitransparent text antialiasing.

Character dimensions are also used during layout. Lines wrap at word boundaries, and words that are too long are split into individual characters. Outlines and rasterized glyphs are cached so that the same letter at the same size does not need to be calculated again.

The output is a list of pixel coordinates. Their color comes from the color property, while font-family selects the font file from the families registered in the application.

Figure: Rendering text.

Processing images and matching colors

Images are simpler: the pixels are already in the file. PNG and JPEG files are decoded using pngjs and jpeg-js, then resized to the dimensions calculated by Yoga. Downscaling averages the colors of the source region while accounting for transparency; upscaling selects a source pixel. Pixels whose alpha is below the threshold are skipped. The finished result is cached.

Next, we need to choose a block for each color. The palette includes concrete, wool, and terracotta. Each material has an approximate average texture color. The renderer compares it with the desired RGB value using the weighted redmean distance and selects the closest match. This handles both image pixels and style colors such as #a144ff.

If you want a specific material, you can write background: black_concrete or color: white_wool directly. Exact matches to browser colors are not guaranteed: the material palette is limited, and blocks retain their textures. Still, even the Angular logo ends up assembled from Minecraft materials.

Figure: Rendering images.

Optimizing drawing and entity updates

At this point, the renderer knows where the backgrounds and the text and image pixels belong. Sending a separate command for every pixel would make the screen too resource intensive.

A container's background is therefore drawn as one rectangle from the start. Text and image pixels are first merged into horizontal runs of the same material. If runs on adjacent rows have the same position, width, and depth, they are merged vertically. For example, a solid vertical stroke in a letter can become a single elongated entity.

Such a rectangle requires just one summon minecraft:block_display command with the appropriate scale, material, and offset. To keep text visible above the background, layers are separated slightly in depth. The deeper an element is nested in the tree, the closer its image is to the viewer.

The next optimization concerns updates. The renderer stores the previous collection of rectangles and compares it with the new one. Matching fragments stay in place, new ones are created with summon, and fragments that disappeared are removed with kill. The comparison includes coordinates, dimensions, material, and depth.

If the temperature changes, we do not need to recreate the background or the other unchanged parts of the interface. If the whole image remains unchanged, no drawing commands are sent. Recalculation is triggered after Angular changes through RendererFactory2.end(): first Yoga updates the geometry, then the renderer assembles the rectangles and sends the difference to Minecraft.

Handling player actions and UI events

Renderer2 provides listen() for events. Angular uses it to register handlers that run in response to the corresponding player action. Events are triggered by approaching an element. Every 500 ms, the application queries player positions through RCON.

The player's position, adjusted for eye height, is projected onto the screen plane. The coordinates calculated by Yoga are then used to identify the element under that point. If the player is close enough and enters a new element's region, mouseenter and click are triggered. Leaving the region triggers mouseleave. To press the same button again, the player must leave its region and return.

Neither the direction the player is looking nor mouse button presses are taken into account. The screen configuration allows a distance of up to 40 blocks from its plane. This means the interface is controlled by player movement, so the buttons are made large.

The click event supports bubbling through the tree and stopPropagation(). The handler runs inside the Angular zone so that state changes trigger an application update.

From the component's perspective, everything is familiar: (mouseenter) enables highlighting, (mouseleave) removes it, and (click) invokes a handler. The button notifies its parent that it has been pressed, the parent changes the selected day, and Angular updates the text and styles. The next forecast appears in Minecraft.

Fixing misaligned text fragments

Testing the renderer in the game client revealed a problem: text looked aligned in the local PNG preview, but individual fragments of small letters were displaced in Minecraft. The strokes did not join up.

Initially, each entity was created at its own center using fractional world coordinates. The model itself was shifted back by half its size. The calculations placed the boundaries of adjacent rectangles together, but the game client showed discrepancies.

Changing the coordinate system solved the problem. All screen entities are now created at a single integer-coordinate point near the center of the screen. Each rectangle's position is specified as a local translation relative to that common point.

The screen is far from the world's origin, around X = -24535. Coordinate precision was considered during the investigation. After switching to a shared origin, the misalignment disappeared, as verified in the game.

Renderer capabilities and limitations

The resulting renderer supports Flexbox through Yoga, dimensions and spacing, rectangular fills, text, PNG/JPEG images, and player proximity events. This is enough for the weather application.

CSS classes are not processed, so styles are defined inline and through Angular bindings. There are no shadows, rounded corners, or complete browser layout rules. Components that access the DOM directly need adaptation.

Update speed depends on the number of changed rectangles, communication over RCON, and the client's entity rendering. Without comparative performance measurements, we cannot claim that this approach is always faster than drawing with real blocks.

The renderer lets an Angular component load weather data, respond to player actions, and update an interface in the game world. To switch to tomorrow's forecast, you walk up to the “Next” button.

Automatically rebuilding and restarting the application

For development, we add an automatic update mode:

npm run dev

The dev.mjs script watches files in src. When a template or TypeScript file is saved, it starts a build and, if that succeeds, restarts the application. On startup, old screen entities are removed and the interface is drawn again. There is no need to run the build manually or enter commands in Minecraft.

For example, you can change font-size: 28px to font-size: 36px in the weather metric component and save the file. After the build and restart, Yoga calculates the new dimensions, and the text on the in-game screen gets larger. You can also change spacing, colors, and element positions, then evaluate the result in Minecraft as soon as the update completes.

The workflow resembles hot reload, but technically it uses a rebuild and a full restart of the Node.js process. Application state is not preserved: the selected day resets and the forecast is loaded again. The update takes as long as building, starting, and drawing require. If the build fails, the previous version of the application keeps running.

Architecture overview

The application runs in Node.js on platformMinecraft. Angular creates the view and updates bindings through a custom Renderer2. The renderer maintains the McNode tree, passes its structure and styles to Yoga, and recalculates sizes and coordinates after changes.

This geometry is used to assemble the image: backgrounds become rectangles, text is rasterized from font outlines, and PNG and JPEG images are scaled to the required sizes. Colors are replaced with Minecraft materials. Pixels of the same color are merged, the new collection of rectangles is compared with the previous one, and RCON commands are sent only for fragments that changed. In the world, the result is displayed using block_display entities with fractional scaling and a shared origin.

The return path starts by polling player coordinates. MinecraftInput projects them onto the screen, checks whether they fall within element boundaries, and invokes handlers registered through listen(). Those handlers change Angular state, starting the rendering cycle again. The interface therefore responds both to new weather data and to player actions.

UI support is limited to the selected set of styles and events. Yoga handles layout, while the renderer provides the drawing and input that a browser would supply in a normal web application.

The weather forecast application

Built on this platform, the application shows a seven-day weather forecast for Dubna. Its 880 × 550 pixel screen contains the date, a weather icon, the selected day's maximum and minimum temperatures, maximum wind speed, and probability of precipitation. Two buttons at the bottom switch between days.

For example, the application initially shows today's forecast. The player approaches the forward button's region, the button is highlighted, and the application switches to tomorrow. The date, metrics, and weather icon change. The player can continue through the forecast or go back. On the first and last days, the corresponding button is disabled and changes color.

This is the feedback loop: the player's position in Minecraft becomes an Angular event. The handler changes application state, and the renderer returns the result to the game world. A click is triggered when the player enters the button's region; pressing the mouse button is unnecessary. The mechanism behind these events is described in the section on handling player actions.

Data comes from Open-Meteo at startup and refreshes every 15 minutes. A loading state is shown during the initial request. If the connection fails, the application keeps the last forecast on screen, shows an error indicator, and retries after one minute.

In the code, the interface is divided into components: a heading, a weather summary, individual metrics, and navigation. A service handles the data, and the buttons send events to their parent. This application exercises text and image rendering, binding updates, and player interaction.

Angular component examples

For example, an individual weather metric is an ordinary component with two input properties. It contains neither Minecraft commands nor Yoga calls:

import { Component, Input } from '@angular/core';

@Component({
  selector: 'weather-metric',
  host: { style: 'flex: 1; gap: 6px; justify-content: center' },
  template: `
    <div style="font-size: 16px; color: white_concrete">{{ label }}</div>
    <div style="font-size: 28px; color: white_wool">{{ value }}</div>
  `,
})
export class WeatherMetricComponent {
  @Input() label = '';
  @Input() value = '';
}

The parent supplies the label and value, and the component defines their appearance. The platform's distinctive feature is visible in the styles: supported inline properties are used instead of CSS classes, and a color can be specified as a block name.

The root template connects the summary and navigation as follows. This page fragment omits the heading and the loading and error states:

@if (day(); as forecast) {
  <weather-summary [day]="forecast"></weather-summary>
}
<weather-day-navigation
  [index]="index()"
  [count]="weather.days().length"
  (step)="move($event)">
</weather-day-navigation>

Navigation receives the selected day's index and the forecast length. Its buttons emit step.emit(-1) or step.emit(1). The parent component handles the event with this method:

move(delta: number): void {
  const next = this.weather.days()[this.index() + delta];
  if (next) this.selectedDate.set(next.date);
}

selectedDate is stored in a signal, while index and day are derived using computed. Changing the selected date updates the data passed to the summary and the binding values in its template.

The button itself is also an Angular component. Below is a shortened version of McButtonComponent: it uses fixed color values, while event handling remains unchanged.

import { Component, EventEmitter, Input, Output, signal } from '@angular/core';
import { MinecraftUiEvent } from '../minecraft/minecraft-platform';

@Component({
  selector: 'mc-button',
  template: '<ng-content></ng-content>',
  host: {
    'style': 'padding: 10px 18px; font-size: 18px',
    '[style.background]':
      'disabled ? "gray_concrete" : hovered() ? "cyan_concrete" : "black_concrete"',
    '[style.color]': 'disabled ? "light_gray_wool" : "white_wool"',
    '(click)': 'onClick($any($event))',
    '(mouseenter)': 'hovered.set(true)',
    '(mouseleave)': 'hovered.set(false)',
  },
})
export class McButtonComponent {
  @Input() disabled = false;
  @Output() pressed = new EventEmitter<MinecraftUiEvent>();
  hovered = signal(false);

  onClick(event: MinecraftUiEvent): void {
    event.stopPropagation();
    if (this.disabled) return;
    this.pressed.emit(event);
  }
}

When the player enters the element's region, hovered changes, updating the background color through its binding. The click event invokes onClick, and pressed notifies the parent of the press. A disabled button does not emit pressed. All player position detection stays inside MinecraftInput, so the component works with ready-made events.

Conclusion

A custom platform and Renderer2 implementation let us display an Angular application in Minecraft while retaining components, bindings, and event handlers. The application runs in Node.js, Yoga calculates element layout, and the renderer turns the interface into commands for block_display.

The weather forecast demonstrates a complete interaction cycle: service data appears on the in-game screen, player movement triggers button events, and changes to Angular state update the image. Automatic rebuilding lets us test template and component changes directly in Minecraft.

This kind of output requires implementing geometry calculations, rasterization, and input handling ourselves. Compatibility is therefore limited to the renderer's supported styles and capabilities. Meanwhile, the weather application's logic remains at the level of Angular components, which do not need to send Minecraft commands or calculate world coordinates.

I'm really glad it worked! The application knows nothing about Minecraft, yet it runs successfully!

You can see the result and watch the forecast being switched inside the game world in this YouTube video.

Comments

[Raknaos]

Implementing RendererFactory2 against a block grid instead of a DOM makes the abstraction look obvious in hindsight - createElement, setStyle, end() as the flush point. The detail I liked most is the two separate dirty flags doing different jobs: the factory's flag schedules a screen update, yogaNode.markDirty() tells Yoga to measure again. Confusing them would either redraw a stale layout or re-measure every frame.

Nine sample points per pixel for glyph coverage is good thrift. Was text wrapping what hurt once the forecast list grew, or is the real bottleneck command throughput between the Node side and the game?