DEV Community: Nicolas Rannou

WASM in Create-React-App 4 in 5mn (without ejecting)

Nicolas Rannou — Tue, 22 Dec 2020 15:24:37 +0000

First word

In this post, we will write some rust code, compile it to WASM and run it from a CRA4 application without ejecting.

Rust to WASM FTW

Why Rust? It's cool. There is a lot of hype around it so I decided to give it a go. The rust book and rust by examples were pretty useful to get an understanding of the basics.

Once I got my head around the basic concepts, time to look into the conversion to WASM. There is a lot of nice documentation out there (from the Rust official documentation of course ;)).

The recommendation is to use wasm-pack. That allows you to build and ship an importable WASM module in less than 2mn.

Create the project

$> wasm-pack new my-wasm-project
$> cd my-wasm-project

Package and public (details)

$> wasm-pack build --scope nicolasrannou
$> cd pkg
/*make sure the package.json includes the right files. At the time I write this post some .js and .wasm files were missing.*/
$> npm publish --access=public

CRACO

To load the exported module I chose CRACO because it appears Create-react-app-rewired is not really maintained anymore and doesn't work well with CRA4.

Adding dependencies in your CRA4 project

$> cd my-cra4-project
$> yarn add @nicolasrannou/my-wasm-project
$> yarn add @craco/craco
$> yarn add wasm-loader

Do not forget to update package.json by replacing react-scripts by craco.

Setup the config file

$> cat craco.config.js

const { addBeforeLoader, loaderByName } = require('@craco/craco');

module.exports = {
  webpack: {
    configure: (webpackConfig) => {
      const wasmExtensionRegExp = /\.wasm$/;
      webpackConfig.resolve.extensions.push('.wasm');

      webpackConfig.module.rules.forEach((rule) => {
        (rule.oneOf || []).forEach((oneOf) => {
          if (oneOf.loader && oneOf.loader.indexOf('file-loader') >= 0) {
            oneOf.exclude.push(wasmExtensionRegExp);
          }
        });
      });

      const wasmLoader = {
        test: /\.wasm$/,
        exclude: /node_modules/,
        loaders: ['wasm-loader'],
      };

      addBeforeLoader(webpackConfig, loaderByName('file-loader'), wasmLoader);

      return webpackConfig;
    },
  },
};

Import the WASM code

$> cat App.tsx
...
  useEffect(async () => {
    const promise = await import("@nicolasrannou/my-wasm-project");

    promise.greet();
  }, []);
...

👋 Until next time!

Create textures from data in ThreeJS

Nicolas Rannou — Fri, 17 Apr 2020 11:48:03 +0000

I've been looking into creating a texture from data in three.js. It is super easy, but there are some caveats, and some parts can be confusing. I fell into some traps many years ago, then fall into it again recently, so I decided to write about it!

What is confusing (me)?
Grayscale textures
- Which format
- Luminance format allows 3 types (WebGL1)
- Type to TypedArray containing the data
- Access the data in the fragment shader
Gimme some concrete examples!
- UnsignedByte Texture
- HalfFloat Texture
- Float Texture
Until next time!

Table of contents generated with markdown-toc

What is confusing (me)?

When creating a new texture, from data, you must set a Format, a Type, and provide you data in a specific type of TypedArray.

const texture = new DataTexture(data, width, height, format, type, ...);

The doc says

"type" must correspond to the "format".

Ok... so how do I know which Type to set for the Format I want to use?

Grayscale textures

In this post, I will only discuss grayscale (single channel) texture for WebGL1 since it is my current focus. Everything in the rest of this post will apply to whatever you are trying to support.

Which format

Alright, I want to create a single channel (grayscale) texture.
In the 'internal format' section, you can find which Format is the best fit for you, depending on the number of channels and the bytes per pixel.

LUMINANCE is the natural fit for grayscale textures. The caveat is that it only supports UnsignedByte texture type for Webgl1.

If you work with WebGL2, R* formats allow you to support a variety of different bit-depth.

To know more about what the different file format means, I found this page useful: https://www.khronos.org/opengl/wiki/Image_Format. It explains what the suffix of the file format means *F, *_SNORM means and how those type of textures are interpreted. That is important regarding the data normalization. (keep reading)

Luminance format allows three types (WebGL1)

Alright, we just learned that for WebGL1, Luminance format goes with UnsignedByte type.

Can we do better?

If your browser supports the OES_texture_float extension, a bunch of new types (Float and HalfFloat) are available for the LUMINANCE format. (official documentation)

Format	Type	Byte per Pixel
RGBA	FLOAT	16
RGB	FLOAT	12
LUMINANCE_ALPHA	FLOAT	8
LUMINANCE	FLOAT	4
ALPHA	FLOAT	4
RGBA	HALF_FLOAT_OES	8
RGB	HALF_FLOAT_OES	6
LUMINANCE_ALPHA	HALF_FLOAT_OES	4
LUMINANCE	HALF_FLOAT_OES	2
ALPHA	HALF_FLOAT_OES	2

Type to TypedArray containing the data

It is pretty straight forward:

Type	Byte per Pixel	Typed Array
UnsignedByte	1	Uint8Array
HalfFloat	2	Uint16Array
Float	4	Float32Array

What is important there is that the number of bits in the type array matches the byte per pixel in the table. Also, for HalfFloat the data should be prepared appropriately.

Access the data in the fragment shader

All the integer textures (including UnsignedByteType) are normalized automatically while uploaded to the shaders, whereas the floating/integral textures (including Float and HalfFloat) are passed as it is.

Based on the Format name, you can know with which type of data you are dealing with and whether that will be normalized for you or not. (ref).

In other words, in the fragment shader, when using an UnsignedByteType texture, the values you get from the texture 2D are normalized between 0 and 1 automatically. For FloatType and HalfFloatType you get the value that was in the typed array without any normalization.

Gimme some concrete examples!

UnsignedByte Texture

const textureSize = 16
const dataSize = 10;
const data = new Uint8Array(dataSize);

for (let i = 0; i < dataSize) {
  data[i] = Math.round(Math.random() * 255); // pass anything from 0 to 255
}

const texture = new DataTexture(data, textureSize, textureSize, LUMINACE, UnsignedByteType);

varying vec2 vUv;
uniform sampler2D uData;

void main(){
  vec3 color;
  vec4 data = texture2D( uData, vUv );
  gl_FragColor = vec4( data.xyz, 1.0 );
}

HalfFloat Texture

To convert a number to half float, do it the three.js way: like this

⚠️ Watch out for precision errors when converting numbers to half float precision!

const textureSize = 16
const dataSize = 10;
const data = new Uint16Array(dataSize);

for (let i = 0; i < dataSize) {
  const largeNumber = Math.random() * 10000; // pass anything from 0 to 10000
  data[i] = toHalfFloat(largeNumber);
}

const texture = new DataTexture(data, textureSize, textureSize, LUMINACE, HalfFloatType);

varying vec2 vUv;
uniform sampler2D uData;
uniform float uMax;
uniform float uMin;

void main(){
  vec3 color;
  vec4 data = texture2D( uData, vUv );
  vec4 normalizedData = (data - uMin) / (uMax - uMin);
  gl_FragColor = vec4( data.xyz, 1.0 );
}

Float Texture

const textureSize = 16
const dataSize = 10;
const data = new Float32Array(dataSize);

for (let i = 0; i < dataSize) {
  const largeNumber = Math.random() * 10000; // pass anything from 0 to 10000
  data[i] = largeNumber;
}

const texture = new DataTexture(data, textureSize, textureSize, LUMINACE, FloatType);

varying vec2 vUv;
uniform sampler2D uData;
uniform float uMax;
uniform float uMin;

void main(){
  vec3 color;
  vec4 data = texture2D( uData, vUv );
  vec4 normalizedData = (data - uMin) / (uMax - uMin);
  gl_FragColor = vec4( data.xyz, 1.0 );
}

Until next time!

Until next time 🙋‍♂️, happy coding!

Web-workers in Create React App (CRA) without unmounting

Nicolas Rannou — Thu, 26 Mar 2020 16:31:02 +0000

There are a lot of great options available when it comes to using web-workers in your web-app. Unfortunately, it can be challenging to use in a Create React Application (CRA) without un-mounting your application.

In this article, we share my journey of using web-workers in CRA (without un-mounting!)

Why not unmounting?
Finding the right tools for YOU
- Attempt 1: Threads.js
- Attempt 2: Workarize
- Attempt 3: Worker-loader
- Attempt 4: Comlink
- How it looks
That's it

Table of contents generated with markdown-toc

Why not unmounting?

Webpack configuration can be overwhelming
It can be difficult to upgrade to the next version of react-scripts
CRA team will not help us if something goes side-ways

Finding the right tools for YOU

The right tool for me would check the items in the following list:

Must: Not unmounting CRA
Must: Support transferable objects
Nice to have: Easy to use
Nice to have: Typescript support
Nice to have: Pooling system

Attempt 1: Threads.js

Threads.js looked very promising and feature complete (according to my requirements).

However, to use it within webpack, you must use its companion loader (threads-plugin).

That means you must access the webpack configuration file and add the loader there. That was a no-go for me because it implies un-mounting CRA.

Attempt 2: Workarize

The next step was to find a tool that leverages webpack loaders rather than plugins. With webpack loaders, we can inline the loader in the import statement, therefore, we do not need to modify the webpack configuration file to use it.

\\ inline the loader in the import statement
import {Something} from 'my-loader!./utils/filename'

workarize and its workarize-loader seemed pretty cool. Although it doesn't support pooling the API looked awesome and super intuitive.

But it doesn't support transferable objects at the time I write this post (03-26-2020), so that was another no-go.

Attempt 3: Worker-loader

Next, I tested worker-loader and its transferable objects.

It worked fine but using web-workers without sugar on top just didn't feel right for me - we can do better!

Attempt 4: Comlink

Then comes comlink. It "makes WebWorkers enjoyable again" (to quote their README).

comlink works nicely with the worker-loader and adds this nice touch on top of the web-workers to make it easy to use.

It may provide features you do not need but works fine in my experience (so far 🤞).

It also works well with Typescript.

It doesn't support pooling but I can live with that!

How it looks

\\ main.ts
import * as Comlink from 'comlink';
/* eslint-disable import/no-webpack-loader-syntax */
import Worker from 'worker-loader!../util/worker';

async function init() {
  const worker = new Worker();
  const obj = Comlink.wrap(worker);
  await obj.inc();
}
init();

\\ worker.ts
import * as Comlink from 'comlink';

const obj = {
  counter: 0,
  inc() {
    this.counter++;
  }
};

Comlink.expose(obj);

That's it

Each tool that was tested is great and you should really consider what you are looking for before making an informed decision.

I can live without pooling as long as it is enjoyable to work with the web-worker so that's why I chose comlink.

TLDR:

✅Must: Not unmounting CRA
✅Must: Support transferable objects
✅Nice to have: Easy to use
✅Nice to have: Typescript support
🔴Nice to have: Pooling system

Thanks for reading!

Save lives with WebGL

Nicolas Rannou — Fri, 10 Aug 2018 10:21:25 +0000

*Images are courtesy of [1] - Evelyn DappaKai HigashigaitoJürgen FornaroSebastian LeschkaSimon WildermuthHatem Alkadhi - Cinematic rendering – an alternative to volume rendering for 3D computed tomography imaging

I'm looking for WebGL / OpenGL / Passionate folks who could help me implement cinematic volume rendering in WebGL.

This project aims to provide real life like visualization of medical data.

[Left cinematic rendering / right volume rendering - image from [1]]

I've implemented some classic volume rendering and I'd like to take it to the next level. However I think my volume rendering implementation is already a bit clumsy so cinematic rendering may be too much of a challenge.

Currently at https://github.com/FNNDSC/ami:

Any directions, help, suggestion would be awesome!

Refs:

[1] - Cinematic rendering – an alternative to volume rendering for 3D computed tomography imaging - https://link.springer.com/article/10.1007/s13244-016-0518-1