DEV Community

Piotr
Piotr

Posted on • Originally published at garden.piotrk.it on

How to (not) run Automerge in React-native

#reactnative #webassembly

TL;DR: I don’t know how how to run Automerge in RN, I’ve tried ⚛️ 🏎 react-native-webassembly but it’s not yet possible due to technical reasons.

There is hope. Callstack published new RN library callstackincubator/polygen which allows to run webassembly in RN, but I have yet to try it.

Problem

I wanted to create an app that has one big note, that is shared and synced between my devices. There are two layers to it:

  • Presentation: A single markdown formatted note, which can be done using https://github.com/Expensify/react-native-live-markdown
  • Storage and sync: There were no obvious answers here, I didn’t want to relay on cloud providers. I found a nice library: https://automerge.org/ which has a nice selling point:
    • Automatic merging: Automerge is a Conflict-Free Replicated Data Type (CRDT
    • Network-agnostic: Allows the use of any connection-orientated protocol
    • Portable: Imlemented in Javascript-WASM and Rust, with FFI bindings

Choosing the right tool for the job

This WASM part caught my eye since there are libraries that claim to bring WASM to react-native:

⚛️ 🏎 react-native-webassembly - A c++ turbo module that is a wrapper around wasm3 - A fast WebAssembly interpreter and the most universal WASM runtime.

callstackincubator/polygen - from the callstack website: “Polygen instead performs Ahead-of-time compilation of WebAssembly modules into C/C++ code using the wonderful wasm2c tool. After that, Polygen generates additional glue code so that the compiled WebAssembly module can be used from JavaScript code.”

So both projects are essentially wrappers around other libraries. I decided to go with the react-native-webassembly since it offered Android support, while polygen (as I’m writing it) iOS only.

react-native-webassembly

Of course adding this library to the project was not simple npm install react-native-webassembly failed to compile

I got somewhat cryptic error:

error: error in backend: failed to perform tail call elimination on a call site marked musttail
Enter fullscreen mode Exit fullscreen mode

Quick google led me to a simmilar issue on PowerPC, but I’m compiling for android… two searches later I got it: Musttail is causing crash when compiling wasm3 for android armv7-a so I read through the issue and I saw wasm3 being mentioned

The issue is with the configuration of wasm3, I fixed it the same way the @bald-man did, disabling musttail. Here’s the whole patch, feel free to take it:

diff --git a/node_modules/react-native-webassembly/cpp/m3_config_platforms.h b/node_modules/react-native-webassembly/cpp/m3_config_platforms.h
index 50b86ac..8deaf8d 100644
--- a/node_modules/react-native-webassembly/cpp/m3_config_platforms.h
+++ b/node_modules/react-native-webassembly/cpp/m3_config_platforms.h
@@ -82,7 +82,7 @@
 # endif
 # endif

-# if M3_COMPILER_HAS_ATTRIBUTE(musttail)
+# if M3_COMPILER_HAS_ATTRIBUTE(musttail) && !defined( __arm__ )
 # define M3_MUSTTAIL __attribute__ ((musttail))
 # else
 # define M3_MUSTTAIL
Enter fullscreen mode Exit fullscreen mode

Open questions

wasm

Before I proceed with the steps that I took, I need to explain what is WASM.

WASM - it’s a low level assembly language, that has both binary and text format. It runs the code in a sandboxed environment. That means that by default it does not have access to any Web APIs. Those are given by providing imports when loading WASM module.

Here’s the example WASM module in WAT (text) format:

(module
  (func $i (import "my_namespace" "imported_func") (param i32))
  (func (export "exported_func")
    i32.const 42
    call $i))
Enter fullscreen mode Exit fullscreen mode

You can convert any .wasm binary into such text format using: https://webassembly.github.io/wabt/demo/wasm2wat/

How react-native-webassemly loads WASM

In react-native-webassembly those imports can be provided like this:

const module = await WebAssembly.instantiate<{
  exported_func: () => number;
  // ...
}>(bufferSource, {
  // Define the scope of the import functions.
  my_namespace: {
    imported_func: (value: number) => console.error(value),
  },
});
Enter fullscreen mode Exit fullscreen mode

Those functions are then passed to wasm3::module by the c++ turbomodule somwehere in lines 367-411 but I don’t quite understand how he gets the callback to call the function he wants (I suspect it has to do something with _doSomethingWithFunction). My understanding goes like this:

Imports are passed as callback from the JS side to the RNWebassembly_instantiate function.

reactNativeWebAssembly.RNWebassembly_instantiate({
    iid,
    bufferSource: bufferSourceBase64,
    stackSizeInBytes,
    callback: ({ func, args, module }) => {
        ...
        return maybeFunction(...args.map(parseFloat));
    },
});
Enter fullscreen mode Exit fullscreen mode

Then on the native side the callback is passed to m3_LinkRawFunctionEx:

Function callback = params.getProperty(runtime, "callback").asObject(runtime).asFunction(runtime);

std::shared_ptr<facebook::jsi::Function> fn = std::make_shared<facebook::jsi::Function>(std::move(callback));

/* export initialization */
for (u32 i = 0; i < io_module->numFunctions; ++i) {
    ...
          m3_LinkRawFunctionEx(io_module, M3GetResolvedModuleName(f).data(), functionName->data(), signature.data(), &_doSomethingWithFunction, static_cast<void*>(id->data()));
Enter fullscreen mode Exit fullscreen mode

And the imported function which needs to be called is called using _doSomethingWithFunction:

m3ApiRawFunction(_doSomethingWithFunction)
{
    ...
    resultDict.setProperty(*context.rt, "module", facebook::jsi::String::createFromUtf8(*context.rt, std::string(_ctx->function->import.moduleUtf8)));
    resultDict.setProperty(*context.rt, "func", facebook::jsi::String::createFromUtf8(*context.rt, functionName));
    resultDict.setProperty(*context.rt, "args", ConvertStringArrayToJSIArray(*context.rt, result, length));

    Value callResult = originalFunction.call(*context.rt, resultDict);
    ...
}
Enter fullscreen mode Exit fullscreen mode

Loading automerge in RN

First try

This was my first attempt:

const AutomergeWasm = require('@/assets/automerge.wasm');
type AutomergeModuleType = typeof import("@automerge/automerge/slim");
const module = await WebAssembly.instantiate<AutomergeModuleType>(AutomergeWasm);
Enter fullscreen mode Exit fullscreen mode

Guess what’s wrong? I forgot imports 🫠, just for reference here’s the error that I’ve got:

error unknown value_type
Enter fullscreen mode Exit fullscreen mode

not telling me much…

Second try

Adding the imports was not that straightforward (or at least I did not know a better way). So I’ve just copied the web/automerge_wasm.jsand exported __wbg_get_imports which provides those imports for the web.

function __wbg_get_imports() {
    const imports = {};
    imports.wbg = {};
    imports.wbg.__wbindgen_object_drop_ref = function(arg0) {
        takeObject(arg0);
    };
    ...
Enter fullscreen mode Exit fullscreen mode

https://www.npmjs.com/package/@automerge/automerge-wasm

And… again…

error unknown value_type
Enter fullscreen mode Exit fullscreen mode

Side note: To check what imports are needed I put the .wasm file into https://webassembly.github.io/wabt/demo/wasm2wat/ and looked at the import statements, then a quick grep -r __wbindgen_object_drop_ref quickly found the missing functions.

  (import "wbg" "__wbindgen_string_get" (func $wasm_bindgen::__wbindgen_string_get::hbef6b8ade2155369 (type $t87)))
Enter fullscreen mode Exit fullscreen mode

Debugging time

How to debug this thing? I’ve decided to go straight into modifying the node_modules/react-native-webassembly/cpp/react-native-webassembly.cpp and then recompiling with npx expo run:android, which proved to work nicely.

I added logs and turns out that it didn’t even run the ‘import’ part, it failed on parsing the module:

wasm3::module mod = env.parse_module(buffer, decoded.length());
Enter fullscreen mode Exit fullscreen mode

So I went to google once again and found this issue: https://github.com/wasm3/wasm3/issues/352

Reference Types are not supported yet

Which is a bummer.

Question: What are reference types? And how do I know that the .wasm file has them?

WASM docs states:

Reference types classify first-class references to objects in the runtime store.reftype::=funcref ∣ externrefThe type funcref denotes the infinite union of all references to functions, regardless of their function types. The type externref denotes the infinite union of all references to objects owned by the embedder and that can be passed into WebAssembly under this type. Source: https://webassembly.github.io/reference-types/core/syntax/types.html#syntax-reftype

How to spot a reftype in the text format? Per the WASM spec:

So i went back to the https://webassembly.github.io/wabt/demo/wasm2wat/ loaded wasm and:

Bingo!

 (module $automerge_wasm.wasm
  (type $t0 (func))
  (type $t1 (func (result i32)))
  (type $t2 (func (result externref)))
  (type $t3 (func (param i32)))
  (type $t4 (func (param i32) (result i32)))
  (type $t5 (func (param i32) (result i32 i32)))
  (type $t6 (func (param i32) (result i32 i32 i32)))
  (type $t7 (func (param i32) (result i64)))
  (type $t8 (func (param i32) (result f64)))
  (type $t9 (func (param i32) (result externref)))
Enter fullscreen mode Exit fullscreen mode

Conclusions

When I started writing this I had no idea what the WASM was and how to load it into react-native. Even though I failed catastrophically this was a nice learning experience. I’m sharing it here as a future reference for myself and to turn those X hours of fighting to run this thing into something concrete.

In the future I’ll probably look into some other library like https://github.com/yjs/yjs which also offers CRDTs but this time there are some people which got it to work in react-native https://github.com/yjs/yjs/issues/381 which may be easier and I might actually get some work done 😅

👀 turns out the reference-types are not supported everywhere and at some point you had to explicitly enable them in wasm-bindgen.

Top comments (0)