WASM instead of C Dependencies?

I have web applications written in Rust and Go that need some basic image processing (reading JPEGs, PNGs, writing JPEGs, WebPs, AVIFs and resizing). This is something I always struggle with, because most libraries for image processing are written in C (libpng, libwebp, mozjpeg; or higher-level ones like vips). While there are usually dependencies in each language build on top of those C dependencies, like bimg for Go, I don’t like having C dependencies in a Rust, Go or even Node.js projects.

Pain during builds

Let’s take bimg as an example (just and example, I had the same experience with similar dependencies in Rust, Go and Node.js):

go get -u github.com/h2non/bimg

Trying to get it fails on my system with:

# pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found

So I have to get vips first:

brew install vips

Now I have a dependency that is not managed by the Go toolchain as the rest of my code. Installing it is different from OS to OS.

And does it work now? Nope.

% go get -u github.com/h2non/bimg
go build github.com/h2non/bimg: invalid flag in pkg-config --cflags: -Xpreprocessor

After allowing the cflag, I can finally get the dependency:

env CGO_CFLAGS_ALLOW="-Xpreprocessor" go get -u github.com/h2non/bimg

This is not the developer experience I am aiming for. Not for myself, not for my future self who forgot how it worked, and not for other devs that have to work with my projects.

Pain during deployments

This is highly subjective, but I really want my Dockerfiles to either be FROM scratch or FROM gcr.io/distroless/static (GoogleContainerTools/distroless). To qualify, my applications must compile to static binaries and must not require libc. The worst case I’d be fine with is FROM gcr.io/distroless/base (static compiled, but libc is fine).

If I want to build a minimal Docker image with a program that as an example depends on vips, I'd have to add a long list of shared objects to the image:

/opt/vips/lib/libvips.so.42
/usr/lib/libgobject-2.0.so.0
/usr/lib/libglib-2.0.so.0
/usr/lib/libintl.so.8
/lib/ld-musl-x86_64.so.1
/usr/lib/libexpat.so.1
/usr/lib/libheif.so.1
/usr/lib/libwebpmux.so.3
/usr/lib/libwebpdemux.so.2
/usr/lib/libwebp.so.7
/usr/lib/libpng16.so.16
/usr/lib/libjpeg.so.8
/usr/lib/libexif.so.12
/usr/lib/libgmodule-2.0.so.0
/usr/lib/libgio-2.0.so.0
/usr/lib/libffi.so.8
/usr/lib/libpcre.so.1
/usr/lib/libaom.so.3
/usr/lib/libde265.so.0
/usr/lib/libx265.so.199
/usr/lib/libstdc++.so.6
/usr/lib/libgcc_s.so.1
/lib/libz.so.1
/lib/libmount.so.1
/lib/libblkid.so.1

WebAssembly as an alternative solution?

So I was wondering if WebAssembly would be ready to act as a solution to provide C libraries to other ecosystems with less headaches. The questions I was wondering about are:

1. How much slower is WASM compare to a C binding?
2. Does WASM allow me to get rid of all C dependencies?

How much slower is WASM compare to a C binding?

WASM will be slower compared to C bindings, this is a fact I wasn’t wondering about. I just wanted to get a rough idea of how much slower. So I did a benchmark - as for all benchmarks take it with a grain of salt.

Testsetup: Encode a 6048x2048px big image using mozjpeg (mozjpeg-sys to be more specific), resize it down to 1008x665px using PistonDevelopers/resize, and decode it again using mozjpeg.

As a reference, when executing the test without going through WASM (so Rust directly compiled to an executable), the image transformation takes around 205ms. I compared this to running the same code compiled to WASM in different WASM runtimes in Go. The results are:

• Wasmer LLVM runtime: 265ms
• Wasmtime: 272ms
• Wasmer Cranelift runtime: 275ms
• wazero: 450000ms (7.5min)

Now I know that it is roughly 25% slower (for the first three) for that specific use-case. Since I’d heavily cache transformed images, I could life with the 25% slowdown.

Does WASM allow me to get rid of all C dependencies?

Yes, for Rust, as there are plenty of WASM runtimes written in Rust (Wasmer and Wasmtime for example). But unfortunately no for Go, as wazero was the only runtime I could find that is written in Go (and it doesn't work well enough for my use-case). Wasmer and Wasmtime are also available in Go, but only through CGO as both consume C APIs provided by the underlying Rust implementation. So I'd get rid of a C dependency by compiling it to WASM just to add a new C dependency to run the WASM.

Conclusion

I’d consider WASM as a good alternative for C dependencies for Node.js projects, if the use-case does allow for the slowdown compared to C bindings.

I personally have projects in Rust, Go and on Cloudflare Workers (V8). So my ideal solution would be a Rust project that:

• I can consume as WASM in Rust or directly as a Rust dependency if I am fine with the C dependencies,
• I can consume as WASM in Cloudflare Workers and
• I can consume via C bindings in Go.

My inner monk is still struggling a bit with the third point. But I already have a POC for it and it at least reduces the amount of shared objects I have to provide to my final Docker image to libc and libgcc1, which would allow me to at least use gcr.io/distroless/cc as a base image.

GoogleChromeLabs/squoosh is actually pretty close to what I'd want, but isn't published as a Rust dependency and heavily relies on Node.js so doesn't work in Cloudflare Workers.

Comments

[mainrs]

Probably not resolving your C nightmare - although I find that almost non-existend when working with Rust and the nix (with flakes) -, github.com/google/wuffs is an interesting approach to handling untrusted user data :)

FYI, nix flakes are an approach to setting up a reproducable development environment for your project. That means that you can actually pin library versions down. They aren't then managed by Go but rather C libraries and Go are managed by nix. Nix would then use your go.mod file to get the list of dependencies it needs to download for your project!

[Markus Ast]

Thanks for sharing. nix flakes sounds very interesting, I'll definitely give it a try 👍. Some might say use Docker instead, but Docker is way too slow on macOS.