What are we trying to achieve
Let's say you have a JavaScript application that runs in bun and you've identified some bottleneck that you'd like to optimize.
Rewriting it in a more performant language may just be the solution you need.
As a modern JS runtime, Bun supports Foreign Function Interface (FFI) to call libraries written in other languages that support exposing C ABIs, like C, C++, Rust and Zig.
In this post, we'll go over how one may use it, and conclude whether one can benefit from it.
How to link the library to JavaScript
This example is using Rust. Creating a shared library with C bindings looks differently in other languages but the idea remains the same.
From JS side
Bun exposes its FFI API through bun:ffi module.
The entrypoint is a dlopen function. It takes a path that's either absolute or relative to the current working directory to the library file (the build output with a .so extension for Linux, .dylib for macOS or .dll for Windows) and an object with the signatures of functions you want to import.
It returns an object with a close method which you may use to close the library once it's not needed anymore and symbols property which is an object containing the functions you chose.
import {
dlopen,
FFIType,
read,
suffix,
toArrayBuffer,
type Pointer,
} from "bun:ffi";
// Both your script and your library don't typically change their locations
// Use `import.meta.dirname` to make your script independent from the cwd
const DLL_PATH =
import.meta.dirname + `/../../rust-lib/target/release/library.${suffix}`;
function main() {
// Deconstruct object to get functions
// but collect `close` method into object
// to avoid using `this` in a wrong scope
const {
symbols: { do_work },
...dll
} = dlopen(DLL_PATH, {
do_work: {
args: [FFIType.ptr, FFIType.ptr, "usize", "usize"],
returns: FFIType.void,
},
});
/* ... */
// It is unclear whether it is required or recommended to call `close`
// an example says `JSCallback` instances specifically need to be closed
// Note that using `symbols` after calling `close` is undefined behaviour
dll.close();
}
main();
Passing data through FFI boundary
As you may notice, the supported types that bun accepts through FFI are limited to numbers, including pointers.
Notably size_t or usize is missing from the list of supported types, even though the code for it exists as of bun version 1.1.34.
Bun doesn't offer any help in passing data more complex than a C string. That means you'll have to work with pointers yourself.
Let's see how to pass a pointer from JavaScript to Rust ...
{
reconstruct_slice: {
args: [FFIType.ptr, "usize"],
returns: FFIType.void,
},
}
const array = new BigInt64Array([0, 1, 3]);
// Bun automatically converts `TypedArray`s into pointers
reconstruct_slice(array, array.length);
/// Reconstruct a `slice` that was initialized in JavaScript
unsafe fn reconstruct_slice(
array_ptr: *const i64,
length: libc::size_t,
) -> &[i64] {
// Even though here it's not null, it's good practice to check
assert!(!array_ptr.is_null());
// Unaligned pointer can lead to undefined behaviour
assert!(array_ptr.is_aligned());
// Check that the array doesn't "wrap around" the address space
assert!(length < usize::MAX / 4);
let _: &[i64] = unsafe { slice::from_raw_parts(array_ptr, length) };
}
... and how to return a pointer from Rust to JavaScript.
{
allocate_buffer: {
args: [],
returns: FFIType.ptr,
},
as_pointer: {
args: ["usize"],
returns: FFIType.ptr,
},
}
// Hardcoding this value for 64-bit systems
const BYTES_IN_PTR = 8;
const box: Pointer = allocate_buffer()!;
const ptr: number = read.ptr(box);
// Reading the value next to `ptr`
const length: number = read.ptr(box, BYTES_IN_PTR);
// Hardcoding `byteOffset` to be 0 because Rust guarantees that
// Buffer holds `i32` values which take 4 bytes
// Note how we need to call a no-op function `as_pointer` because
// `toArrayBuffer` takes a `Pointer` but `read.ptr` returns a `number`
const _buffer = toArrayBuffer(as_pointer(ptr)!, 0, length * 4);
#[no_mangle]
pub extern "C" fn allocate_buffer() -> Box<[usize; 2]> {
let buffer: Vec<i32> = vec![0; 10];
let memory: ManuallyDrop<Vec<i32>> = ManuallyDrop::new(buffer);
let ptr: *const i32 = memory.as_ptr();
let length: usize = memory.len();
// Unlike a `Vec`, `Box` is FFI compatible and will not drop
// its data when crossing the FFI
// Additionally, a `Box<T>` where `T` is `Sized` will be a thin pointer
Box::new([ptr as usize, length])
}
#[no_mangle]
pub const extern "C" fn as_pointer(ptr: usize) -> usize {
ptr
}
Rust doesn't know JS is taking ownership of the data on the other side, so you have to explicitly tell it to not deallocate the data on the heap using ManuallyDrop. Other languages that manage memory will have to do something similar.
Memory management
As we can see, it's possible to allocate memory in both JS and Rust, and neither can safely manage others memory.
Let's choose where you should allocate your memory and how.
Allocate in Rust
There are 3 methods of delegating memory cleanup to Rust from JS and all have their pros and cons.
Use FinalizationRegistry
Use FinalizationRegistry to request a cleanup callback during garbage collection by tracking the object in JavaScript.
{
drop_buffer: {
args: [FFIType.ptr],
returns: FFIType.void,
},
}
const registry = new FinalizationRegistry((box: Pointer): void => {
drop_buffer(box);
});
registry.register(buffer, box);
/// # Safety
///
/// This call assumes neither the box nor the buffer have been mutated in JS
#[no_mangle]
pub unsafe extern "C" fn drop_buffer(raw: *mut [usize; 2]) {
let box_: Box<[usize; 2]> = unsafe { Box::from_raw(raw) };
let ptr: *mut i32 = box_[0] as *mut i32;
let length: usize = box_[1];
let buffer: Vec<i32> = unsafe { Vec::from_raw_parts(ptr, length, length) };
drop(buffer);
}
Pros
- It's simple
Cons
- Garbage collection is engine specific and non-deterministic
- Cleanup callback is not guaranteed to be called at all
Use toArrayBuffer's finalizationCallback parameter
Delegate garbage collection tracking to bun to call a cleanup callback.
When passing 4 parameters to toArrayBuffer, the 4th one must be a C function to be called on cleanup.
However, when passing 5 parameters, the 5th parameter is the function and the 4th parameter must be a context pointer that gets passed it.
{
box_value: {
args: ["usize"],
returns: FFIType.ptr,
},
drop_box: {
args: [FFIType.ptr],
returns: FFIType.void,
},
drop_buffer: {
args: [FFIType.ptr, FFIType.ptr],
returns: FFIType.void,
},
}
// Bun expects the context to specifically be a pointer
const finalizationCtx: Pointer = box_value(length)!;
// Note that despite the presence of these extra parameters in the docs,
// they're absent from `@types/bun`
//@ts-expect-error see above
const buffer = toArrayBuffer(
as_pointer(ptr)!,
0,
length * 4,
//@ts-expect-error see above
finalizationCtx,
drop_buffer,
);
// Don't leak the box used to pass buffer through FFI
drop_box(box);
#[no_mangle]
pub unsafe extern "C" fn box_value(value: usize) -> Box<usize> {
Box::new(value)
}
/// # Safety
///
/// This call assumes the box hasn't been mutated in JS
#[no_mangle]
pub unsafe extern "C" fn drop_box(raw: *mut [usize; 2]) {
let box_: Box<[usize; 2]> = unsafe { Box::from_raw(raw) };
drop(box_);
}
/// As per bun docs, expected signature like in JavaScriptCore's `JSTypedArrayBytesDeallocator`
/// https://developer.apple.com/documentation/javascriptcore/jstypedarraybytesdeallocator?language=objc
///
/// # Safety
///
/// This call assumes the buffer hasn't been mutated in JS
#[no_mangle]
pub unsafe extern "C" fn drop_buffer(ptr: *mut i32, len: *mut usize) {
// reconstruct the context box to not leak it
let length: usize = unsafe { *Box::from_raw(len) };
let buffer: Vec<i32> = unsafe { Vec::from_raw_parts(ptr, length, length) };
drop(buffer);
}
Pros
- Delegate logic out of JavaScript
Cons
- A lot of boilerplate and chances to leak memory
- Missing type annotation for
toArrayBuffer - Garbage collection is engine specific and non-deterministic
- Cleanup callback is not guaranteed to be called at all
Manage memory manually
Just drop the memory yourself after you don't need it anymore.
Luckily TypeScript has a very helpful Disposable interface for this and the using keyword.
It's an equivalent to Python's with or C#'s using keywords.
See the docs for it
{
drop_box: {
args: [FFIType.ptr],
returns: FFIType.void,
},
drop_buffer: {
args: [FFIType.ptr, "usize"],
returns: FFIType.void,
},
}
class RustVector implements Disposable {
constructor(
private inner: Int32Array,
private dropBuffer: (ptr: Int32Array, len: number) => void,
) {}
[Symbol.dispose](): void {
this.dropBuffer(this.inner, this.inner.length);
}
}
const buffer = new Int32Array(toArrayBuffer(as_pointer(ptr)!, 0, length * 4));
// Don't leak the box used to pass buffer through FFI
drop_box(box);
using wrapper = new RustVector(buffer, drop_buffer);
/// # Safety
///
/// This call assumes the box hasn't been mutated in JS
#[no_mangle]
pub unsafe extern "C" fn drop_box(raw: *mut [usize; 2]) {
let box_: Box<[usize; 2]> = unsafe { Box::from_raw(raw) };
drop(box_);
}
/// # Safety
///
/// This call assumes the buffer hasn't been mutated in JS
#[no_mangle]
pub unsafe extern "C" fn drop_buffer(ptr: *mut i32, length: usize) {
let buffer: Vec<i32> = unsafe { Vec::from_raw_parts(ptr, length, length) };
drop(buffer);
}
Pros
- Cleanup is guaranteed to run
- You have control of when you want to drop the memory
Cons
- Boilerplate object for
Disposableinterface - Manually dropping memory is slower than using garbage collector
- If you want to give away the ownership of the buffer you have to make a copy and drop the original
Allocate in JS
This is much simpler and safer as deallocating is handled for you.
However, there is a significant drawback.
Since you can't manage JavaScript's memory in Rust, you can't go over the buffer's capacity as that will cause a deallocation. That means you have to know buffer size before passing it to Rust.
Not knowing how many buffers you need beforehand will also incur a lot of overhead as you'll be going back and forth through FFI just to allocate.
const CAPACITY = 20;
// The buffer is initialized with zeroes
const buffer = new Int32Array(CAPACITY);
mutate_buffer(buffer, CAPACITY);
/// # Safety
///
/// Ensure that the length of the buffer is at least `capacity` size
#[no_mangle]
pub unsafe extern "C" fn mutate_buffer(ptr: *mut i32, capacity: usize) {
assert!(!ptr.is_null());
assert!(ptr.is_aligned());
assert!(capacity < usize::MAX / 4);
// Manually drop so that vec doesn't get cleaned up
let mut buffer: ManuallyDrop<Vec<i32>> =
ManuallyDrop::new(unsafe { Vec::from_raw_parts(ptr, 0, capacity) });
let other: Vec<i32> = vec![/* ... */];
assert!(buffer.len() + other.len() <= buffer.capacity());
buffer.extend_from_slice(&other);
}
A sidenote on strings
If the output you're expecting from the library is a string you may have considered the microoptimization of returning a vector of u16 rather than a string since typically JavaScript engines use UTF-16 under the hood.
However, that would be a mistake because transforming your string to a C string and using bun's cstring type will be mildly faster.
Here's a benchmark done using a nice benchmark library mitata
function readStringU16(
create_string_u16: (ptr: Uint16Array, len: number) => void,
): string {
const length = 12;
const buffer = new Uint16Array(length);
create_string_u16(buffer, length)!;
const decoder = new TextDecoder("utf-16");
const copy = decoder.decode(buffer);
return copy;
}
function readStringC(create_string_c: () => CString): string {
const cstr = create_string_c();
const copy = cstr.toString();
return copy;
}
const readU16 = readStringU16.bind(null, create_string_u16);
const readC = readStringC.bind(null, create_string_c);
summary(() => {
bench("read u16", readU16);
bench("read c", readC);
});
await run({ colors: false });
const STRING: &str = "Hello, World";
#[no_mangle]
pub unsafe extern "C" fn create_string_u16(ptr: *mut u16, capacity: usize) {
let buffer: &mut [u16] = unsafe { slice::from_raw_parts_mut(ptr, capacity) };
let src: Vec<u16> = STRING.encode_utf16().collect::<Vec<u16>>();
buffer.copy_from_slice(&src);
}
#[no_mangle]
pub extern "C" fn create_string_c() -> *const c_char {
ManuallyDrop::new(CString::new(STRING).unwrap()).as_ptr()
}
benchmark avg (min … max) p75 p99 (min … top 1%)
-------------------------------------- -------------------------------
read u16 1.12 µs/iter 1.21 µs █▂
(905.96 ns … 1.83 µs) 1.79 µs ██▆▅▆▇▇▇▃▂▂▄▃▃▂▂▁▂▂▁▁
read c 727.82 ns/iter 809.01 ns █▂
(590.53 ns … 1.03 µs) 969.92 ns ▆███▆▆▇▃▆▄▄▆▃▄▆▆▃▂▃▂▂
summary
read c
1.54x faster than read u16
What about WebAssembly?
It's time to address the elephant in the room that is WebAssembly.
Should you choose nice existing WASM bindings over dealing with C ABI?
The answer is probably neither.
Is it actually worth it?
Introducing another language to your codebase will require more than just a single bottleneck to be worth it DX-wise and performance-wise.
Here is a benchmark for a simple range function in JS, WASM and Rust.
// rs.rs
#[no_mangle]
pub unsafe extern "C" fn rs_range(ptr: *mut i32, start: i32, end: i32) {
let len: usize = usize::try_from(end - start).unwrap();
let buffer: &mut [i32] = unsafe { slice::from_raw_parts_mut(ptr, len) };
let src: Vec<i32> = (start..end).collect();
buffer.copy_from_slice(&src);
}
// wasm.rs
#[wasm_bindgen]
pub fn wa_range(start: i32, end: i32) -> Vec<i32> {
(start..end).collect()
}
function tsRange(start: number, end: number): number[] {
return [...Array(end - start).keys()].map((x) => x + start);
}
function wrapWa(
wa_range: (start: number, end: number) => Int32Array,
): (start: number, end: number) => number[] {
return (start, end) => {
return Array.from(wa_range(start, end));
};
}
function wrapRs(
rs_range: (ptr: Int32Array, start: number, end: number) => void,
): (start: number, end: number) => number[] {
return (start, end) => {
const buffer = new Int32Array(end - start);
rs_range(buffer, start, end);
return Array.from(buffer);
};
}
await init();
const waRange = wrapWa(wa_range);
const rsRange = wrapRs(rs_range);
summary(() => {
bench("ts", () => tsRange(100, 50000));
bench("wa", () => waRange(100, 50000));
bench("rs", () => rsRange(100, 50000));
});
await run({ colors: false });
benchmark avg (min … max) p75 p99 (min … top 1%)
-------------------------------------- -------------------------------
ts 1.33 ms/iter 1.25 ms █
(802.90 µs … 5.29 ms) 4.22 ms ▂██▅▂▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁
wa 1.58 ms/iter 1.72 ms █
(1.17 ms … 4.36 ms) 4.09 ms ██▄▃▄▃▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁
rs 1.47 ms/iter 1.43 ms ▂█
(1.17 ms … 4.11 ms) 3.84 ms ██▃▂▂▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
summary
ts
1.1x faster than rs
1.18x faster than wa
Native library barely beats out WASM and consistently loses to the pure TypeScript implementation.
And that's it for this tutorial for/exploration of bun:ffi module. Hopefully we all have walked away from this a little bit more educated.
Feel free to share thoughts and questions in the comments
Top comments (0)