DEV Community: Kyle Hanson

Announcing Puff - A hybrid Rust/Python Framework to build Apps for the Cloud

Kyle Hanson — Mon, 24 Oct 2022 17:43:31 +0000

☁ Puff ☁

Python and Rust are near perfect compliments for each other.

One of the primary challenges of writing Python is how to optimize core loops and logic of your application. Python has a lot of integrations that make it possible to write C code and get another level of performance, but those libraries historically have not been that viable due to a plethora of issues involving ease of use, portability and usefulness.

Rust is a great language that makes it easy to write high-performance code safely. However, the added safety can add a large degree of cognitive overhead that can sometimes get in the way and prevent you from just getting it done.

Puff recognizes that Python is a great logic layer but bad at computations and optimizations. Puff also recognizes that Rust is great at computations and optimizations but sometimes it is difficult to express extremely high level abstractions.

Puff Repo

Puff is a "deep stack" framework to allow Rust to sit on another level as Python. It allows you to build a cohesive set of tools in Rust that use Cargo and the Rust ecosystem and easily talk to Python.

Puff makes the bold attempt at building a framework with full compatibility:

Python Compatible
Rust Compatible
AsyncIO Compatible
ASGI / WSGI Compatible
Tokio Compatible
Django Compatible
Axum Compatible
FastAPI Compatible

A hybrid approach utilizes the best of both languages. Use Python to control logic and high-level concepts. Use Rust to optimize tight loops and IO. Easily move between both in a fluid motion.

Full Service

Modern apps are complex and require API endpoints, database pools, redis, websockets, PubSub and more. Puff believes that those things should be part of a "standard library" so they all work well together. With this philosophy, Puff includes:

Greenlets
Asyncio
Uvloop
Postgres
Redis
Multi-node PubSub
Http/Http2 Server
GraphQL

Puff's Graphql engine has a couple of key features. It supports subscriptions allowing you to easily write websocket interfaces. The engine also accepts raw SQL returned from Python functions. It will take the SQL, run it on the current database pool and transform the rows to JSON without having to return to Python.

Most importantly it works on a "layer" basis. Most GraphQL libraries work on a "node" basis going to each object to resolve the fields. Puff works by "layers", collecting all information needed for every object in a field before it resolves it. This means you can make the minimum number of queries necessary.

See later in this post for a GraphQL example.

Quick Start

Puff requires Python >= 3.10, Rust / Cargo. Python's Poetry is optional.

Your Rust Puff Project needs to find your Python project. Even if they are in the same folder, they need to be added to the PYTHONPATH.

One way to set up a Puff project is like this:

cargo new my_puff_proj --bin
cd my_puff_proj
cargo add puff-rs
poetry new my_puff_proj_py
cd my_puff_proj_py
poetry install puff-py

And add the puff plugin for poetry.

[tool.poetry.scripts]
cargo = "puff.poetry_plugins:cargo"

Now from my_puff_proj_py you can run your project with poetry run cargo to access cargo from poetry and expose the virtual environment to Puff.

The Python project doesn't need to be inside off your Rust package. It only needs to be on the PYTHONPATH. If you don't want to use poetry, you will have to set up a virtual environment if needed and set PYTHONPATH when running Puff.

Example

Puff allows you to access Rust from Python easily.

use puff_rs::prelude::*;
use puff_rs::program::commands::PythonCommand;


// Use pyo3 to generate Python compatible Rust classes.
#[pyclass]
struct MyPythonState;

#[pymethods]
impl MyPythonState {
    fn hello_from_rust(&self, py_says: Text) -> Text {
        format!("Hello from Rust! Python Says: {}", py_says).into()
    }

    // Async Puff functions take a function to return the result with and offload the future onto Tokio.
    fn hello_from_rust_async(&self, return_func: PyObject, py_says: Text) {
        run_python_async(return_func, async {
            tokio::time::sleep(Duration::from_secs(1)).await;
            debug!("Python says: {}", &py_says);
            Ok(42)
        })
    }
}


fn main() -> ExitCode {
    let rc = RuntimeConfig::default().set_global_state_fn(|py| Ok(MyPythonState.into_py(py)));
    Program::new("my_first_app")
        .about("This is my first app")
        .runtime_config(rc)
        .command(PythonCommand::new("run_hello_world", "my_python_app.hello_world"))
        .run()
}

Python:

from puff import global_state

rust_obj = global_state()

def hello_world():
    print(rust_obj.hello_from_rust("Hello from Python!"))
    # Rust functions that execute async code need to be passed a special return function.
    print(puff.wrap_async(lambda ret_func: rust_obj.hello_from_rust_async(ret_func, "hello async")))

See puff-py repo and puff repo for more examples including Django, FastAPI, and more.

GraphQL Example

Here is an example of a Puff GraphQL app. Notice you can create Python objects and return them or you can return SQL strings.

The GraphQL example in the Repo has information about how to run it.

from puff import graphql
from dataclasses import dataclass
from typing import Optional, Tuple, List, Any

pubsub = puff.global_pubsub()
CHANNEL = "my_puff_chat_channel"

@dataclass
class SomeInputObject:
    some_count: int
    some_string: str


@dataclass
class SomeObject:
    field1: int
    field2: str

@dataclass
class DbObject:
    was_input: int
    title: str

    @classmethod
    def child_sub_query(cls, context, /) -> Tuple[DbObject, str, List[Any], List[str], List[str]]:
        # Extract column values from the previous layer to use in this one.
        parent_values = [r[0] for r in context.parent_values(["field1"])]
        sql_q = "SELECT a::int as was_input, $2 as title FROM unnest($1::int[]) a"
        # returning a sql query along with 2 lists allow you to correlate and join the parent with the child.
        return ..., sql_q, [parent_values, "from child"], ["field1"], ["was_input"]


@dataclass
class Query:

    @classmethod
    def hello_world(cls, parents, context, /, my_input: int) -> Tuple[List[DbObject], str, List[Any]]:
        # Return a Raw query for Puff to execute in Postgres.
        # The elipsis is a placeholder allowing the Python type system to know which Field type it should tranform into.
        return ..., "SELECT $1::int as was_input, \'hi from pg\'::TEXT as title", [my_input]

    @classmethod
    def hello_world_object(cls, parents, context, /, my_input: List[SomeInputObject]) -> Tuple[List[SomeObject], List[SomeObject]]:
        objs = [SomeObject(field1=0, field2="Python object")]
        if my_input:
            for inp in my_input:
                objs.append(SomeObject(field1=inp.some_count, field2=inp.some_string))
        # Return some normal Python objects.
        return ..., objs

    @classmethod
    def new_connection_id(cls, context, /) -> str:
        # Get a new connection identifier for pubsub.
        return pubsub.new_connection_id()


@dataclass
class Mutation:
    @classmethod
    def send_message_to_channel(cls, context, /, connection_id: str, message: str) -> bool:
        print(context.auth_token) #  Authoritzation bearer token passed in the context
        return pubsub.publish_as(connection_id, CHANNEL, message)


@dataclass
class MessageObject:
    message_text: str
    from_connection_id: str
    num_processed: int


@dataclass
class Subscription:
    @classmethod
    def read_messages_from_channel(cls, context, /, connection_id: Optional[str] = None) -> Iterable[MessageObject]:
        if connection_id is not None:
            conn = pubsub.connection_with_id(connection_id)
        else:
            conn = pubsub.connection()
        conn.subscribe(CHANNEL)
        num_processed = 0
        while msg := conn.receive():
            from_connection_id = msg.from_connection_id
            # Filter out messages from yourself.
            if connection_id != from_connection_id:
                yield MessageObject(message_text=msg.text, from_connection_id=from_connection_id, num_processed=num_processed)
                num_processed += 1


@dataclass
class Schema:
    query: Query
    mutation: Mutation
    subscription: Subscription

Status of Project

Puff is an experiment, but the results are very promising in terms of performance and productivity. The Zen of Puff and the Deep Stack is that no one language has the answer and fluidity is key.

SortedSetKV - An ultrafast Elixir key value DB with optional integer index

Kyle Hanson — Thu, 17 Feb 2022 21:08:28 +0000

Have you ever found yourself needing a Key Value database with a secondary index? Redis' sorted sets almost fit the bill, but do not allow you to store associated values with them. Therefore you are left to manage the secondary index yourself.

I wrote a lightweight wrapper around Rusts' sled library. You insert keys into a database that is persisted on disk. It was originally written to help expire keys based on a TTL, but it can also be used as a time-series database or anything where you need a secondary u64 index.

You can check it out on Github.

SortedSetKV is stored on disk and can grow beyond your RAM limit. There is no GenServer abstraction and calls are made directly to Rust. Its super fast and easy to use.

This is alpha software, so be safe!

Basic usage

{:ok, db} = SortedSetKV.open("mypath")
# Add a key to a set, with a value and a score.
# The last parameter tells to only add if the old score is less than new score.
:ok = SortedSetKV.zadd(db, "mycollection", "hello", "world", 42, true)
:ok = SortedSetKV.zadd(db, "mycollection", "foo", "bar", 420, true)
:ok = SortedSetKV.zadd(db, "mycollection", "noscore", "", nil, true)
:ok = SortedSetKV.zadd(db, "mycollection", "novalue", nil, 100, true)
# Returns whether it exists and its score
{true, 42} = SortedSetKV.zscore(db, "mycollection", "hello")
{true, 420} = SortedSetKV.zscore(db, "mycollection", "foo")
{true, nil} = SortedSetKV.zscore(db, "mycollection", "noscore")
{true, 100} = SortedSetKV.zscore(db, "mycollection", "novalue")

# A key must have a score or a value to exist:
:ok = SortedSetKV.zadd(db, "mycollection", "noexists", nil, nil, true)
{false, nil} = SortedSetKV.zscore(db, "mycollection", "noexists")

Retrieving Values

# Get a key with a minimum score
{value, score} = SortedSetKV.zgetbykey(db, "mycollection", "hello", 0)
# A key with a score lower than the minscore will return nil
nil = SortedSetKV.zgetbykey(db, "mycollection", "foo", 500)
# see if any keys exist with the score
true = SortedSetKV.zexists(db, "mycollection", 0, 500)

Conditional Add

With zadd and zupdate, you can optionally only update the score if the new score is greater than the old score or if the old score is not set. This is important for monotonic TTL updates.

:ok = SortedSetKV.zadd(db, "mycollection", "hello", "world", 42, true)
{"world", 42} = SortedSetKV.zgetbykey(db, "mycollection", "hello", 0)
:ok = SortedSetKV.zadd(db, "mycollection", "hello", "value2", 0, true)
# only adds if the score is greather than
{"world", 42} = SortedSetKV.zgetbykey(db, "mycollection", "hello", 0)

:ok = SortedSetKV.zscoreupdate(db, "mycollection", "hello", 0, true)
{"world", 42} = SortedSetKV.zgetbykey(db, "mycollection", "hello", 0)

# Setting the value to false overrides this
:ok = SortedSetKV.zadd(db, "mycollection", "hello", "value2", 10, false)
{"value2", 10} = SortedSetKV.zgetbykey(db, "mycollection", "hello", 0)

:ok = SortedSetKV.zscoreupdate(db, "mycollection", "hello", 0, false)
{"value2", 0} = SortedSetKV.zgetbykey(db, "mycollection", "hello", 0)

Iterating keys with scores

Iterate over keys based on their score.

offset = 0
limit = 100
["hello"] = SortedSetKV.zrangebyscore(db, "mycollection", 0, 50, offset, limit)
# Filter by prefix and score
["foo"] = SortedSetKV.zrangebyprefixscore(db, "mycollection", "fo", 0, 500, offset, limit)

Removing Values

# Remove key
:ok = SortedSetKV.zrem(db, "mycollection", "hello")
# Remove all keys by score and returns how many it deleted
_ = SortedSetKV.zrembyrangebyscore(db, "mycollection", 0, 500, limit)

Queue

SortedSetKV comes with basic queue functionality. You can push or pop elements on either end of the queue.

:ok = SortedSetKV.rpush(db, "mylist", "value")
:ok = SortedSetKV.rpush(db, "mylist", "value2")
"value" = SortedSetKV.lpop(db, "mylist")
"value2" = SortedSetKV.lpop(db, "mylist")
nil = SortedSetKV.lpop(db, "mylist")
:ok = SortedSetKV.rpush(db, "mylist", "1")
:ok = SortedSetKV.rpush(db, "mylist", "2")
:ok = SortedSetKV.lpush(db, "mylist", "0")
"0" = SortedSetKV.lpop(db, "mylist")
"2" = SortedSetKV.rpop(db, "mylist")

TTL

If you use millisecond timestamps as the score, it behaves like a TTL.

{:ok, db} = SortedSetKV.open("mypath")
# Add a key to a set, with a value and a score
:ok = SortedSetKV.zadd(db, "mycollection", "hello", "world", :os.system_time(:millisecond) + 5000)
# Get key only if it is in TTL
SortedSetKV.zgetbykey(db, "mycollection", "foo", :os.system_time(:millisecond))

# Clean up exipired Keys
SortedSetKV.zrembyrangebyscore(db, "mycollection", 0, :os.system_time(:millisecond))

You can use a GenServer like this to customize your TTL cleanup. Because Elixir executes all Rust Nifs on one thread, you will not want to block for very long. It is wise to only delete a few keys at a time.

defmodule TTLCleanup do
    use GenServer
    require Logger

    @review_time 5_000

    def start_link(conn) do
      GenServer.start_link(__MODULE__, conn, [])
    end

    def init(conn) do
      Process.send_after(self(), :review_storage, @review_time)
      {:ok, conn}
    end

    def handle_info(:review_storage, conn) do
      Logger.debug("TTL cleanup")

      :ok = scan(conn, "collection1")
      :ok = scan(conn, "collection2")
      :ok = scan(conn, "collection3")

      Process.send_after(self(), :review_storage, @review_time)

      {:noreply, conn}
    end

    def scan(conn, collection) do
      new_agg =
        SortedSetKV.zrembyrangebyscore(conn, collection, 0, :os.system_time(:millisecond), 100)

      case new_agg do
        v when is_number(v) and v <= 99 ->
          :ok

        v when is_number(v) ->
          scan(conn, collection)
      end
    end
end

Conclusion

SortedSetKV is a different way to think about KeyValue storage. Whether you are storing time-series data or need to make a TTL expiration cache, SortedSetKV might be a good option.

Planetary Scale Rust and Golang by embedding WebAssembly in Elixir

Kyle Hanson — Sat, 05 Dec 2020 14:35:35 +0000

This blog post was heavily inspired by the Lunatic project and bkolobara’s blog post “Writing Rust the Elixir Way”.

Writing Rust the Elixir way

Bernard Kolobara ・ Nov 28 ・ 7 min read

Compiling languages to WebAssembly is the next great-leap forward in software. With portable binaries that safely run in a sandbox, WASM makes it simple to embed multiple guest languages in a host language. Combining these excellent executional qualities of WebAssembly with Elixir’s fault-tolerance and distributed nature, has massive potential. Elixir’s powerful messaging system layered on top of its supervisor-trees for monitoring processes provides a robust framework for a global network of interconnected WebAssembly actors.

The project

To investigate embedding WebAssembly inside of Elixir, I built a toy serverless function platform for executing WebAssembly code. Put simply, you upload a .wasm file and then call a function inside by going to a url. The project is meant to investigate the overheads involved with running WebAssembly. To this end, a series of rudimentary benchmarks were done on the project and are included at the end.

The finished code is here:

hansonkd / assemblex

Distributed WebAssembly VM

Assemblex

This is an toy WebAssembly Serverless Function deployment system. With Assemblex, you can deploy WebAssembly binaries across nodes and hot reload new code across the network easily.

Assemblex programs follow the WaPC protocol. Current Languages supported are:

Rust
Golang
AssemblyScript
Zig

Try it out

# Clone repo and all submodules
git clone --recurse-submodules -j4 git://github.com/hansonkd/assemblex.git
cd assemblex
# Start one node in one terminaal
iex --name a@127.0.0.1 -S mix phx.server
# Start another node in a different terminal
PORT=4001 iex --name b@127.0.0.1 -S mix phx.server
# Build and release the WASM binary on first node
cd rust-wasm-hello-world/
cargo build --release
curl -F 'wasm=@target/wasm32-unknown-unknown/release/hello_world.wasm' http://localhost:4000/deploy
# Watch Node A's Terminal until it says its deployed and try to call the Wasm Function:
echo "$(curl --no-progress-meter http://localhost:4000/call/hello_world)"
# Wait about 15seconds for the wasm to propagate to the other node. Watch Node B's terminal

…

View on GitHub

WaPC

WebAssembly by itself is rather restrictive: only allowing number types to be passed back and forth between the host. The host is responsible for reading the raw memory of the WebAssembly VM to extract strings. Web Assembly Procedure Calls aim to create a consistent interface across languages to pass binary data. Rustler is used to safely embed the WaPC runtime inside Elixir. WaPC currently supports Rust, Golang, AssemblyScript and Zig.

Concurrency

The standard WaPC runtime has one major restriction which does not mesh well with the Elixir universe; it operates synchronously. Every VM you spin up needs to have its own separate thread. It works if you have a small amount of workers running, but in Elixir you can be serving thousands of requests at once, which in our case each request needs its own VM. This blog post about adding preemptive scheduling to WasmTime inspired me to transform the WaPC runtime into an asynchronous scheduler. The current scheduler is not as advanced as Lunatic, yielding only on host callbacks, but it does allow you to recursively call the WaPC module without deadlocking. Adding preemptive calls should be as simple as running the Lunatic normalizer over the Wasm as the “yield” host callback has already been added.

Lasp

Every distributed system needs a backbone and ours is Lasp-Lang, a self-described “comprehensive programming system for planetary scale Elixir and Erlang applications”. We're going to depend on lasp for one main thing: a conflict-free, in-memory, eventually-consistent CRDT store. Nodes auto-configure and network themselves together and deploy our Wasm binary in just a few lines of code. Lasp is special because the networking backend uses partisan, which orchestrates a loosely connected network of nodes. Unlike regular erlang, each node does not automatically connect to all of the other nodes. Partisan works by intelligently composing the edges of the graph to maximize resiliency while minimizing overhead. The result is a self-configuring network capable of hosting hundreds or even thousands of nodes.

Phoenix

Elixirs Phoenix framework is used to provide a scalable HTTP interface to our app.

Running the Distributed Service

The service consists of 3 main things, deployment, supervision and execution.

Deployment

Lasp exposes a KV store that is eventually consistent across all nodes. Deployment is as simple as setting the key that will hold the Wasm binary and letting lasp do its thing and synchronize the value across all nodes. We will then update the version counter to signal the supervisor task to hotswap the new binary on the executor.

defmodule AssemblexWeb.PageController do
  ...
  def deploy(conn, %{"wasm" => upload}) do
    bytes = File.read!(upload.path)
    WapcSupervisor.deploy(bytes)
    text(conn, "Ok. Deploy Queued")
  end
end

...

defmodule Assemblex.WapcSupervisor do
  ...

  def deploy(new_code) do
    :lasp.update({"app-wasm", :lwwregister}, {:set, :erlang.unique_integer([:monotonic, :positive]), new_code}, self)
    :lasp.update({"app-version", :state_gcounter}, :increment, self)
  end

  ...
end

Then to initiate the deploy, just upload the .wasm to the deploy endpoint and let lasp do the rest.

curl -F 'wasm=@hello_world.wasm' http://localhost:4000/deploy

Supervision

Our app needs to know when to update itself. Lasp includes a “monotonic read” function that blocks until the value is at least that big. We will use the function to block until it changes to a value bigger than than one bigger of the current version. After that, the binary is loaded, the new thread started and registered with the local ProcessRegistry while the old process is shut down, unregistered and the loop starts again.

defmodule Assemblex.WapcSupervisor do
  use Task

  def start_link(arg) do
    Task.start_link(__MODULE__, :run, [arg])
  end

  def run(arg) do
    loop({0, nil})
  end

  defp loop({version, sender}) do
    :lasp.read({"app-version", :state_gcounter}, {:value, version + 1})
    {:ok, new_code} = :lasp.query({"app-wasm", :lwwregister})
    {:ok, pid} = WapcExecutor.spawn(new_code)
    ProcessRegistry.add(pid)
    if sender != nil do
      ProcessRegistry.remove(sender)
      Process.exit(sender, :normal)
    end
    loop({version + 1, pid})
  end

  ...
end

Execution

A Phoenix Controller gives our entry point into executing the code. We will pull one process off of the ProcessRegistry and send it to the WapcExecutor. A new VM will spin up, call the guest function and return the bytes.

defmodule AssemblexWeb.PageController do
  ...
  def call_wasm(conn, %{"function" => function} ) do
    {:ok, body, conn} = Plug.Conn.read_body(conn)
    resp = WapcExecutor.call(ProcessRegistry.get_process(), function, body)
    case resp do
        {:ok, resp_body} -> conn |> put_resp_content_type("application/octet-stream") |> send_resp(200, resp_body)
        {:error, err} -> conn |> put_status(500) |> text(inspect(err))
    end
  end
  ...
end

Another important part of execution is keeping a pool of warmed VMs ready to fire. We use a SyncChannel with a bound and utilize the concept described in this blog post for golang. The effect of using this programming pattern is we have an elastic resource pool. If no VM is in the pool, we will create a new one. If more than the channel’s bound exist and there is no demand, the instances will be discarded. All of this is done for us automatically with sync_channel

let bound = 100; // The number of instances you want to keep warm if there is no demand.
let (pool_return, pool_get) = sync_channel(100);

// Take a VM from the pool
match pool_get.try_recv() {
    Ok(instance) => {
        // Use the pre-warmed instance
    },
    Err(_) => {
        // Create  anew instance.
    }
}

// Return the VM to the pool
pool_return.try_send(instance.clone());

Of course, wasm wouldn’t be much use if it couldn’t call into the host. The WapcImport GenServer adds functionality so wasm apps can call Elixir functions.

defmodule Assemblex.WapcImports do
  use GenServer

  @impl true
  def init(opts) do
    {:ok, nil}
  end

  @impl true
  def handle_info({:invoke_callback, ".", "host", "do_work", payload, ref} = msg, state) do
    Task.start(fn ->
        # Do Work async so we dont block other calls.
        res = "<Some work result>"
        ExWapc.receive_callback_result(ref, true, res)
    end)

    {:noreply, state}
  end

  def handle_info({:invoke_callback, _bs, _ns, _func, _payload, ref} = msg, state) do
    ExWapc.receive_callback_result(ref, false, "No function found.")
    {:noreply, state}
  end
end

Try it for yourself

Finally, you can see how it works for yourself! All the code is available to try on your own at the Assemblex github repository. To get started with a multi-node setup, try the following:

# Clone repo and all submodules
git clone --recurse-submodules -j4 git://github.com/hansonkd/assemblex.git
cd assemblex

# Ensure a Start a redis sever is running
redis-server

# Start one node in one terminaal
iex --name a@127.0.0.1 -S mix phx.server
# Start another node in a different terminal
PORT=4001 iex --name b@127.0.0.1 -S mix phx.server


# Build and release the WASM binary on first node
cd rust-wasm-hello-world/
cargo build --release
curl -F 'wasm=@target/wasm32-unknown-unknown/release/hello_world.wasm' http://localhost:4000/deploy

# Watch Node A's Terminal until it says its deployed and try to call the Wasm Function:
echo "$(curl --no-progress-meter http://localhost:4000/call/hello_world)"

# Wait about 15seconds for the wasm to propagate to the other node. Watch Node B's terminal and for the message saying it was deployed.

# Call Node B to see the new code:
echo "$(curl --no-progress-meter http://localhost:4001/call/hello_world)"

# Spinning up new nodes should automatically connect and download the wasm app
PORT=4002 iex --name c@127.0.0.1 -S mix phx.server

Analysis

Obviously there is a lot of potential for unwanted overhead, so it is important to understand the trade-offs of running WebAssembly. In addition to the WebAssembly VM overhead, the erlang NIF interface has overhead of its own to consider. Pre-warmed VMs help a lot for reducing the penalty for calling and once you are inside the WebAssembly code it executes almost as fast as native code. Taking these things into consideration, lets take a look at some benchmarks.

In order to test the code, we will look at calling Elixir -> WebAssembly and have the WebAssembly function call back into Elixir. For these benchmarks we will query Redis 3 times and then concatenate the values together. In addition, we will monitor the performance of the same operation in native elixir. These benchmarks will use the following rust code compiled to WebAssembly and the WapcImport function as follows:

extern crate wapc_guest as guest;

use guest::prelude::*;

#[no_mangle]
pub extern "C" fn wapc_init() {
  register_function("hello_world", hello_world);
}

fn hello_world(_msg: &[u8]) -> CallResult {
    let mut res = host_call("host", "redis", "GET", b"some_key1")?;
    res.extend(host_call("host", "redis", "GET", b"some_key2")?);
    res.extend(host_call("host", "redis", "GET", b"some_key3")?);
    Ok(res)
}

defmodule Assemblex.WapcImports do
  use GenServer

  @impl true
  def init(opts) do
    {:ok, _} = Redix.command(:redix, ["SET", "some_key1", "0"])
    {:ok, _} = Redix.command(:redix, ["SET", "some_key2", String.duplicate("a", 50)])
    {:ok, _} = Redix.command(:redix, ["SET", "some_key3", String.duplicate("b", 500)])
    {:ok, nil}
  end

  @impl true
  def handle_info({:invoke_callback, "host", "redis", "GET", payload, ref} = msg, conn) do
    Task.start(fn ->
        {:ok, res} = Redix.command(:redix, ["GET", payload])
        ExWapc.receive_callback_result(ref, true, res)
    end)

    {:noreply, conn}
  end

  def handle_info({:invoke_callback, _bs, _ns, _func, _payload, ref} = msg, conn) do
    ExWapc.receive_callback_result(ref, false, "No function found.")
    {:noreply, conn}
  end
end

defmodule AssemblexWeb.PageController do
  use AssemblexWeb, :controller
  ...
  def native(conn, _) do
    {:ok, res} = Redix.command(:redix, ["GET", "some_key1"])
    {:ok, res2} = Redix.command(:redix, ["GET", "some_key2"])
    {:ok, res3} = Redix.command(:redix, ["GET", "some_key3"])

    conn |> put_resp_content_type("application/octet-stream") |> send_resp(200, res <> res2 <> res3)
  end
  ...
end

We will test 2 pool sizes: 100 and 1. Using multiple pool sizes will give us some data about the trade-offs and advantages to using more pre-warmed instances. A poolsize of 0 easily timeouts so it is excluded.

Running these benchmarks with apache-benchmark using varying concurrency values yields the following graph (both axes are logarithmic scale):

So with an adequate pool size, WebAssembly looks like a viable competitor to Elixir! There is still memory issues to consider (running all those VMs and allocating all that linear memory adds up when you have many concurrent requests. Maybe we should add a limit to the number of VMs being run at once.

Whats Next?

Currently the only host functionality we expose is through the WapcImports GenServer. In the future with WASI, we will have a full standard library at our disposal, including networking in the WebAssembly module. It would also be nice to implement the preemptive yields from the lunatic project and have the ability to put limits on the guest VM’s memory usage. Integrating a distributed supervisor like hord lets you easily distribute long-lived WebAssembly actors across your network and the network will automatically reboot and move them if they unexpectedly die.

Right now Assemblex is a toy project. However, even as a toy, we were able to implement some pretty incredible things. Multi-node deploys, a network capable of supporting hundreds of nodes, hot-reloading, and the beginnings of a robust WebAssembly execution framework. In order to get further however, it needs a lot more work, refactoring and love.

In this post we used Rust for WebAssembly, but we could have easily used WaPC’s Golang library. You can imagine extending the project to allow multiple wasm applications to be deployed on the same Elixir instance. There is an entire world of distributed and asynchronous Elixir libraries ready to be integrated and offer their functionality to their Golang and Rust guests.

If you are interested in the ideas presented here, please feel free to reach out to me at me@khanson.io and I would be happy to discuss.