DEV Community: lynett.eth

Misadventures in Docker and WASM Benchmarking

lynett.eth — Tue, 01 Jul 2025 14:06:13 +0000

Hi, I'm Lyn! I'm the sales lead at Altivox Networks, a cloud compute company with a presence in Springfield, MA. We are at a perpetual state of war with expensive compute.

I may be a sales lead, but I am also a programmer (a pretty bad one). My only experience with compiling to WebAssembly before this was playing around with Hyperware, so I thought, "why not try it out?"

Recently, I came across wasmtime, a fast and secure runtime for WebAssembly on the server, which sent me down on a path of researching about WebAssembly.

If WASM+WASI existed in 2008, we wouldn't have needed to create Docker. That's how important it is. WebAssembly on the server is the future of computing. -Solomon Hykes

Specs

We'll be running these benchmarks inside a VM on my server running Proxmox, which should have similar CPU performance to Altivox's Budget VPS line.

OS: Ubuntu 22.04
CPU: 8 vCPUs (Intel Xeon E5-2660v3)
RAM: 32GB DDR4-2133

This benchmark will be using Docker version 28.1.1, build 4eba377. Docker has support for wasm through the wasmedge runtime!

Initial setup

We'll be benchmarking three different environments: Docker (musl, distroless), wasmtime standalone and native standalone (musl). Using musl because static linking for distroless + it's faster. The benchmark programs are written in Rust <3

(I could not get Docker + WASM working.)

There will be two benchmark programs:

Recursive Fibonacci
HTTP webserver

I'll be running the Fibonacci program 10 times for each setup using hyperfine. The webserver will be stress tested using bombardier.

The setup

The setup is a kludgy mess, but it works. I decided to target the new wasm32-wasip2 / WASI v0.2 target rather than wasm32-wasip1 (formerly wasm32-wasi).

First, I prepared Docker's WebAssembly support. A quick web search couldn't find any information for Docker Engine, so I went ahead and asked Perplexity for this information. It told me to add the following to /etc/docker/daemon.json:

{
 "features": {
    "containerd-snapshotter": true
  }
}

Then I restarted the Docker daemon. But my troubles didn't end there: see, enabling this gets rid of your preexisting images, so you have to rebuild them. I built the images beforehand (uh oh)

docker buildx build -t fibonacci-regular .
docker buildx build -t fibonacci-wasm . -f wasm.dockerfile

And after that, I needed to install the shim. Unfortunately, there doesn't seem to be any good way other than downloading from https://github.com/containerd/runwasi/releases and dropping the binaries into /bin or some other folder in PATH.

Oops. The shims (wasmtime and wasmedge) for some reason simply did not work: they would just hang without outputting anything, and I had to kill it with 3 ^Cs.

Giving up

Okay. I just ran docker run --rm --runtime=io.containerd.wasmedge.v1 --platform=wasi/wasm secondstate/rust-example-hello:latest to see if it was an issue with my program, and nope, same hanging.

Docker still works though. docker run --rm hello-world outputted a beautiful little hello world message. Drats. Whatever, we can scrap the Docker shim...

Finally, the gosh-darned results

As expected, when it came to time taken to compute, the native code outperformed the code running in Docker, which then outperformed the WASM code. However, it's not really a big difference.

Fibonacci

Benchmark 1: docker run --rm fibonacci-regular
  Time (mean ± σ):     16.324 s ±  0.469 s    [User: 0.045 s, System: 0.048 s]
  Range (min … max):   15.749 s … 17.393 s    10 runs

Benchmark 2: ./target/x86_64-unknown-linux-musl/release/fib
  Time (mean ± σ):     15.698 s ±  0.935 s    [User: 15.690 s, System: 0.006 s]
  Range (min … max):   14.561 s … 17.415 s    10 runs

Benchmark 3: wasmtime ./target/wasm32-wasip2/release/fib.wasm
  Time (mean ± σ):     23.533 s ±  0.769 s    [User: 23.535 s, System: 0.045 s]
  Range (min … max):   22.184 s … 25.009 s    10 runs

Summary
  ./target/x86_64-unknown-linux-musl/release/fib ran
    1.04 ± 0.07 times faster than docker run --rm fibonacci-regular
    1.50 ± 0.10 times faster than wasmtime ./target/wasm32-wasip2/release/fib.wasm

Webserver

Compiling the webserver written in Actix Web to WASM did not work due to a Tokio compile error:

error: Only features sync,macros,io-util,rt,time are supported on wasm.
   --> /home/lyn/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.45.1/src/lib.rs:475:1
    |
475 | compile_error!("Only features sync,macros,io-util,rt,time are supported on wasm.");
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I used an HTTP stress tester called Bombardier to benchmark the webserver. I originally worried that the bottleneck would be the stress tester itself, as it's written in Go, but for lack of a better option I went through with it.

docker run --rm -p 3030:3030 webserver-regular

./bombardier -c 200 -d 10s http://localhost:3030
Bombarding http://localhost:3030 for 10s using 200 connection(s)
[=========================================================================================================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     38065.93    4459.31   49631.32
  Latency        5.25ms     1.77ms    85.75ms
  HTTP codes:
    1xx - 0, 2xx - 380270, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     9.17MB/s

./target/x86_64-unknown-linux-musl/release/webserver

./bombardier -c 200 -d 10s http://localhost:3030
Bombarding http://localhost:3030 for 10s using 200 connection(s)
[=========================================================================================================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     62399.35    8079.63   80232.54
  Latency        3.20ms     0.94ms    44.35ms
  HTTP codes:
    1xx - 0, 2xx - 623764, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    15.05MB/s

To conclude

Well, this was fun. It did sate my curiosity, and now I know a little more about WebAssembly. Though WebAssembly on the server isn't production ready yet (especially with all the random issues I ran into, such as Docker runtime and Tokio compilation), it's still an interesting idea!

Though containers compute at near-native speeds, the addition of the Docker networking stack added a lot of overhead even for a simple port binding.

Could WebAssembly runtimes beat Docker on the networking aspect? I don't know, but my guess is yes. Anyhow, til next time! I've got to get back to work.

Ready to try self-hosting out for yourself? Coolify offers an excellent self-hosted PaaS experience combined with a solid server like those provided by Altivox!

You Don't Need Serverless

lynett.eth — Tue, 24 Jun 2025 16:31:13 +0000

Disclaimer: I'm the sales lead at Altivox Networks, a cloud compute company with a presence in Springfield, MA and with plans for expansion into Amsterdam, NL.

I'm a big fan of cheap things that just work. Servers are one of them.

Not very cash money of you

According to industry reports, 33% of organizations are spending over $12 million annually on public cloud services, with small to medium businesses averaging over $1.2 million yearly. A good chunk of this is going to serverless or overpriced servers.

Take a new trendy web app (meow?) that handles steady traffic throughout the day. I'll do some quick calculations based on Vercel Pro's pricing to compare costs.

Let's say you:

have 4 other team members ($20/seat)
get around 200,000 function invocations per day, or 6,000,000 per month ($0.60/M reqs minus 1M included reqs)
use 256MB RAM and 150ms on average per request, 90% of which is idle time ($0.18/GB-hr minus 1,000 GB-hr included)
get 100GB in and 1.5TB out ($0.06/GB Fast Origin Transfer minus 100GB included, $0.15/GB Fast Data Transfer minus 1TB included)

For simplicity's sake, let's say we have enough scale for Fluid Compute to fill up all the idle time with other requests.

That's about $20 (plan) + $80 (seats) + $3 (invocations) + $832.50 (CPU time) + $30 (Fast Origin Transfer) + $75 (Fast Data Transfer), which sums to just over 1 thousand US dollars per month.

Meanwhile, let's compare that pricing to a more server-based stack.

A 12GB Budget VM ($11.99/mo) from Altivox, which comes with 4 vCPU cores and 1TB included bandwidth can handle 200,000 rpd (2.3 rps) very very comfortably.

Extra bandwidth is $4/TB, so that's an extra $2.40 in transfer costs. Total server cost is $14.39, a large drop compared to the more 'modern', 'efficient' serverless stack.

What about latency / being close to my users?

Spin up more servers in different regions. Altivox has limited scale, but there are various other inexpensive hosting providers like Hetzner and OVH.

To be within 200ms of your users, you only really need five servers: Europe, Tokyo, US East, US West and Australia. Even without calculations, you can already tell that it beats Vercel easily.

Server management is not hard

Serverless companies actually make money off telling developers that the alternatives are difficult to manage, rather than providing high cost efficiency, as shown by the estimates above. This is also false due to modern tools like Docker and Kubernetes which make managing your apps easy.

Deploying an application today looks like this:

docker build -t myapp .
docker run -d -p 80:3000 myapp

That's it. No dependency issues, no environment mismatches, no configuration drift. Want zero-downtime deployments? Docker Compose with a load balancer handles it fine. Need even more scale? Get another VM, or maybe even bare metal.

Compare that to debugging why your serverless function works locally but fails in production because of Node.js version mismatches between your development environment and Vercel's runtime. These environment inconsistencies literally cannot exist when you control the entire stack, especially when combined with devcontainers or other reproducible dev environments.

Complexity is transferred

Generally, serverless doesn't eliminate complexity, rather, it shifts it to a different place. Instead of managing servers, you're now managing:

Function timeout limits and memory configurations
Cold start optimization strategies
Vendor-specific deployment configurations
Configuring efficient CDN and caching etc etc.

You've traded straightforward server administration for vendor-specific serverless cloud architecture which changes based on your choice. Though the learning curve for a totally new dev is slightly simpler than managing Linux servers, these are skills that are solely useful for proprietary platforms, which leads to...

Vendor lock-in

Traditional server applications, especially containerized ones, are portable. A Docker container running on DigitalOcean will run identically on Hetzner, OVH, or your laptop. Your nginx config, database setup, and deployment scripts work everywhere.

Serverless apps become strongly architecturally-coupled to their platforms. When you use AWS, you don't just use AWS Lambda, you use API Gateway's request/response format, CloudWatch's logging system, IAM's permission model, and tons of other AWS-only services. When you use Vercel, it's the same thing. Moving away ends up being a total rewrite of your code!

This lock-in has real costs beyond just switching friction. It limits your ability to negotiate pricing, constrains your architectural choices, and makes you dependent on a single vendor's roadmap and reliability.

Building Linux skills

Server management teaches fundamental concepts: how operating systems allocate resources, how networks route traffic, how databases store and retrieve data, how applications scale under load. These concepts remain relevant even in serverless development, because they're not perfectly abstracted away!

The skills learned by developing with servers are transferable and valuable throughout a dev career. If you understand Linux, networking, and system administration, you become a better developer overall, regardless of your deployment target.

A practical alternative

Serverless is not always a bad option. For very event-driven workloads or applications with extremely variable traffic patterns, serverless can make sense. Do the numbers before making important decisions!

But for the majority of web apps: APIs, web services, background workers; traditional deployment models offer better pricing and more control.

Modern server management isn't the nightmare it was 10 years ago. With Docker, excellent VPS providers, and mature tooling, running your own infrastructure is more accessible than ever. You'll likely save money, gain flexibility, and build valuable skills.

Major cloud providers profit from your dependencies and won't teach you these fundamentals. But for most applications, well-managed servers outperform serverless on cost, perf, and maintainability while keeping you in control of your own technology stack.

Ready to try it out for yourself? Coolify offers an excellent self-hosted PaaS experience combined with a solid server like those provided by Altivox!