Baking a Fast Lambda MicroVM: Lessons Learned in the Trenches

I built a browser-based Claude Code sandbox that runs each user in their own AWS Lambda MicroVM, with a persistent home directory on an Amazon S3-backed filesystem. If you want the build story and why MicroVMs are the right primitive for it, that's a separate post: Lambda MicroVMs + S3 Files: My Claude Code Sandbox for iPad.

This post is the performance story. It works its way out from one specific annoyance, but the lessons apply to any snapshot-booted MicroVM and to optimizing S3 Files mounts in general.

One thing worth knowing: I figured all of this out by pairing with Claude Code. We researched the platform together, formed hypotheses, and tested them through trial and error until things got fast. I built a sandbox to run a coding agent, and then the coding agent helped me make the sandbox fast. Everything in this post came out of that collaboration. Make of that what you will.

The challenge was straightforward. Every fresh VM made the user stare at "Mounting workspace…" for about 26 seconds. I got that down to a few seconds, and the first claude launch, which used to stall for over a minute, to a second or two. The interesting part is that the fix had nothing to do with the mount.

First, measure, don't guess

The obvious suspect was the mount itself. It's NFS over TLS to an S3-backed filesystem, so surely that's the slow part. Before I optimized a thing, I timed it on a warm VM:

time mount -t s3files -o accesspoint=fsap-... fs-... /mnt/test
# real  0m0.6s

Sub-second. Even after dropping the caches to force a cold read, the mount syscall itself was never the problem. The network path was a red herring.

The real timeline came from something simpler: file modification times on a cold VM. The /run lifecycle hook arrived about 4 seconds in, but my "mount finished" marker didn't land until roughly 26 seconds after that, for a mount that succeeded on its first attempt. That's 26 seconds of apparently doing nothing, on an operation that takes 0.6 seconds when warm. That gap is the whole story.

The learning: snapshot demand-paging

To see what's happening in that gap, you need one fact about how these VMs boot. A MicroVM starts from a memory and disk snapshot captured at image build time. Restoring a snapshot is what makes the boot fast. But there's a catch that shapes every performance decision on the platform.

Disk pages are lazy. Memory is restored eagerly at boot, but disk content is demand-paged: the first time any file is read on a fresh VM, its bytes get pulled from snapshot storage on the fly. That much is documented platform behavior. What the docs don't tell you is the speed. In my measurements, that first touch ran at roughly 3 MB/s. (Treat that number as an observation from my VMs in mid-2026, not a spec; it may well improve.)

Now look at what mount -t s3files actually is. It's a Python script. Before it can mount anything, the VM has to fault in a lot of cold bytes:

• the Python 3.13 interpreter and standard library, about 67 MB
• efs-proxy, the TLS proxy binary, at 24 MB
• the mount helpers, mount.nfs, the shells, and so on

That's roughly 100 MB of first-touch reads at 3 MB/s. There's your 25-plus seconds. The mount command wasn't slow. It was paging in the entire toolchain it needed before it could do a job that takes half a second. It was building the track before it could run on it.

The fix: teach the platform your cold path with /validate

The platform has a designed answer to demand-paging, and it's one of the image build hooks: /validate.

When you register a validate hook, the image build doesn't stop at capturing the snapshot. The platform boots a test VM from that snapshot and calls your /validate endpoint. Officially this is your end-to-end smoke test: exercise the app, return 200 if the image is good. But something else is happening while your validate code runs, and it's the part that matters here: the platform watches which disk pages get touched, and prefetches those pages on every future launch. The docs say it plainly: run realistic work through your app during validate, and Lambda prefetches what it saw you use.

Think about that in terms of the 26-second mount. The whole problem is that the mount toolchain demand-pages on first touch. If the platform prefetches exactly those pages, the problem evaporates. So the entire fix is: during /validate, touch everything your real startup path touches.

Registering the hook is one addition to the image's hooks config:

--hooks '{"port":9000,"microvmImageHooks":{
  "ready":"ENABLED","readyTimeoutInSeconds":180,
  "validate":"ENABLED","validateTimeoutInSeconds":300}, ...}'

And the handler runs the real cold path once. Mine exercises the Claude CLI, the login shells, git, and most importantly the S3 Files mount toolchain:

// /validate: the platform samples which pages this touches and
// prefetches them on future launches. Touch the REAL startup path.
'sudo -u coder HOME=/home/coder claude --version',
'bash -lc true', 'zsh -lc true || true', 'git --version',
// a real mount attempt (bogus access point, it fails, but only AFTER
// paging in python3.13, the mount helpers, and the TLS proxy)
'mount -t s3files -o "accesspoint=fsap-000...f" "$S3_FILES_FS_ID" /mnt/validate-test || true',

The results: the first mount on a fresh VM dropped from ~26 seconds to a few seconds, and the first claude launch (which used to page in a roughly 240 MB CLI bundle over 60 to 90 seconds) now starts in a second or two.

Two hard-won details that make the difference between this working and silently doing nothing:

• Your per-user work doesn't run during validate, so exercise its toolchain explicitly. This is the trap. The validate-time VM is launched by the platform, not by a user, so none of your per-user runtime work happens. In my case no access-point id arrives, my mount script correctly declines to mount anything, and the mount toolchain (the one path I most needed prefetched) would never get touched, so the sampler would never see it. That's what the deliberate bogus-mount line above is for. It fails (there's nothing real to mount), and failure is fine: on its way to failing it pages in the Python interpreter, the mount helpers, and the TLS proxy, and page-touches are all the sampler needs. If your startup has any run-time-only work, figure out what binaries it needs and touch them during validate by hand.
• Return 200 only when you're actually done. The contract is: the platform polls your validate endpoint; answer 503 while the workload is still running, 200 when it's finished. Answer 200 too early and the sampling window closes before your cold path has been walked. Kick the work off in the background, track completion with a marker file, and gate the 200 on it.

And one property worth appreciating: this is the platform doing the work for you. No hand-maintained list of files to keep warm, no RAM spent pinning things in memory, and when your toolchain changes, the next image build re-samples automatically. You describe your cold path by running it once; the platform does the rest.

A note on telemetry

While reading the mount logs, I found two errors on every single mount. Before you read on: neither of these is Lambda or CloudWatch misbehaving. Both are consequences of choices in my image, and I'm sharing them because if you customize your image the way I did, you'll hit them too.

The first one I caused directly. My image installed botocore for the mount helper's CloudWatch logging, but it installed it under Python 3.9, the stock Amazon Linux 2023 interpreter. Later, the image swaps the system Python to 3.13 for other tooling my sandbox needs, and mount.s3files runs under that 3.13, where no botocore exists. Two build steps, each fine alone, that quietly disagreed about which Python they were talking about, and the helper's log upload was broken from the start. The second is narrower: the particular efs-utils build in my image derives a CloudWatch log-stream name containing a colon, which CloudWatch's naming rules (correctly) reject. Different trigger, same category: something my image ships, not something the platform does wrong.

I disabled both rather than "fixing" them. Installing botocore for 3.13 would have put a sizable cold import (the package and its dependencies are on the order of 90 MB on disk) right back onto the mount's critical path, re-buying the exact problem I had just paid to remove. The local diagnostics under /var/log/amazon/efs/ are untouched, so I didn't give up anything I was actually using. Sometimes the honest fix for broken telemetry is to admit you never had it, and stop paying to pretend you do.

To be clear, this is not "no boto3 in the sandbox." The rule is narrower: keep the system interpreter lean, because that's the one the mount helper runs under. If a project inside the VM needs boto3, install it as a project dependency, a virtualenv in the home directory. The mount path never imports from there, and as a bonus it lands on the persistent home rather than the snapshot disk, so it survives VM recycles too. System-wide for the platform, per-project for you.

The results

The before and after, measured hook-to-ready on a cold VM. One honest caveat for all the numbers in this post: the documented platform behavior is that memory restores eagerly, disk demand-pages, and validate-time sampling drives prefetch. The rest (the paging speed, the shared bandwidth, the exact boot times) is what I observed and measured on my VMs in mid-2026, not a guarantee. The platform is young; the numbers may improve, and the techniques matter more than the constants.

Path	Before	After
Cold boot, hook to workspace ready	~26s	typically a few seconds (1–11s observed)
First claude launch on a fresh VM	60–90s	0.5–2s
Suspend to resume remount	n/a	~2s
The mount syscall itself	~0.6s	~0.6s (never the problem)

What applies where

A useful way to file these tricks away is to ask which technology each one actually belongs to. Some are about snapshot boot and apply to any snapshot-booted VM with no S3 Files in sight. Some are about S3 Files and apply even on a plain EC2 instance. And the headline lesson of this post lives specifically in the overlap.

Snapshot tricks. These hold for any snapshot-booted compute, whatever you're mounting:

1. "Cold start" is mostly first-touch disk I/O. Profile it with file modification times and drop-caches timing before you blame the network. The syscall that looks slow is often waiting on bytes.
2. Implement /validate, and walk your real cold path in it. The platform samples the pages your validate workload touches and prefetches them on every future launch. It only learns what you show it, so exercise the actual startup path, including toolchains that normally only run with per-user input.
3. Exercise the real startup path in validate, not a blanket prefetch. A broad find | cat sweep teaches the sampler about pages a session never touches. Run the actual binaries and commands your app runs; that's exactly the set worth prefetching.
4. The memory snapshot is a budget, not a free lunch. Memory restores eagerly, so every megabyte resident at capture makes every run and resume a little slower. Keep build-time state lean and let prefetch handle the disk side.
5. The ready hook decides what makes it into the snapshot. The platform captures the snapshot only after your /ready hook reports success, and readyTimeoutInSeconds is the budget for that. If ready answers before your build-time setup finishes, the snapshot captures a half-initialized VM and every future VM inherits it. Whatever build-time state you want baked in, sequence it ahead of ready.

S3 Files tricks. These hold wherever you mount S3 Files, snapshot or not:

1. The mount syscall is cheap. Sub-second on a warm system. If mounting looks slow, the cost is somewhere else.
2. Keep hardlink-dependent tools off the mount. The filesystem rejects hardlinks, and tools like uv rely on them for caches. Point their state at a local disk path instead of the NFS home.
3. Access points are your multi-tenancy primitive. One filesystem, one access point per user, each scoped to its own directory, giving isolation without one filesystem per user.

The intersection. S3 Files inside a snapshot-booted VM, where this post lives:

1. The mount toolchain is the cold start. mount.s3files is Python plus a TLS proxy, and on a fresh VM all of it demand-pages before any network work begins. That's why the validate workload has to touch it: it's the least obvious, most expensive thing on the cold path.
2. Bake what's shared, defer what's per-user, and fail safe when the per-user part is missing. The snapshot is shared across every user's VM, so the per-user mount can't live in it. It has to happen at run time, from the lifecycle hook. And when the per-user piece is absent, degrade safely instead of guessing.
3. Answer the lifecycle hook fast; do the slow work behind a readiness marker. The hook has a timeout measured in seconds; the mount has a retry budget measured in tens of seconds. Return 200 immediately, mount in the background, and gate the user-facing shell on a ready file.
4. Design for resume, not just run. Only the /run hook carries your payload; /resume arrives empty-handed. My /run hook persists the access-point id to a local file so /resume can re-mount the same user's home without being told whose VM it is. Get this right and resume is the fast path it should be: the toolchain pages are already in, so the remount takes about 2 seconds.

All of these tricks came out of a real, working project: a browser-based Claude Code sandbox I use every day. If you want to see the full build, that's the companion post: Lambda MicroVMs + S3 Files: My Claude Code Sandbox for iPad.

I would love to hear what you learn baking your own VMs. You can find all my socials at edjgeek.com.

#ServerlessForEveryone