DEV Community

ajithmanmu
ajithmanmu

Posted on

Running Untrusted Code on AWS Lambda MicroVMs

#serverless #aws #lambda #security

A malware scanner won't flag this code:

def transform(df):
    import os
    os.system("rm -rf /")
    return df
Enter fullscreen mode Exit fullscreen mode

It's perfectly valid Python. A malware scanner looks for known threats, and ordinary-looking code like this isn't one, so it passes.

Lambda MicroVMs are a new release from AWS for running code you don't trust. I built a small pipeline to try them out: tenants upload a CSV and a Python function to transform it, both untrusted.

Scanning doesn't help when the code itself is the risk. You run it in an isolated sandbox instead, and a Lambda MicroVM is that sandbox: a Firecracker virtual machine with its own kernel, isolated in hardware, booting from a snapshot in about a second. A disposable computer you hand to a stranger.

By default, though, a MicroVM can still reach the internet, and it runs with more access than it needs. Locking it down is what this post is about.

The software principle behind it is defense in depth: the untrusted code runs behind several independent layers - the sandbox, the sealed network, the stripped-down role, the malware scan — and an attack has to get past every one of them.

Scan the Data, Contain the Code

The pipeline handles two kinds of untrusted input, and each needs a different approach.

The file is data sitting at rest. A malicious upload like a virus-laden PDF or a zip bomb has a known shape, so a scanner can catch it. Scanning works here.

The code is the part a scanner can't help with, as we saw. So instead of trying to detect what it will do, you contain it: run it somewhere it can't do harm.

The design splits by what each input is: scan the data, contain the code. Here's the full pipeline.

architecture.png — end-to-end pipeline flow

Stack: Lambda MicroVMs (Firecracker), API Gateway, S3, GuardDuty Malware Protection for S3, EventBridge, Step Functions, Lambda, DynamoDB, Athena. Terraform for all of it.

The Security Layers

The untrusted code runs behind several layers, each handling a different way things could go wrong.

  • Hardware isolation. Every job runs in its own MicroVM, a Firecracker virtual machine with its own kernel. Jobs never share a machine.
  • Resource limits. The function runs in a child process with a memory cap and a wall-clock timeout. A runaway loop or a memory bomb kills its own process, not the sandbox.
  • A sealed network. The MicroVM's outbound traffic runs through a private VPC subnet with no internet gateway and no NAT. The route table only allows traffic inside the VPC, and a security group with no egress rules blocks the rest. Nothing can leave for the internet.
  • A least-privilege role. The sandbox's IAM role can do one thing: write logs. Even if the code reads its own credentials, they're worth nothing.
  • Per-tenant isolation. Each tenant's files are read through an S3 Access Point scoped to that tenant, so one tenant's job can't reach another tenant's data.
  • Malware scanning. GuardDuty scans every uploaded file before any code runs. A malicious file is rejected up front, and the code inside it never executes.

The network layer is deliberately two barriers. The security group denies outbound traffic, and even if a rule were wrong, the subnet has no route to the internet in the first place. Either one alone would stop a leak.

network.png — sealed network (VPC / subnet / security group / route table)

The Demos

I ran a clean job, a few malicious ones, and a malware file through the pipeline to see the layers work.

A clean job runs end to end. Valid data, valid code. The file is scanned, the transform runs in the sandbox, and the result is written to the clean bucket.

demo-success.png — successful run (full green path)

The sandbox contains bad code. A transform that allocates memory without limit hits the 512MB cap and gets killed. The code runs, but the sandbox catches the failure and records an error. Nothing reaches the clean bucket.

demo-contained.png — contained error (memory bomb killed at the cap)

An infinite loop is caught the same way, by the wall-clock timeout:

{"reason": "timeout", "detail": "exceeded 15s wall clock"}
Enter fullscreen mode Exit fullscreen mode

And a transform that tries to reach the internet gets nowhere, now that we've sealed the network:

OPEN egress   → "OPEN - reached 1.1.1.1:443"
SEALED egress → "BLOCKED - no route out"
Enter fullscreen mode Exit fullscreen mode

Malware never reaches the sandbox. Upload the EICAR test file and GuardDuty tags it THREATS_FOUND. The scan gate routes it straight to rejected, and the transform step is never entered.

demo-eicar.png — EICAR rejected, skips the sandbox

Tenants can't read each other's data. Reading one tenant's file through another tenant's access point:

{"result": "BLOCKED - AccessDenied"}
Enter fullscreen mode Exit fullscreen mode

Every run is recorded. The pipeline writes one of three verdicts for each job.

demo-ledger.png — DynamoDB verdict ledger (ok / error / rejected

  • rejected: the file was malware. The scan caught it.
  • error: the code misbehaved. Containment caught it.
  • ok: clean file, clean code. Promoted to the trusted bucket.

What I Learned

  • A MicroVM reaches the internet by default. Isolated doesn't mean sealed. The network lockdown is on you, and it's the layer that stops a leak.
  • CPU-time limits are unreliable on a throttled MicroVM. It gets a fraction of a vCPU, so CPU-seconds and wall-clock seconds drift apart. Use a wall-clock timeout.
  • Access points don't isolate on their own. Scoping the access point's policy wasn't enough. The tenant restriction also had to live in the IAM role and the bucket policy, because same-account access is the union of all three. Miss one and the isolation quietly does nothing.
  • The tooling lags the service. The AWS CLI and the SDK bundled into Lambda didn't know the new MicroVM API yet. Budget time to upgrade both before anything works.

Is It Worth It?

If you're running one trusted internal script, no. This is a lot of machinery for a problem you don't have.

If you're running code you didn't write, against data you can't afford to leak, then yes, and MicroVMs are the right tool. Plain Lambda reuses its environment between runs, and a container shares the host kernel, so neither is safe for hostile code. EC2 is isolated but puts you back to running servers. A MicroVM gives you that isolation with none of the overhead, and everything else in this post is the work of making it safe.

The full build is on GitHub: Terraform, the MicroVM container, the control plane, and every demo above. github.com/ajithmanmu/trust-nothing-data-pipeline

I'm an AWS Community Builder writing about serverless and cloud security. If you're building on MicroVMs or running untrusted code on AWS, I'd like to hear how you're handling it.

Top comments (0)