How I Ran a Containerized Playwright Scraper on a Free VM — and What Actually Broke

The goal was simple: download data on a schedule, automatically, with no local machine involved. No manual triggers, no keeping a laptop on overnight. Just a container, a free VM, and a cron job.

Getting there took longer than expected. Here's what actually broke and how I fixed it.

The setup

• Oracle Cloud Always Free AMD micro VM (Ubuntu 24.04, x86_64, ~954MB RAM)
• Playwright running Microsoft Edge (headless, via Xvfb) inside a Docker container
• Two .NET apps — a Playwright downloader and a CSV consolidator — combined into one multi-stage image
• GHCR as the image registry
• Cron for scheduling on the VM itself — no CI, no external orchestrator

The build and push happen from a local WSL machine via VS Code tasks. The VM just pulls and runs.

Why containerize at all?

The alternative was installing the .NET SDK directly on the VM and running the app with dotnet run. That works, but it ties the VM to a specific runtime version, makes the setup hard to reproduce, and diverges from how the rest of the project is deployed.

Containerizing means the VM needs exactly two things: Docker and a cron job. Everything else — the SDK, Xvfb, Edge, both apps — lives inside the image. If the VM is rebuilt or replaced, setup is: install Docker, add the cron entry, pull the image, done.

The first real problem: OOM kills

The AMD Always Free shape has ~954MB RAM and zero swap by default. Playwright launching a real browser under load hits memory hard. Without swap, the OOM killer terminates the browser process outright — no graceful degradation, no error message in the logs, just a hard kill.

The fix is a 2GB swap file, persisted to /etc/fstab so it survives reboots:

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

After this, sustained load across a full run completed with OOMKilled: false. This isn't a one-time workaround — it's a permanent property of the VM shape.

One other thing worth noting: the free ARM64 shape (Ampere A1) has more RAM and would avoid the swap requirement entirely — but Microsoft Edge has no native Linux ARM64 build. Staying on AMD x86_64 was the only viable path for this stack.

The second problem: xvfb-run hangs silently

Playwright needs a display. The standard approach is xvfb-run, but in a minimal container image it hangs indefinitely with zero output.

The reason: xvfb-run uses xdpyinfo (from x11-utils) to confirm the virtual display is ready before handing off. That package isn't installed by default. Xvfb itself starts fine — the wrapper just never moves past its own readiness check, silently.

The fix: start Xvfb manually in the entrypoint, poll for the actual X11 socket file, then export DISPLAY and exec the app directly:

Xvfb :99 -screen 0 1280x800x24 &
while [ ! -S /tmp/.X11-unix/X99 ]; do sleep 0.1; done
export DISPLAY=:99
exec "$@"

No dependency on xdpyinfo, no silent hang.

The third problem: output vanishing silently

The apps write files to paths configured in appsettings.json. Those paths are fine on a dev machine. Inside a container, the app creates those paths on its own ephemeral filesystem — and when the container is removed, the files go with it. Logs still report success. Only running ls on the host mount folder reveals the problem.

The fix: both apps call .AddEnvironmentVariables() after .AddJsonFile(...) so environment variables override config file values. The Dockerfile sets container-native ENV defaults for all output paths. The bind mounts in docker run wire the container paths to real host folders:

docker run \
  --name my-pipeline \
  -v /opt/myapp/downloads:/app/downloads \
  -v /opt/myapp/diagnostics:/app/diagnostics \
  -v /opt/myapp/logs:/app/logs \
  -v /opt/myapp/consolidated:/app/consolidated \
  -v /opt/myapp/archive:/app/archive \
  ghcr.io/username/my-pipeline:latest

One subtle catch: Docker won't auto-create bind-mount source directories reliably. Create them explicitly with mkdir -p before the first run, or output silently goes to the container's own filesystem. This is now part of the run task — the directories are created automatically before docker run fires.

The fourth problem: the entrypoint script encoding traps

The entrypoint is a shell script. Copying it from a host file into the image hit two separate encoding issues, both producing the same error — exec format error (exit 255):

1. CRLF line endings — Windows editors save files with \r\n. The \r becomes part of the interpreter name in the shebang line, breaking it.
2. UTF-8 BOM — some editors prepend a BOM before the shebang. The kernel sees the BOM first and can't identify the interpreter.

The standard fix for CRLF is sed 's/\r$//' — but this does not fix a BOM. A BOM-aware sed pattern can also silently fail under a UTF-8 locale. Both issues produce the same exit code, making it easy to think you've fixed one when the other is still present.

The actual fix: write the entrypoint directly in the Dockerfile via a heredoc:

COPY <<'EOF' /app/entrypoint.sh
#!/bin/sh
Xvfb :99 -screen 0 1280x800x24 &
while [ ! -S /tmp/.X11-unix/X99 ]; do sleep 0.1; done
export DISPLAY=:99
exec "$@"
EOF
RUN chmod +x /app/entrypoint.sh

The heredoc guarantees LF line endings and no BOM regardless of host OS or editor. The encoding problem simply can't occur.

Scheduling: cron on the VM itself

No CI runner, no GitHub Actions schedule, no external trigger — cron calls docker run directly on the VM. The local machine doesn't need to be on.

One thing to know: cron is not installed by default on minimal Ubuntu cloud images. It needs to be installed and enabled explicitly:

sudo apt-get install -y cron
sudo systemctl enable cron
sudo systemctl start cron

The cron entry cleans up any leftover container from a previous run before starting a new one:

30 1 * * 2-6 docker rm -f my-pipeline 2>/dev/null; docker run --name my-pipeline [mounts and flags] >> /opt/myapp/logs/cron.log 2>&1

All container output is appended to a cron log — one place to check after any scheduled run.

What's proven

After a full run across all configured targets:

• Exit code 0
• 78 files saved, 0 failed, 0 skipped
• OOMKilled: false

The 2GB swap held under sustained load. The pipeline now runs on schedule with no local machine involvement.

What's still open

A few things are deliberately deferred:

• Off-site backup — currently data lives only on the VM's local disk. A Google Drive backup via service account is planned but not started.
• Diagnostics under real failure — the screenshot/HTML capture on Playwright startup failure is wired up but hasn't been verified end-to-end under a real failure yet.

None of these block the automated pipeline. They're next.

---

If you've hit any of these in your own container/VM setup — especially the Xvfb hang or the BOM trap — I'd be curious what your fix looked like.