Building a Production-Grade Self-Hosted Code Execution Engine (Isolation, Limits, Observability)

Running untrusted code safely in production is one of those problems that looks simple — until you actually have to operate it.
Over the past year I’ve been building a self-hosted polyglot execution backend designed for real workloads rather than demos or sandboxes.
Some principles that proved essential:

1. Isolation must be structural, not optional Each execution runs inside its own container with strict CPU, memory, time, and output limits. No shared runtime, no trust assumptions.
2. Deterministic failure modes matter Missing toolchains, timeouts, or resource violations are explicit states — not crashes. Execution systems must fail predictably to be operable.
3. Stateless execution enables scaling Containers are ephemeral and disposable. The platform scales horizontally without session coupling.
4. Observability is part of the contract Health, diagnostics, capabilities, and structured execution logs are exposed as first-class endpoints. You can’t operate what you can’t see.
5. Safety limits define reliability Resource enforcement (ulimits, cgroups, container quotas) is not just security — it’s stability. Most outages in execution systems come from missing limits. This kind of architecture is useful for: online judges AI tool execution automation platforms education sandboxes CI runners If you’re working on similar infra or self-hosted execution systems, I’d love to exchange notes. GitHub: https://github.com/TotyLabs Demo: https://huggingface.co/TotyLabs/GozoLite