- 3× engineering org growth — the scale at which "works on my machine" became a structural failure
- Mac → V1 → V2 — two complete migrations over ~3 years; the first one hit a wall
- 0 support tickets about network latency and connection drops after migrating to Coder V2
- ~100% of backend and infrastructure engineers on cloud development environments
- Tailscale/WireGuard mesh networking — why V2 felt like a local machine when V1 did not
- Frontend engineers kept on local machines — pragmatic carve-out where locality genuinely mattered
Discord's engineering team had tripled in size and was drowning in "works on my machine" bugs — some engineers on macOS, some on Ubuntu, all of them slowly. The solution was radical: no one gets a local dev environment anymore. Except frontend. That's its own story.
The Story
Despite the challenges and the need for two migrations (Mac→V1→V2), our move to remote dev machines using Coder has been remarkably successful. The timing was fortuitous, as we embarked on this journey before the pandemic began.
— Denbeigh Stevens, Senior Software Engineer, via Discord Engineering Blog
For most of Discord's early history, backend engineers set up development environments on their personal laptops — primarily MacBooks, some Ubuntu. This was manageable when the engineering team was small. As Discord's product grew and the engineering organisation tripled in headcount, the cracks became structural failures. Homebrew (a popular package manager for macOS that installs open source software but lacks guarantees of reproducibility across machines) upgrades would silently break an engineer's dev setup. New team members spent their first week not shipping code but untangling environment issues unique to their laptop. The DevEx team accumulated a growing backlog of one-off tickets that amounted to: your environment is subtly different from everyone else's, and we have to figure out why. There was no single source of truth for what a correct Discord development environment looked like.
The solution was radical in its simplicity: stop maintaining two local environments and move all backend and infrastructure development to a single Linux-based Cloud Development Environment (a remote machine running in a cloud provider's data centre that developers access via their editor's remote extension, giving them full Linux capabilities without managing local hardware). Discord evaluated Coder (an open-source platform for creating and managing cloud development environments at scale, providing templated workspace provisioning, lifecycle management, and developer access controls) in late 2020. The partnership began, and Discord started the first of what would turn out to be two separate migrations.
Problem
Two Environments, Infinite Edge Cases
As Discord's engineering organisation tripled, the DevEx team found itself firefighting unreproducible environment issues specific to individual MacBooks. Homebrew upgrades broke setups silently. Ubuntu engineers had subtly different library versions. No two environments were truly identical, making debugging a game of "is this a code bug or a local environment bug?"
Cause
Scale Broke the MacBook Model
The SDLC (Software Development Lifecycle — the full process from writing code to shipping it, including build, test, and deployment) had grown too complex for unmanaged local environments. Discord's backend requires a highly complex environment with many moving parts. Running it inside Sysbox containers on Kubernetes V1 introduced layers of virtualisation that were difficult to debug when things went wrong.
Solution
Coder V1 → V2: From Containers to VMs
The initial Coder V1 migration moved engineers to Kubernetes-based container environments. Networking latency and frequent disconnections plagued engineers outside San Francisco. In 2023, Discord migrated to Coder V2, which replaced the Kubernetes-container model with full VMs using Tailscale and WireGuard for networking — dramatically more stable and performant.
Result
Zero Support Tickets About Connection Drops
After V2, Discord stopped receiving support tickets and questions about high latency and connection drops entirely. Engineers reported that development felt faster and smoother. The DevEx team stopped spending time on "works on my machine" debugging and started spending time on tooling improvements that actually moved the needle.
The Fix
The Two-Migration Architecture
Discord's CDE story is not a clean one-migration success. It required two complete migrations: from MacBooks to Coder V1 (Kubernetes), then from Coder V1 to Coder V2 (VMs). This reframes the story from "we had a good idea and executed it" to "we had a good idea, hit a wall, and had the organisational courage to do it again better."
- 3× — engineering organisation growth that made local environment maintenance untenable
- 0 — support tickets about network latency and connection drops after migrating to Coder V2
- Mac→V1→V2 — two complete developer environment migrations over ~3 years
- ~100% — backend and infrastructure engineers now on cloud development environments; frontend engineers retained local machines due to asset transfer latency
# Simplified Coder V2 workspace template (Terraform)
# Discord provisions Linux VMs for each engineer via this kind of template
resource "coder_workspace" "discord_backend" {
# Each engineer gets their own dedicated VM
name = "${data.coder_workspace.me.name}-backend"
# Full VM — not a container; no virtualisation layers to debug through
instance_type = "n2-standard-8" # 8 vCPU, 32GB RAM
disk_size_gb = 100
# /home is persistent — survives template updates and restarts
# Engineers keep their repos, configs, and customisations
persistent_home = true
# Tailscale/WireGuard handles secure network tunnel
# Engineer's laptop <--> VM via encrypted peer-to-peer mesh
# Direct path, not hub-and-spoke through a central gateway
network = "tailscale-mesh"
# Standard Discord development image — same for every engineer
# DevEx team ships updates as image updates; everyone gets them simultaneously
image = "discord/devenv:latest"
service_account = "dev-env-sa@discord-dev.iam.gserviceaccount.com"
}
# VS Code remote extension connects to this VM
# Engineers experience it as if it were a local machine
# No more "works on my machine" — there is only one machine definition
One pragmatic concession emerged: frontend engineers who worked heavily with large HTML and JavaScript files found that the network overhead of transferring those assets during live editing created noticeable latency in their save-and-rebuild loops. The DevEx team made a deliberate exception — frontend work was excluded from the mandatory CDE migration, with those engineers continuing to develop locally. This was not a failure of the approach; it was an honest acknowledgement that different workloads have different locality requirements.
The /home directory persistence strategy
One of Discord's key architectural decisions for developer experience was keeping the /home directory persistent across VM restarts and template updates. Engineers could update templates and base images without losing their repositories, settings, personal tools, and workspace customisations. This made CDEs feel like a machine that belonged to them rather than an ephemeral container that could be wiped at any time. Template updates ship like container image updates — everyone gets the new tooling simultaneously while their working state is preserved.
The pandemic timing advantage
Discord began the CDE migration in late 2020 — just as the pandemic forced most tech companies to distribute their engineering teams globally. Because Discord had already committed to the cloud dev environment path, they were better positioned than most to operate as a fully distributed engineering organisation. Engineers could spin up identical development environments from anywhere in the world without IT shipping them a configured MacBook. What looked like an infrastructure investment turned out to be a resilience investment too.
The communication gap they would fix
Despite all-hands announcements, early signalling, and a thorough beta period, Discord's DevEx team acknowledged they could have communicated the migration better. Engineers discovered gaps in documentation only after cutover — a pattern common in large infrastructure migrations. Requesting hundreds of engineers to overhaul their entire development workflow is a major ask, and the team would invest more in change management communications and pre-migration network diagnostics next time.
Architecture
Before the migration, Discord's development workflow had engineers accessing services running on their local machines — or attempting to replicate the production environment locally, with all the version drift that entailed. After V2, every backend engineer has a templated VM in the cloud connected via Tailscale/WireGuard, accessed through VS Code's remote extension. The DevEx team manages one base image, ships updates once, and every engineer gets them simultaneously.
Before: Local MacBook/Ubuntu Dev Environment (Non-Reproducible)
View interactive diagram on TechLogStack →
Interactive diagram available on TechLogStack (link above).
After: Coder V2 Cloud Development Environment Architecture
View interactive diagram on TechLogStack →
Interactive diagram available on TechLogStack (link above).
Lessons
Reproducibility is a prerequisite for scale. Local environments drift silently — Homebrew upgrades, OS patches, personal tooling installs — and the drift compounds as headcount grows. If your DevEx team spends more time on per-engineer environment debugging than on tooling improvements, that is the signal you have outgrown local development.
Cloud Development Environments (remote virtual machines hosted in a cloud provider that developers access via their editor's remote extension) do not magically fix everything. V1 on Kubernetes introduced its own complexity via container layering. Always validate that your chosen CDE solution matches your workload's complexity requirements, and be willing to migrate again if the first choice proves incorrect.
Invest in network diagnostics before go-live, not after. When you move development to a networked environment, the developer experience is only as good as the network between laptop and VM. Discord built latency diagnostics after the V1 migration rather than before, leaving them partially blind to worst-case experiences during the transition. Network observability for your CDE is a first-class requirement.
Recruit internal champions from diverse engineering disciplines before starting a large tooling migration. A homogeneous beta group of enthusiastic volunteers will miss the edge cases that matter to the median engineer. Champions from backend, infra, mobile, and data teams surface a diversity of failure modes early enough to fix them before you've annoyed the whole company.
Not all workloads belong in the cloud. Discord's frontend engineers stayed on local machines because asset transfer latency genuinely hurt their experience. Pragmatic carve-outs that acknowledge real workload differences are better engineering than dogmatic 100% migrations that quietly make a subset of people miserable.
Engineering Glossary
Cloud Development Environment (CDE) — a remote virtual machine hosted in a cloud provider's data centre that developers access via their editor's remote extension (VS Code remote, JetBrains Gateway). Gives developers full Linux capabilities without managing local hardware or dealing with per-machine environment drift.
Coder — an open-source platform for creating and managing cloud development environments at scale, providing templated workspace provisioning, lifecycle management, and developer access controls. Discord used Coder V1 (Kubernetes-based) and then migrated to Coder V2 (VM-based).
Homebrew — a popular package manager for macOS that installs open source software. Lacks guarantees of reproducibility across machines — Homebrew upgrades can silently change library versions, breaking development setups in ways that are difficult to reproduce or debug systematically.
SDLC (Software Development Lifecycle) — the full process from writing code to shipping it, including build, test, and deployment. A sufficiently complex SDLC makes unmanaged local environments increasingly difficult to maintain reproducibly as engineering headcount grows.
Sysbox — a container runtime that allows running full operating systems inside Docker containers by emulating kernel features. Used by Coder V1 to simulate a full Linux environment inside Kubernetes pods. Required running Docker-in-Docker for Discord's development workflow, creating multiple virtualisation layers that complicated debugging.
Tailscale — a mesh VPN built on WireGuard that routes traffic directly peer-to-peer between devices rather than through a central gateway. Used by Coder V2 for direct encrypted tunnels between engineers' laptops and their cloud VMs, eliminating the hub-and-spoke latency of traditional VPN architectures.
WireGuard — a modern, minimal VPN protocol built directly into the Linux kernel, providing lower latency than OpenVPN or IPSec with a drastically smaller codebase. The underlying transport for Tailscale's peer-to-peer mesh networking.
This case is a plain-English retelling of publicly available engineering material.
Read the full case on TechLogStack →
(Interactive diagrams, source links, and the full reader experience)
TechLogStack — built at scale, broken in public, rebuilt by engineers.
Top comments (0)