Cloud-based development environments are popular among companies of all sizes as a way to improve collaboration and productivity among engineering teams. But most people don’t realize how much software is built on remote development environments. In this post, we want to share a dozen examples of companies that have made the transition, why they did it, and who led the change.
We’re doing this because even as founders in this space, it’s often hard to explain how and why companies make the transition. This is partly because engineering orgs often have an allergic reaction to productivity measurements. Still, the more significant issue is that developing on cloud-based environments is a different way of working: it’s hard to compare, and the driving force behind the decision differs at many companies.
If you want to jump straight to the sources, summaries, and info - you can jump straight to the “Companies building software on Cloud Environments” section. But first, a quick overview of what we’re seeing.
What companies?
Companies of all sizes, but the tipping point is 50 to 70 engineers. With 100 engineers, the benefit becomes clear to engineering leaders. Our case studies are biased toward more prominent companies (i.e., the ones with resources to maintain an engineering blog).
Why did they make the switch?
Primarily to simplify workflows and testing, unlock more computing power/memory to cut build, compile, and test times, and improve stability and consistency. Teams also found they could run more microservices in development, speed up onboarding times for new projects and new engineers, and reduce the context engineers need to get work done.
Who leads these initiatives?
At small companies, this is often led by engineering leaders. This is owned by DevOps/SRE, Developer Experience or Productivity, or Infrastructure teams at mid and larger companies.
What were common concerns?
Teams were worried about high latency, coding without the internet, changing workflows, and tool lock-in. Much of this stems from experience with Cloud IDEs in the past, but with products like Nimbus, you can have a local-like experience with remote development. An internet connection is required, but teams opt not to optimize for rare “coding on a flight scenario.”
Are teams buying or building?
Most of these examples have homegrown tooling, but more eng leaders are buying tooling as the market and product develop quickly. In the past, solutions failed because they limited tooling options (e.g., VSCode only) and couldn’t deploy on their customer’s cloud. But with this change, we’re seeing teams look to replace their internal tooling to relieve engineering capacity and focus on core products.
Companies building software on Cloud Environments:
Stytch - 30 engineers
- Read about it here: https://stytch.com/blog/remote-dev-1/
- Pains: Development needed too many services, context, people, and setup time. Issues from differing production and local configs. Difficulty in testing.
- Summary: Stytch found five primary productivity drags from local development. This included: too many services, too much context needed, development was too people-intensive, and config and testing issues. They considered 3 options instead of moving to the cloud but realized that cloud-based development was the only way to meet all their requirements. Stytch found that centralization, scale, and efficiency were the most significant benefits of remote development.
Doximity - 200 engineers
- Read about it here: https://technology.doximity.com/articles/github-codespaces-is-great-but-mutagen-is-better
- Pains: Docker Desktop for Mac was slow and resource-intensive, and lack of observability into local development issues
- Led by: Site Reliability Engineering
- Summary: The SRE team created Dox Compose Cloud, a standardized remote development environment that gives increased visibility and uniformity across development and production environments. The tool is similar to GitHub Codespaces, but Dox Compose Cloud allows users to continue developing on their local machine using their editor of choice and allows SRE teams to have visibility over development environments and resolve issues more quickly.
Monday - 230 engineers
- Read about it here: https://engineering.monday.com/development-environments-in-the-cloud/
- Pains: Slow onboarding, resource-heavy docker composition, could not run more than a few microservices, and difficulty maintaining environments
- Led by: Developer Experience
- Summary: Monday.com used a third-party tool to set up cloud-based development environments. This was done for their monolith and 30 microservices, as well as Ambassador (its HTTP gateway). Monday.com’s tooling enabled developers to work on their Kubernetes environments, replicas of production environments, without the need to install or maintain tools locally. This improved the time it takes to spin up the environment from scratch, reduced resource usage, and provided a more consistent development experience.
Pipedrive - 300 engineers
- Read about it here: https://medium.com/pipedrive-engineering/dev-environment-evolution-in-pipedrive-from-a-developers-perspective-cdc8a5b7e3c1
- Pains: Stability and consistency issues, development environments were resource intensive, slow builds, complex setup, unintuitive alternative solutions
- Led by: Developer Experience
- Summary: Pipedrive’s dev environment evolved from a Docker workstation to a Kubernetes workstation to using Devbox (testboxes) to transfer the most demanding components of the dev environments to the cloud. The switch allowed for better parity between environments, resolved dependencies between microservices, and reduced developer hours troubleshooting dev env issues. It also allowed for faster onboarding and easier maintenance of the dev environment.
Plaid - 400 engineers
- Read about it here: https://plaid.com/blog/scaling-with-devenv/
- Pains: Docker Desktop for Mac was slow and resource-intensive, had slow build times and Docker image pull times, and poor development experience and reliability.
- Led by: Developer Efficiency Team
- Summary: Plaid’s Developer Efficiency Team decided to create “Devenv,” an internal CLI tool backed by a remote environment, to address several scaling challenges they were experiencing. Originally Devenv was built for local development but was moved to the cloud (backed by a standardized set of EC2 instances). Devenv allows developers to set up and validate the development environment; clone and update repositories; start, stop, and rebuild dockerized services; run unit and integration tests; perform lint checks. Nearly 2 years after the rollout of remote devenv, 87% of users exclusively use remote devenv, and 97% of all commands are run on remote devenv.
Rippling - 500 engineers
- Read about it here: https://www.rippling.com/blog/how-we-used-the-cloud-to-supercharge-remote-development
- Pains: Limited hardware resources, slow onboarding, slow testing times, environment consistency
- Led by: Engineering Shared Services
- Summary: Rippling’s application is a massive Python monolith that has grown quickly. The size and complexity made it slow and unwieldy for local development. The team experimented with third-party remote solutions but decided to build this in-house because they didn’t like the browser-based solutions they evaluated. They ultimately created a solution that synced local code changes to remote EC2 servers where the code would be executed. The main benefits were a simplified and standardized setup that accelerated onboarding and faster app and test running times.
Slack - 1400 engineers
- Read about it here: https://slack.engineering/remote-development-at-slack/
- Pains: Fragile and inconsistent environments, slow build times, and slow setup times
- Led by: Developer Productivity
- Summary: Slack’s engineering team wanted to improve the consistency and speed at which engineers could write code. As a starting point, they updated their rich CLI tools to build features that let them interact with dev environments. They found that putting environments on EC2 servers did more than just cut down setup - it simplified workflows, opened new possibilities, cut bootstrap times, and allowed engineers to focus on writing code without worrying about environment setup and maintenance.
Palantir - 1500 engineers
- Read about it here: https://blog.palantir.com/the-benefits-of-remote-ephemeral-workspaces-1a1251ed6e53
- Pains: Lack of local compute resources, slow compile and clone times, felt productivity was limited
- Led by: DevOps
- Summary: Palantir was always a fan of ephemeral environments, and they discuss the benefits of working this way in a large enterprise environment. At Palantir, these cloud-based workspaces improved local build times, increased developer productivity, and reduced onboarding costs. The article also covers the complexities and challenges of implementing this type of environment and how Palantir addressed them. Interestingly, Palantir saw that this might be more expensive than local development but that productivity gains offset the costs.
GitHub - 2500 engineers
- Read about it here: https://github.blog/2021-08-11-githubs-engineering-team-moved-codespaces/
- Pains: Long build times
- Summary: GitHub used Codespaces to migrate its core repository to a remote development environment. This has shortened build times (from 45 minutes to 10 seconds), reduced the time required to recover from local development breakdowns, and improved collaboration. Developers can access development environments from any device using the remote development environment (via VSCode).
- Full disclosure: Codespaces is a competitor to Nimbus.
Shopify - 4500 engineers
- Read about it here: https://shopify.engineering/shopifys-cloud-development-journey
- Pains: Project and product complexity had grown, there were huge monoliths, and developers lacked local resources to work effectively. Additionally, it took a long time to ship software, and it was slow to reproduce environments and collaborate.
- Led by: Developer Tooling and Infrastructure
- Summary: Shopify outgrew its local development environment when the products started becoming too complex, and it became slow to support developers. Shopify’s written about this many times, but in this article, they wrote about how they used a remote development environment supported by a lightweight virtual machine to deliver development environments. This allows developers to partition their work and better control supporting processes, improving performance and reducing shipping times. The remote development environment enables developers to collaborate and validate changes in running applications without duplicating the entire environment.
LinkedIn - 8000 engineers
- Read about it here: https://engineering.linkedin.com/blog/2021/building-in-the-cloud-with-remote-development
- Pains: Long build times
- Led by: Developer Productivity and Happiness
- Summary: By utilizing a remote development environment, LinkedIn’s Developer Productivity and Happiness team were able to cut the initial setup and build times for the majority of its products from up to 30 minutes to 10 seconds. The goal of the Remote Programming effort was to give all developers access to quick, dependable, consistent, and straightforward remote development environments. These remote development environments, or RDevs, are generated on fast hardware in LinkedIn’s private cloud and are containers set up for a specific product. It’s interesting to note that LinkedIn created this internally without using Codespaces.
Top comments (0)