Ephemeral Environments: A Getting Started Guide

The article explores the differences between traditional persistent staging environments and modern ephemeral environments for software testing. It outlines the challenges of using shared persistent environments, including infrastructure overhead, queueing delays, and risk of "big bang" changes. In contrast, ephemeral environments provide automated setup, isolation, and easy creation/deletion. The article then gives instructions for implementing ephemeral environments on your own or using an environment-as-a-service solution to simplify the process.

The Challenges of Traditional Environments

Ideally, code changes should be tested in an environment identical to production before going live. However, achieving this with traditional persistent staging environments comes with several drawbacks in practice.

Infrastructure Overhead

The staging environment must replicate all production infrastructure components like frontends, backends, databases, etc. This means extra work to maintain and coordinate infrastructure changes across both environments. It's easy for staging to diverge from production if infrastructure changes are forgotten or not perfectly mirrored.

Queueing Delays

With only one staging environment, developers must wait their turn before changes can be deployed. This hinders release velocity and productivity. Some developers may attempt risky workarounds to release faster, causing problems from untested changes.

Potential for "Big Bang" Changes

If changes are not consistently deployed from staging to production, staging can become significantly ahead. This means finally deploying to production contains multiple commits at once, increasing the risk of something breaking.

These challenges show why traditional environments often fail to enable safe testing as intended. Modern ephemeral environments provide an improved approach.

The Benefits of Ephemeral Environments

Ephemeral environments offer several key advantages over traditional persistent staging environments.

Automated Infrastructure

Ephemeral environments spin up on-demand, automatically creating required infrastructure to match the current production setup. This ensures consistency without manual intervention from engineers. Broken environments can be quickly replaced.

Complete Isolation

Each pull request gets its own freshly spawned environment running in parallel. This eliminates queueing delays and allows testing without side effects from other changes. No risky "big bang" deployments to production.

Short Life Span

Ephemeral environments exist only as long as needed, configurable to be created when a pull request opens and destroyed when it merges. No more paying for unused environments, driving major cost savings.

Together, these benefits empower developers to test safely and release rapidly. Ephemeral environments address the common pitfalls of traditional setups.

Implementing Ephemeral Environments

Setting up ephemeral environments involves some initial work, but the payoff is significant.

Prerequisites

Some key infrastructure must already exist:

• Containerized service instances (e.g. Docker, Kubernetes) for easy spin up/tear down
• CI/CD pipeline managing deployment and code integration

Configuration Steps

Main implementation steps:

1. Set up production infrastructure declaratively
2. Create test database with sample data
3. Add declarative infrastructure with dynamic naming based on branches/commits
4. Trigger deployment of full stack in CI/CD pipeline
5. Generate secure URL for accessing deployed instance
6. Replace old environments with new ones when code updates
7. Configure auto-removal after inactivity periods
8. Prevent direct deployment to production from pipeline
9. Add manual production deployment trigger These steps simplify the workflow, but fully automating ephemeral environments still requires significant initial effort.

Conclusion

In summary, ephemeral environments provide modern solutions to longstanding challenges with traditional persistent staging environments. By automating provisioning and tearing down isolated environments on demand, they enable rapid and safe iteration without queueing delays or infrastructure overhead.

Implementing ephemeral environments does require an upfront investment, including adopting declarative infrastructure, CI/CD pipelines, and containerization. However, the long-term productivity and stability gains make it worthwhile for most development teams.

For those lacking the resources to build their own ephemeral environment workflow, environment-as-a-service solutions like Coherence handle the complexity automatically. With integrations, automation, and easy management, these services allow focusing on core product work rather than infrastructure.

As software delivery accelerates, development teams need agile testing environments that move as fast as they do. Ephemeral environments deliver on that need and promise to become the new standard for pre-production testing and review.

Read more at https://www.withcoherence.com/post/ephemeral-environments.