MeetUp Overview - Building & Scaling Game Dev in the Cloud

I recently had the pleasure of presenting at a meet-up where I delved into a topic that's both exciting and challenging: moving AAA game development into the cloud. For those who missed it, I wanted to take this opportunity to share some of the key insights and solutions discussed.

You can catch the recorded session on YouTube here:

The Evolution of Game Development

Game development has come a long way from its early days. Today’s AAA games are akin to blockbuster movies, characterised by their massive budgets, global reach, and complex production processes. The development of these high-profile games typically happens in dedicated studios equipped with cutting-edge hardware. These studios invest heavily in powerful gaming PCs, extensive graphics design tools, and vast amounts of storage.

However, as technology evolves, there’s an increasing push to leverage the cloud for game development. The cloud offers scalability, flexibility, and the potential for significant cost savings. But moving such a complex and resource-intensive process to the cloud presents its own set of challenges.

Current State of AAA Game Development

To set the stage, let’s clarify what a AAA game is. AAA games are high-profile, high-budget titles developed by major studios with the expectation of global success. They require extensive localisation, support for multiple platforms (such as Xbox and PlayStation), and involve large teams working on everything from graphics and coding to complex game logic.

Traditionally, the development of these games occurs in physical studios, where developers use high-spec gaming PCs. These machines come equipped with powerful GPUs, ample RAM, and fast CPUs, necessary to handle the enormous data volumes and intricate processes involved. Game development is often done in silos, with studios working in isolation to create content that eventually gets integrated into the final product. The development cycle is lengthy, sometimes taking years, and involves managing terabytes to petabytes of data.

Challenges & Solutions for Cloud-Based Game Development

Challenge 1 - Handling High Demands of Game Development

The Challenge: Game development requires high-performance, low-latency hardware. Developers, artists, and testers need access to powerful PCs with fast disk speeds, high-end GPUs, and substantial CPU resources. They also need peripherals and controllers to test and interact with the game seamlessly.

Our Solutions:

1. AWS EC2 GPU-Based Machines: We utilised powerful EC2 instances like G4 and G5, which provide the necessary computing power and graphics capabilities. These instances help deliver a desktop-like experience for game developers and artists working remotely.
2. Parsec: This technology enables low-latency remote desktop access by creating a peer-to-peer network between the host PC and the remote machine. Using UDP traffic, it streams data efficiently, minimising latency.
3. Architecture Design: We set up a Virtual Private Cloud (VPC) in AWS, with GPU EC2 machines and Parsec High Performance Relay integrated into the architecture. This setup ensures secure and efficient routing of traffic to minimise latency for users across different regions.

Challenge 2 - Data Transfer and Bandwidth

The Challenge: AAA games generate vast amounts of data. Managing this data, providing fast access to shared storage, and dealing with high CPU demands for compilation are crucial. Additionally, accessing physical gaming consoles for remote testing poses a significant challenge.

Our Solutions:

1. EC2s & SN DBS: We employed SN DBS technology to allow for distributed compilation by sharing CPU resources across multiple machines. This reduces build times significantly.
2. FSx for Cache: AWS FSx provides high-speed, managed storage that can be mounted to EC2 instances. This solution addresses the need for fast data access and cache management.
3. Site-to-Site VPN: To integrate with physical studios and consoles, we used a VPN to connect AWS infrastructure with on-premises systems, allowing remote access and testing.
4. Global Network Setup: AWS Transit Gateway with inter-region peering was used to create a global network. This setup enabled us to connect distributed development teams and studios effectively.

Challenge 3 - Building the Game

The Challenge: Building a game involves several time-consuming steps, including checking out the source code, compiling the Unreal Engine, cooking data assets, and packaging the game for distribution. Each step requires significant computational power and speed.

Our Solutions:

1. SN DBS with Autoscaling: We leveraged SN DBS for distributed compilation and autoscaling to dynamically adjust the number of machines based on the workload. This setup ensures efficient resource utilization and cost-effectiveness.
2. FSx for Shared Storage: Fast access to storage is critical for game builds. FSx provided high-speed cache and storage solutions to support the compilation and packaging processes.
3. TeamCity CI System: We used TeamCity for continuous integration and orchestration of the build process, enabling automated scaling and management of build resources.
4. Cost Optimisation: To manage costs effectively, we utilised EBS gp3 volumes with storage-optimised machines. By stripping multiple volumes together, we achieved high IO performance at a lower cost.

Conclusion

Transitioning AAA game development to the cloud is a complex but rewarding endeavour. The benefits include dynamic scaling, global distribution, and overcoming physical limitations of traditional studios. However, challenges such as high data volumes, demanding hardware requirements, and the need for physical console access remain significant hurdles.

Our approach involved a combination of advanced AWS technologies, strategic architectural design, and careful cost management to address these challenges. The result is a scalable, efficient cloud-based game development environment that can support multiple teams and global distribution.

For those interested in exploring cloud-based game development or discussing these solutions further, feel free to reach out. I’m always happy to share more insights and experiences.

Thank you for reading!