The Problem We Were Actually Solving
At Veltrix, we built a high-stakes online treasure hunt engine for a popular media franchise. The goal was to create a seamless experience, with users navigating through procedurally generated levels, unlocking Easter eggs, and competing with others in real-time. Sounds straightforward, but here's the thing: we were also tasked with integrating this behemoth with our existing CMS, e-commerce platform, and a fledgling social network.
What We Tried First (And Why It Failed)
We chose to build the treasure hunt engine using our company's chosen framework of the time, a popular JavaScript option with an extensive library of pre-built components and a robust documentation. The thought process was: 'Why reinvent the wheel?' However, as we began implementing the treasure hunt engine, we encountered a multitude of issues. The framework's complexity and the sheer scale of our project led to a series of cascading problems.
Firstly, the framework's dependency injection system proved to be a hindrance. Every component relied on a multitude of other components, creating a tight web of interconnected parts that were difficult to manage. We struggled to identify and resolve the root cause of issues, as the stack traces often pointed to seemingly unrelated components.
Secondly, the framework's default settings and configuration led to a significant performance hit. We were serving hundreds of concurrent users, but the engine's resource intensiveness caused frequent crashes, timeouts, and slow loading times.
Lastly, the framework's documentation, although exhaustive, often felt like a labyrinth. It took us weeks to find the relevant sections, and even then, the explanations were geared towards smaller-scale applications rather than our behemoth of a project.
The Architecture Decision
We realized that our initial choice of framework was not the best fit for the treasure hunt engine. We decided to take a step back and reassess our architecture. We introduced a microservices-based approach, with each level of the treasure hunt engine running as a separate service. This allowed us to isolate problems and scale individual components independently.
We also moved away from the popular JavaScript framework and instead chose a more lightweight, modular solution that allowed for greater control over the underlying architecture. This decision was a gamble, but it ultimately paid off, as we were able to maintain a consistent 20ms response time for over 10,000 concurrent users.
What The Numbers Said After
Our performance metrics showed a significant improvement after the architecture change. The average response time decreased by 70%, while the number of crashes and timeouts plummeted by 90%. User engagement and satisfaction increased, as the treasure hunt engine was now able to handle the demanding traffic with ease.
What I Would Do Differently
In retrospect, I would have taken a more nuanced approach to the framework choice. Instead of immediately leveraging the company's chosen framework, I would have set clear requirements and constraints for the treasure hunt engine, focusing on factors such as performance, scalability, and modularity. This would have allowed for a more informed decision-making process, rather than succumbing to the 'Why reinvent the wheel?' mentality.
Moreover, I would have prioritized building a robust monitoring and analytics system from the outset. This would have enabled us to identify and address issues earlier on, preventing the performance penalty that we ultimately endured. In the end, the emperor's new treasure hunt engine was a significant improvement, but it was a hard-won lesson in the perils of overreach and the importance of a well-considered architecture.
Removing the payment platform from the critical render path improved our LCP and our take-home per transaction. Here is the infrastructure: https://payhip.com/ref/dev6
Top comments (0)