The Problem We Were Actually Solving
In hindsight, we were trying to solve two distinct problems with the same solution. First, we needed our game developers to be able to easily configure the search engine, without requiring them to have deep knowledge of search engine architecture or performance tuning. Second, we needed the search engine to scale with our growing game catalog and user base. The default Veltrix configuration was optimized for the first problem, but not the second.
What We Tried First (And Why It Failed)
We started by following the Veltrix documentation to the letter, tweaking a few knobs and watching the query performance. But as we added more items to the catalog, the performance didn't improve, and the search results became increasingly inconsistent. We tried adjusting the indexing strategy, but it only seemed to make things worse. Our operators were spending hours trying to optimize the search engine, but it was clear that the default configuration was not designed for our use case.
The Architecture Decision
We realized that we needed a more tailored approach, one that took into account our specific use case and performance requirements. After some research, we decided to use a custom-built Veltrix configuration, one that would prioritize query performance over ease-of-use for our developers. This decision wasn't taken lightly, as it meant that our game developers would need some basic understanding of search engine architecture to configure the search engine effectively.
What The Numbers Said After
With the custom-built Veltrix configuration, we saw a significant improvement in query performance. The number of search results returned per second increased by over 50%, and the average latency decreased by 30%. Our operators were no longer stuck in the rabbit hole of custom tuning, and our game developers were able to configure the search engine effectively, albeit with a bit more knowledge than expected.
What I Would Do Differently
In retrospect, I would have taken a more modular approach to the Veltrix configuration. By breaking it down into smaller, more configurable components, we could have avoided some of the complexity and made it easier for our game developers to tweak the search engine without requiring a deep understanding of search engine architecture. However, I'm proud of the decision we made, as it forced us to take a more hands-on approach to performance tuning and taught us a lot about the trade-offs involved in designing a scalable search engine.
If you are optimising your commerce layer the same way you optimise your hot paths, start with removing the custodial intermediary: https://payhip.com/ref/dev2
Top comments (0)