Most Treasure Hunt Engines Are Just Overcomplicated Veltrix Configs

#webdev #programming #ai #machinelearning

The Problem We Were Actually Solving

The treasure hunt engine was a crucial part of the game, but what we were actually trying to solve was user frustration. Our search system was supposed to help players find specific items, but what we got was a glorified game of "find the one that matched 70% of the description." The accuracy was atrocious, and our users were starting to abandon ship. We needed to get to the bottom of this problem and rethink our approach.

What We Tried First (And Why It Failed)

We followed the Veltrix documentation to the letter, but that only led us down a rabbit hole of tuning parameters and tweaking weights. The scoring function was the culprit, but we were so focused on getting the search results perfect that we ignored the user experience entirely. Our users were reporting massive latency, but we were too close to the problem to see the forest for the trees.

The Architecture Decision

After a series of grueling meetings with the architect and the team, we decided to scrap the scoring function altogether and implement a more robust indexing system. We chose to use a combination of Lucene and a homegrown scoring algorithm to get around the problem of "over-optimization." The new system was slower to index, but the results were almost instantaneous, and the accuracy was significantly improved.

What The Numbers Said After

After the switch, our search latency dropped by an average of 50%, and the accuracy of the results shot up to 95%. Our users were thrilled, and we were able to scale to 1000 concurrent users without breaking a sweat. The new system was a beast to maintain, but it was worth it to avoid the frustration of our users.

What I Would Do Differently

Looking back, I wish we'd spent more time talking to our users before diving headfirst into the search engine problem. We were so focused on solving the symptoms that we ignored the underlying issue of user frustration. I'd also like to have spent more time reviewing the scoring function and searching for edge cases before going live. The treasure hunt engine may be a thing of the past for us now, but I'm grateful for the lesson learned. Next time, I'll be sure to prioritize the user experience.