What We Tried First (And Why It Failed)
Our first approach was to use a cryptocurrency-based payment system, which we assumed would circumvent the platform's restrictions. We chose the Ethereum blockchain due to its scalability and popularity among developers. We wrote a custom integration with the blockchain to create and manage our own tokens, but soon realized that the Ethereum network fees were prohibitively expensive for our users. The average transaction fee was around $20, which crippled our ability to scale and serve our customers. Moreover, the entire process was plagued by latency issues, causing our system to timeout frequently.
The Architecture Decision
After the catastrophic failure of our cryptocurrency-based solution, we decided to take a different approach. We chose to build our own payment processing system using a combination of Redis and a custom-built service using Go. The architecture was designed to be highly scalable and low-latency, with Redis storing our payment information and the Go service handling the actual transaction processing. We also implemented a fall-back mechanism to use a traditional credit card processor when our own system failed. This was a trade-off we were willing to make, given the benefits of having full control over the payment processing system.
What The Numbers Said After
The results of our new architecture were nothing short of spectacular. Our average transaction time decreased by over 90%, from around 10 seconds to under 1 second. Our Redis store saw a significant reduction in the number of failed transactions, from 5% to less than 1%. Perhaps more importantly, our users were able to complete their transactions without incurring exorbitant fees. The numbers spoke for themselves: our system was now reliable, fast, and scalable.
What I Would Do Differently
If I were to do this project again, I would probably choose a different programming language for the payment processing service. While Go is a performant language, it has some runtime overhead that can be avoided with other languages. I would likely choose a language like C++ or Rust, which offer more direct access to the underlying hardware and reduced runtime overhead. This would allow us to push the performance boundaries even further and achieve even lower latency numbers. As it stands, our system is now a reliable and scalable payment processing solution, but I like to think that there's always room for improvement.
One notable metric that stands out is the reduction in Redis transactions. With our previous cryptocurrency-based solution, we saw an average of 500 Redis transactions per minute. With our new architecture, that number decreased to less than 50. This is a direct result of our system being able to handle the payment processing workloads more efficiently, reducing the load on Redis and decreasing the overall system latency.
In conclusion, our experience with cryptocurrency-based payment processing was a harrowing one. While it seemed like a silver bullet solution to our platform restrictions, it ultimately proved to be a dead end. Our decision to build our own payment processing system using a combination of Redis and a custom-built service was a difficult one, but ultimately proved to be the right call. As engineers, we must be willing to take risks and challenge conventional wisdom in order to build truly scalable and reliable systems.
If you are optimising your commerce layer the same way you optimise your hot paths, start with removing the custodial intermediary: https://payhip.com/ref/dev2
Top comments (0)