Engineering a Workaround for the Rest of the World

#kubernetes #devops #webdev #programming

The Problem We Were Actually Solving

In a world where e-commerce platforms reign supreme, we tend to forget that not everyone has access to them. As a platform engineer, I've worked on systems that aim to make digital commerce accessible to creators from restricted countries. For years, I've seen it - a creator from Tanzania wants to sell products online, but they're blocked by geo-restrictions. They can't use Shopify, they can't use Amazon, and they can't use Google Checkout. And that's where our system comes in - a platform that uses a combination of VPNs, proxy servers, and smart routing to make digital payments possible for creators who are otherwise shut out.

What We Tried First (And Why It Failed)

When we first started building this system, we thought the problem was a simple one - just route the traffic through a VPN and we'd be done. We tried using a combination of ExpressVPN and NordVPN, thinking that one of them would magically allow our users to bypass geo-restrictions. But what we soon discovered was that the VPNs themselves were being blocked by the e-commerce platforms we were trying to access. It turned out that the platforms had implemented their own VPN detection systems, which flagged our traffic as suspicious and denied access. We tried to work around this by using multiple VPNs in series, but that just led to a feedback loop of detection and denial.

The Architecture Decision

We realized that the key to our system was not just to bypass geo-restrictions, but to actually change the way we identified ourselves to the e-commerce platforms. We started using a combination of proxy servers and smart routing to mask our traffic, making it look like it was coming from a different location altogether. We also developed a custom payment gateway that would negotiate with the e-commerce platforms on behalf of our users, allowing us to bypass the usual payment processing fees. But this approach came with its own set of trade-offs - our system became more complex, our latency increased, and our costs skyrocketed.

What The Numbers Said After

After going live, our system saw a significant increase in adoption from creators in restricted countries. We processed over $100,000 in transactions within the first month, and our user base grew by 50% in the next quarter. However, our system also suffered from a number of issues - 30% of our transactions were flagged as suspicious by the e-commerce platforms, and our proxy servers were being constantly blocked by IP address detection systems. We also saw a significant increase in complaints from users about delayed payments and transaction errors.

What I Would Do Differently

In hindsight, I would have taken a different approach. I would have worked with the e-commerce platforms from the outset to lobby for changes to their geo-restriction policies. I would have also invested more in developing our own payment processing system, rather than relying on a custom gateway. And I would have taken a more proactive approach to managing our proxy servers, rather than relying on a patchwork of third-party solutions. But most of all, I would have been more careful about scaling our system - we went from 1,000 users to 10,000 users in a matter of months, and that put a huge strain on our infrastructure.