Our young but innovative company created an innovative algorithm to bring, the most nutritious food for your beloved cat or dog, to your doorstep at exactly the time you need it. It might sound like something where no IT is needed, right? It's just bags with dogfood, how hard can it be? Well, this article will take you on a journey through what it entails to work for an e-commerce company, specifically Just Russel, as a technical engineer.
First, we will discuss how well connected the engineering department is as a department in the larger organisation. Secondly, we will go over the risks and problems that might occur in such organisations and which pitfalls you should keep an eye on. Next, we will discuss how our company, Just Russel has tackled all of these issues and we will discuss why it's not just e-commerce. Finally, we will show how even a stable industry as e-commerce, can still be disrupted by innovation, especially on the technical side.
A journey through e-commerce
When people think e-commerce, they instantly think about the Amazon's and Ali-expresses of the world. They think of large warehouses filled with stacks of boxes from floor to ceiling. They think of rows and rows of cubicles filled with customer support. They think of fleets upon fleets of delivery vans and drivers.
All of these are true images and challenges you find in an e-commerce platform. Each of the previously mentioned departments need to be connected to each other. The core holding all of these processes together in an efficient manner is conjured in a lesser known part of e-commerce companies, the engineering department.
Think of your typical consumption at an e-commerce site.
It starts with getting you to the website. Marketing optimises this based on data of previous users, retrieved from the website. This data needs to be collected, stored, queried and aggregated into dashboards that can be consumed. A step further is the automatic proposals for the marketing team based on the previous results. As marketing is a large chunk of every consumer facing product, being more efficient in this regard can quickly increase the margin the business wins on new customers.
As you click on the link you found on google, a new webpage opens up. It loads in the blink of an eye, welcoming you on the shop in a frictionless manner. All of the images pop into the screen seamlessly, and ... The site is in Dutch, my language! Well of course the programmer thought of internationalisation, as most e-commerce companies aren't active in a single country. When looking behind the scenes, the amount of daily webpage loads is in the hundred thousands to million. And you though scaling and optimising a landing page was easy?
You sign-up and enter the application. Here things get even more complex. We are no longer in a static website, no, we are looking at dynamic data specific for me!. Your data, along with that of the other thousands of clients, is powered by a database running relentlessly behind the scenes. Especially in more complex shops, like Just Russel, where we handle subscriptions and our food algorithms are computing the best food for your dog individually, it gets very complex. Well, we wouldn't be programmers if complex doesn't mean a challenge to be taken on.
Now, you decided it's time to buy something from this shop. You go through the food computation and customise your bags. You fill in your address and now it's the moment to pay. The engineers behind the scenes created (and maintained) the internal plumbing to get your payment handled and validated. As you paid the amount, a request is sent to a parcel service such that you, the client living tens of kilometers away, receive the parcel in a few days.
If you think high-level about e-commerce, it isn't rocket science. You buy something, the warehouse picks your order and the delivery people ship it to you. How hard can it be right? I hope I've shown you in the last few paragraphs that it can still be darn hard. All the processes that are taking place, plus, the sheer scale of consumers using the application, creates for all kinds of interesting topics to focus engineering efforts on.
Challenges and risks
After the last chapter, it shouldn't come as a surprise that there are quite some challenges and risks to engineering an e-commerce platform for scale. The "for scale" is very important here as it is indeed quite easy to build a simple platform that handles up to 100 users. Thinking about the scale of thousands of concurrently active users means thinking on a whole other scale. In the following section we will go over a few of these key challenges, how you can spot them before they come an issue and what to focus on to alleviate the consequences they bring.
Integrations
A first challenge that is easy to forget is how many integrations the core e-commerce platform has. Let me name a definitely non-exhaustive list to get the point across:
- Parcel services
- Payment services
- ERP services
- CRM services
- Analytics connections
Each of these services extracts data from the system and uses it for its own purposes. As you can imagine, keeping a reign on this range of different integrations can be very hard. Good systems design and separation of concerns is a necessity.
Grouping your integrations and making them standardised early
on is very important for this as well. Imagine your company wants to switch to a new parcel service. That means another connection to maintain. If you didn't group your interfaces and make them standardised at your end, then you will have a hard time if a new feature needs to be introduced into both integrations. If you just switch, that is not the biggest issue, but imagine you need a new parcel service for the US, while keeping the old one for the EU. This means maintaining both and making sure both keep functioning consistently.
A few tips when working with this many integration:
- Make sure the integrations are standardised. Each type of integration needs to follow only the functions you need from it. Each should be wrapped in a standardised interface.
- Make sure concerns between integrations are neatly separated. Is your parcel service and payment service the same? Nice, but keep them in separate implementations. Looking at the code, you should not be able to know they are the same company delivering this functionality.
- Document, document, document. Where and when you use these interfaces should be documented clearly, as this will increase understanding for both consumers & implementers of the interfaces. In time, this will make change management across the organisation easier.
Top comments (0)